Story
arxiv_cs_ai ยท Nov 13, 2026 ยท paper
arxiv.orgNov 13, 2026
original source linked
In brief
Interactive LLM services must meet latency service-level objectives (SLOs) for the time to first token (TTFT) and the time between tokens (TBT), as streaming and soft real-time applications require predictable token d...