Story
arxiv_cs_lg ยท Nov 18, 2026 ยท paper
arxiv.orgNov 18, 2026
original source linked
In brief
A fixed dispatch rate for LLM inference becomes stale as serving fleets resize and request sizes change. Generative output length is unknown at admission, and shared provider pools can lose capacity. We present Cap-an...