Story

arxiv_cs_lg ยท Nov 13, 2026 ยท paper

Source brief

Callo: Serving Cache-Miss Requests with Speculative Decoding on Invalidated KV under Prefill-Decode Disaggregation

arxiv.orgNov 13, 2026
original source linked

In brief

Agent sessions may involve frequent context edits, which effectively invalidate cached KV state and force requests to wait for re-prefill before decoding. We present Callo, a disaggregated serving system that leverage...

Continues in

Speculative Decoding โ€” open the evidence trace โ†’

Feed lens
agenteval

Continue reading

Read the original at arxiv.org โ†’Open in live feedRead that dayโ€™s brief

Earlier in this thread 4 items