vLLM x TileRT: Specialized Decode for Latency-Critical Serving
vLLM pairs prefill with a separate TileRT decode engine behind one shared serving layer, with no changes needed to existing deployments.
AI Storyline
3 items · 2 sources · 3 days
Operational story trace
Vllm Serving
Follow in this browser to see new updates on your Live feed.
Latest change
Netflix disclosed the internals of its in-house LLM serving platform, built on Triton and vLLM — the thread's first large-scale production-adoption proof point after vLLM's July engine upgrades.
Earlier contextThe story so far
vLLM shipped a TileRT-powered latency-optimized decode engine that pairs with existing prefill behind one shared serving layer, then an Attention-FFN Disaggregation plugin adding GPU and Ascend NPU backends for MoE serving. Both landed as drop-in additions through the same V1 connector interface, with no changes required to already-running deployments.
editor-curated · source-linked
Arc
Jul 14Jul 27 · now
TILERT DECODE · Jul 14vLLM adds a TileRT-powered latency-optimized decode engine1 source · show source ▾
MOE DISAGGREGATION · Jul 23vLLM ships an AFD plugin, disaggregating attention and FFN for MoE serving1 source · show source ▾
Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving
vLLM's AFD plugin disaggregates attention and FFN computation for MoE models, adding GPU and Ascend NPU backend support.
NOW · Jul 27Netflix details its in-house Triton + vLLM serving platform1 source · show source ▾
Netflix Details Its In-House LLM Serving Platform with Triton and vLLM - infoq.com
Netflix goes public with its in-house LLM serving platform built on Triton and vLLM — the thread's first large-scale production-adoption signal.
Day 1 Tuesday, Jul 14, 2026
vLLM x TileRT: Specialized Decode for Latency-Critical Serving
vLLM pairs prefill with a separate TileRT decode engine behind one shared serving layer, with no changes needed to existing deployments.
Day 2 Thursday, Jul 23, 2026
Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving
vLLM's AFD plugin disaggregates attention and FFN computation for MoE models, adding GPU and Ascend NPU backend support.
Day 3 Monday, Jul 27, 2026
Netflix Details Its In-House LLM Serving Platform with Triton and vLLM - infoq.com
Netflix goes public with its in-house LLM serving platform built on Triton and vLLM — the thread's first large-scale production-adoption signal.
What to watch — open questions
- Does Netflix's in-house platform build on vLLM's TileRT decode and AFD disaggregation upgrades, or run a separate internal stack on top of vLLM?
- Does AFD's Ascend NPU backend and TileRT's decode latency gains hold up on independent benchmarks outside vLLM's own posts?
How this thread was built
editor wrote the arc · 3 beats
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.