EAGLE-3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark
vLLM's own blog: AMD Quark-trained EAGLE-3 drafts cut serving latency up to 2x for Kimi-K2.5 on AMD Instinct GPUs.
AI Storyline
4 items · 2 sources · 4 days
Operational story trace
Serving Vllm
Follow in this browser to see new updates on your Live feed.
Latest change
Netflix disclosed the internals of its in-house LLM serving platform, built on Triton and vLLM — the thread's first large-scale production-adoption proof point after three straight vLLM engine upgrades this month.
Earlier contextThe story so far
Across three July posts, vLLM's own blog shipped AMD-backed EAGLE-3 speculative decoding, a TileRT-powered latency-optimized decode path, and an Attention-FFN disaggregation plugin for MoE serving — each landing through the same V1 connector interface without disrupting existing deployments.
editor-curated · source-linked
Arc
Jul 13Jul 27 · now
SPECULATIVE DECODING · Jul 13vLLM ships EAGLE-3 speculative decoding for AMD Instinct GPUs1 source · show source ▾
TILERT DECODE · Jul 14vLLM adds a TileRT-powered latency-optimized decode engine1 source · show source ▾
vLLM x TileRT: Specialized Decode for Latency-Critical Serving
vLLM pairs prefill with a separate TileRT decode engine behind one shared serving layer, with no changes needed to existing deployments.
MOE DISAGGREGATION · Jul 23vLLM ships an AFD plugin, disaggregating attention and FFN for MoE serving1 source · show source ▾
Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving
vLLM's AFD plugin disaggregates attention and FFN computation for MoE models, adding GPU and Ascend NPU backend support.
NOW · Jul 27Netflix details its in-house Triton + vLLM serving platform1 source · show source ▾
Netflix Details Its In-House LLM Serving Platform with Triton and vLLM - infoq.com
Netflix goes public with its in-house LLM serving platform built on Triton and vLLM — the thread's first large-scale production-adoption signal.
Day 1 Monday, Jul 13, 2026
EAGLE-3 Speculative Decoding on AMD Instinct GPUs: Training and Serving with vLLM and AMD Quark
vLLM's own blog: AMD Quark-trained EAGLE-3 drafts cut serving latency up to 2x for Kimi-K2.5 on AMD Instinct GPUs.
Day 2 Tuesday, Jul 14, 2026
vLLM x TileRT: Specialized Decode for Latency-Critical Serving
vLLM pairs prefill with a separate TileRT decode engine behind one shared serving layer, with no changes needed to existing deployments.
Day 3 Thursday, Jul 23, 2026
Announcing vLLM AFD Plugin: Disaggregating Attention and FFN for Flexible MoE Serving
vLLM's AFD plugin disaggregates attention and FFN computation for MoE models, adding GPU and Ascend NPU backend support.
Day 4 Monday, Jul 27, 2026
Netflix Details Its In-House LLM Serving Platform with Triton and vLLM - infoq.com
Netflix goes public with its in-house LLM serving platform built on Triton and vLLM — the thread's first large-scale production-adoption signal.
What to watch — open questions
- Does Netflix's in-house platform use any of the three July engine upgrades (EAGLE-3, TileRT, AFD), or run a separate internal stack on top of vLLM?
- Do the EAGLE-3 AMD throughput gains and AFD's Ascend NPU backend hold up on independent benchmarks outside vLLM's own posts?
How this thread was built
editor wrote the arc · 4 beats
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.