{"slug":"speculative-decoding","label":"Speculative Decoding","item_count":3,"day_count":3,"source_count":3,"first_seen":"2026-08-13T17:43:44+00:00","last_updated":"2026-08-26T16:50:02+00:00","generated_at":"2026-09-01T20:11:18.457385+00:00","sources":["arxiv_cs_ai","arxiv_cs_lg","vllm_blog"],"days":[{"date":"2026-08-13","items":[{"title":"DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees","url":"http://arxiv.org/abs/2608.13524v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel. Diffusion-based drafters further reduce proposal latency by predicting an entire token block i...","sid":"80e7ec208d50f270","published":"2026-08-13T17:43:44+00:00","editor_note":"DARTree proposes diffusion-based draft trees that predict a whole token block per step instead of one token at a time."}]},{"date":"2026-08-23","items":[{"title":"Exploring Speculative Decoding in vLLM on AMD GPUs","url":"https://vllm.ai/blog/2026-08-23-speculative-decoding-amd-gpus","source":"vllm_blog","type":"news","summary_1line":"A practical guide to speculative decoding in vLLM on AMD GPUs, covering draft-and-verify mechanics, MTP, EAGLE-3, DFlash, DSpark, configuration, tuning, and benchmark results.","sid":"b4fa4d7778a6247d","published":"2026-08-23T00:00:00+00:00","editor_note":"vLLM publishes a practical speculative-decoding tuning guide for AMD GPUs covering MTP, EAGLE-3, DFlash, and DSpark."}]},{"date":"2026-08-26","items":[{"title":"AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs","url":"http://arxiv.org/abs/2608.26004v1","source":"arxiv_cs_ai","type":"paper","summary_1line":"Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task a...","why_it_matters":"Matches feed focus: agentic, eval.","sid":"aad81dd5a952ad5d","published":"2026-08-26T16:50:02+00:00","editor_note":"AsymSpec proposes context-asymmetric speculative decoding aimed specifically at agentic LLM pipelines' inflated context cost."}]}],"editorial":{"tldr":"Speculative decoding — verifying multiple draft tokens per step to cut inference latency — is now developing on two separate fronts: new drafting research and production serving guides. A mid-August paper (DARTree) pushed the research side toward diffusion-based draft trees that predict whole token blocks instead of single tokens.","stale":false,"whats_new":"A new paper (AsymSpec, Aug 26) targets speculative decoding specifically at agentic pipelines, arguing that generic techniques don't address the latency cost of long, tool-call-and-retrieval-inflated context that agents accumulate.","why_it_matters":"If your agent pipeline is context-heavy (tool loops, retrieval, multi-turn memory), the generic speculative-decoding tuning already documented for production (vLLM/EAGLE-3/DFlash/DSpark) may not capture agent-specific savings — a context-aware variant like AsymSpec is what to watch for next.","take_for_builders":"Don't assume the generic speculative-decoding tuning already documented for vLLM (EAGLE-3/DFlash/DSpark) captures agent-specific savings — if your pipeline's context keeps growing from tool calls and retrieval, watch for a runnable implementation of context-asymmetric techniques like AsymSpec before investing engineering time.","beats":[{"kicker":"DRAFTING RESEARCH","tone":"rising","headline":"DARTree proposes diffusion-based draft trees to cut proposal latency further","summary":"A diffusion drafter predicts an entire token block at once instead of one token at a time, aiming to reduce the latency of the draft step itself.","sids":["80e7ec208d50f270"]},{"kicker":"PRODUCTION TUNING","tone":"now","headline":"vLLM publishes a hands-on speculative-decoding tuning guide for AMD GPUs","summary":"The guide covers draft-and-verify mechanics and configuration across MTP, EAGLE-3, DFlash, and DSpark, with benchmark results — a practical reference for teams already running vLLM.","sids":["b4fa4d7778a6247d"]},{"kicker":"AGENTIC SPECIALIZATION","tone":"turn","headline":"AsymSpec argues agentic LLM pipelines need context-asymmetric speculative decoding","summary":"Long accumulated context from retrieval, tool use, and multi-turn interaction inflates inference cost in ways generic speculative decoding and context-compression tricks don't fully address, the paper argues.","sids":["aad81dd5a952ad5d"]}],"open_questions":["Does AsymSpec's context-asymmetric approach get benchmarked against production serving stacks (vLLM/EAGLE-3/DFlash) rather than only academic baselines?","Is there a reference implementation of AsymSpec that agent-pipeline builders can actually try, or is it paper-only for now?","How much of DARTree's diffusion-drafting latency win holds up outside its own benchmark suite?"],"generated_at":"2026-08-27T05:20:00+00:00"}}