Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding - NVIDIA Developer
NVIDIA Developer publishes the DFlash speculative-decoding technique, claiming up to 15x inference throughput gains on Blackwell GPUs.
AI Storyline
2 items · 2 sources · 2 days
Operational story trace
DFlash speculative decoding on NVIDIA Blackwell
Follow in this browser to see new updates on your Live feed.
Current stateAdopted in production
Latest change
Modal named DFlash as the technique behind its own state-of-the-art inference-latency results, working with Z Lab and SGLang to release open-source pre-trained speculator models — including one for Qwen that improves over multi-token prediction by more than 50%.
Earlier contextThe story so far
Speculative decoding — drafting candidate tokens ahead of the target model to cut serving latency — is a known lever for LLM inference cost. NVIDIA's June 23 developer post introduced DFlash, a speculator built on the target model's own KV projections, claiming up to 15x throughput gains on Blackwell GPUs.
editor-curated · source-linked
Arc
Jun 23Jun 24 · now
LAUNCH · Jun 23NVIDIA introduces DFlash, claiming up to 15x inference gains on Blackwell1 source · scout · show source ▾
ADOPTED · Jun 24Modal ships open-source DFlash speculator models with Z Lab and SGLang1 source · scout · show source ▾
Achieve state-of-the-art inference latencies with speculative decoding
Modal names DFlash as the technique behind its own state-of-the-art latency results, releasing open-source speculator models built with Z Lab and SGLang.
Day 1 Tuesday, Jun 23, 2026
Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding - NVIDIA Developer
NVIDIA Developer publishes the DFlash speculative-decoding technique, claiming up to 15x inference throughput gains on Blackwell GPUs.
Day 2 Wednesday, Jun 24, 2026
Achieve state-of-the-art inference latencies with speculative decoding
Modal names DFlash as the technique behind its own state-of-the-art latency results, releasing open-source speculator models built with Z Lab and SGLang.
What to watch — open questions
- Does DFlash's speedup hold on non-Blackwell GPUs, or is it Blackwell-specific?
- Will inference frameworks beyond SGLang add native DFlash speculator support?
How this thread was built
scout surfaced 2editor wrote the arc · 2 beatswatcher 1 status change
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.