{"slug":"dflash-speculative-decoding-on-nvidia-blackwell","label":"DFlash speculative decoding on NVIDIA Blackwell","item_count":2,"day_count":2,"source_count":2,"first_seen":"2026-06-23T15:14:05+00:00","last_updated":"2026-06-24T00:00:00+00:00","via_scout":true,"generated_at":"2026-07-12T05:05:17.899001+00:00","sources":["modal_blog","search_llm_ops_news"],"days":[{"date":"2026-06-23","items":[{"title":"Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding - NVIDIA Developer","url":"https://news.google.com/rss/articles/CBMixAFBVV95cUxOZzFLRVlRQk80eUVvMEFHOXhvYjBhbmRRdTFFdFFPcmJ1cVZ2aW5wc3J1RkxqLXpvUEgzaUlyblp2amNNdGR0VzhuZ3pjay1mUW1ZNGdaX1BQZjVhdnp5Qjh3M3Q0amhoUUZJaUNpOF9NcjRIZmw2ckFydUVQZDlHekYzSXdIZ0NRaDN6aGVJdWwtYl9FUGp2WDB0Q0F3YnlENi1YODJYdFNWTmg2LUZwSWFIOVhDT3JPZ2x1bDNXUnBWMkdk?oc=5","source":"search_llm_ops_news","type":"news","summary_1line":"Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding NVIDIA Developer","sid":"99bd515fd5fd8083","published":"2026-06-23T15:14:05+00:00","editor_note":"NVIDIA Developer publishes the DFlash speculative-decoding technique, claiming up to 15x inference throughput gains on Blackwell GPUs."}]},{"date":"2026-06-24","items":[{"title":"Achieve state-of-the-art inference latencies with speculative decoding","url":"https://modal.com/blog/achieve-sota-specdec","source":"modal_blog","type":"news","summary_1line":"How Modal and Decagon worked together to cut inference latency - and you can too.","sid":"62173e9d865bdec2","published":"2026-06-24T00:00:00+00:00","editor_note":"Modal names DFlash as the technique behind its own state-of-the-art latency results, releasing open-source speculator models built with Z Lab and SGLang."}]}],"editorial":{"tldr":"Speculative decoding — drafting candidate tokens ahead of the target model to cut serving latency — is a known lever for LLM inference cost. NVIDIA's June 23 developer post introduced DFlash, a speculator built on the target model's own KV projections, claiming up to 15x throughput gains on Blackwell GPUs.","stale":false,"whats_new":"Modal named DFlash as the technique behind its own state-of-the-art inference-latency results, working with Z Lab and SGLang to release open-source pre-trained speculator models — including one for Qwen that improves over multi-token prediction by more than 50%.","why_it_matters":"If you're running inference on Blackwell GPUs with SGLang, this is now an off-the-shelf latency lever: Modal reports that Blackwell plus SGLang plus a low-overhead regional deployment can match or beat proprietary inference providers without custom infra.","take_for_builders":"Running inference on Blackwell with SGLang? Evaluate the open-source DFlash speculator models before building custom speculative decoding — Modal reports it matching proprietary inference-provider latency without proprietary infra.","status":{"state":"Adopted in production","tone":"rising","changed":"2026-06-24","detail":"Modal confirms the same DFlash technique in its own latency benchmarks and ships open-source pre-trained speculator models built with Z Lab and SGLang."},"beats":[{"kicker":"LAUNCH","tone":"launch","headline":"NVIDIA introduces DFlash, claiming up to 15x inference gains on Blackwell","summary":"DFlash generates draft tokens in parallel using the target model's own KV projections, invented by Jian Chen and collaborators at Z Lab.","sids":["99bd515fd5fd8083"]},{"kicker":"ADOPTED","tone":"rising","headline":"Modal ships open-source DFlash speculator models with Z Lab and SGLang","summary":"Modal reports DFlash as its best-performing speculative-decoding technique, with a Qwen speculator improving over multi-token prediction by 50%+.","sids":["62173e9d865bdec2"]}],"open_questions":["Does DFlash's speedup hold on non-Blackwell GPUs, or is it Blackwell-specific?","Will inference frameworks beyond SGLang add native DFlash speculator support?"],"provenance":{"99bd515fd5fd8083":{"surfaced_by":"scout"},"62173e9d865bdec2":{"surfaced_by":"scout"}},"generated_at":"2026-07-12T05:04:38Z"}}