LLM Digest

Story

arxiv_cs_cl · Nov 9, 2026 · paper

Source brief

BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache Translation

arxiv.orgNov 9, 2026
original source linked

In brief

LLM inference has two phases with different bottlenecks: LLM prefill runs once per request and is compute-intensive. Decoding repeats for every output token and is limited by memory bandwidth. To address the challenge...

Read the original at arxiv.org →Open in live feed Read that day’s brief

BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache Translation

Earlier in this thread 4 items

BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design

RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill

Verify Before You Fetch: Reliable KV Cache Compression for LLM Inference

D-REAL: Acceptance-Length Optimization with Unbiased Gradients for Speculative Decoding