Story
arxiv_cs_ai · Nov 6, 2026 · paper
Source brief
FOSCAR: Accelerating Decode Attention over 2- and 4-Bit KV Caches through Affine Folding
arxiv.orgNov 6, 2026
original source linked
In brief
Large-language-model decode attention streams the key-value (KV) cache for every generated token. Packing that cache into 2-bit or 4-bit quantized integers reduces memory traffic but does not automatically translate i...