Story
arxiv_cs_ai · Nov 10, 2026 · paper
Source brief
Precision on Demand: Escalating Weight Bits at Uncertain Decoding Steps in a Small Local Model
arxiv.orgNov 10, 2026
original source linked
In brief
Local decoding usually fixes one weight precision for the whole generation. We study a different allocation: store 8-bit weights, decode with their top 4 bits, and redo a step at 8 bits only when the low-precision output distribution looks uncertain (entropy above a threshold or