Story
arxiv_cs_cl · Nov 12, 2026 · paper
Source brief
LazyWeights: Decode-on-Demand Edge LLMs
arxiv.orgNov 12, 2026
original source linked
In brief
Memory decides which LLMs an edge device runs, and how fast: capacity on a $4$ GB Jetson Nano, bandwidth on an $8$ GB Jetson Orin Nano Super. Deployed formats (GPTQ, AWQ, $\texttt{llama.cpp}$'s K- and I-quants) spend equal bits on every integer code of a matrix, although the code