Story
arxiv_cs_lg ยท Oct 27, 2026 ยท paper
arxiv.orgOct 27, 2026
original source linked
In brief
Mixture-of-Experts (MoE) models are bottlenecked by loading routed-expert weights from memory during interactive decoding at small batch sizes. Because most intermediate channels in a routed expert contribute little t...