Story
arxiv_cs_lg · Nov 18, 2026 · paper
arxiv.orgNov 18, 2026
original source linked
In brief
Muon updates each hidden weight matrix along the polar factor of its momentum, the steepest-descent direction under the spectral norm. That choice fixes two allocations without consulting the loss: every singular direction of the momentum receives the same step, and in a non-squa