Story
arxiv_cs_cl · Nov 17, 2026 · paper
arxiv.orgNov 17, 2026
original source linked
In brief
Hyper-Connections (HC) expand the residual stream of a Transformer into $N$ parallel streams, and manifold-constrained HC (mHC) stabilizes them at scale. However, mHC benefits little from more than four streams, partly because each sublayer writes a single output to all streams,