LLM Digest
Subscribe

Story

arxiv_cs_cl · Nov 17, 2026 · paper

Source brief

TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates

arxiv.orgNov 17, 2026
original source linked

In brief

Hyper-Connections (HC) expand the residual stream of a Transformer into $N$ parallel streams, and manifold-constrained HC (mHC) stabilizes them at scale. However, mHC benefits little from more than four streams, partly because each sublayer writes a single output to all streams,

Continue reading

Read the original at arxiv.org →Open in live feedRead that day’s brief

Earlier in this thread 4 items