Story

arxiv_cs_cl ยท Nov 19, 2026 ยท paper

Source brief

HiSA: Sparsifying the Hidden Dimension of RoPE Attention in LLMs

arxiv.orgNov 19, 2026
original source linked

In brief

Attention in large language models (LLMs) scores every query against every key with a full dot product. We investigate whether this dot product must be computed completely for attention to be effective. We propose Hid...

Continue reading

Read the original at arxiv.org โ†’Open in live feedRead that dayโ€™s brief

Earlier in this thread 4 items