LLM Digest

Story

arxiv_cs_lg · Nov 4, 2026 · paper

Source brief

Systolic Linear Attention: Co-Designing Tile-Grouped Gated Delta Attention for TPUs

arxiv.orgNov 4, 2026
original source linked

In brief

Gated-delta linear attention layers such as Gated DeltaNet (GDN), Kimi Delta Attention (KDA), and GDN-2 are tuned for GPUs, where small tensor-core tiles and fast vector registers make per-channel decay rescaling and sequential in-block solves cheap. On Google TPUs, which concent

Read the original at arxiv.org →Open in live feed Read that day’s brief

Systolic Linear Attention: Co-Designing Tile-Grouped Gated Delta Attention for TPUs

Earlier in this thread 4 items

DAMI: Dual-Axis Memory Initialization for Enhanced Long-Context Retrieval in Gated Linear Attention

ARLoop: Fully-Open Pre-trained Looped Transformers with Attention Residuals

Coarse Semantic Blocks for Efficient Training-Free Sparse Attention

Google and Meta Open-Source TorchTPU to Bring Native PyTorch Support to Google TPUs