Story
arxiv_cs_cl ยท Nov 13, 2026 ยท paper
Source brief
Folding Language Models into Themselves
arxiv.orgNov 13, 2026
original source linked
In brief
Knowledge distillation usually trains a compact student from scratch, which needs a large amount of data even though the teacher's weights already encode most of what the student has to learn. In this work, we explore...