Story
arxiv_cs_ai · Oct 27, 2026 · paper
Source brief
What Improves During Pretraining? Tracking Knowledge Acquisition with Point-in-Time Models
arxiv.orgOct 27, 2026
original source linked
In brief
Language models pretrained on web corpora collected over multiple years learn both general and domain-specific knowledge. How these different kinds of knowledge are learned as pretraining progresses remains unclear. R...
Feed lens
eval