LLM Digest
Subscribe

AI Storyline

5 items · 3 sources · 5 days

View as JSON

Operational story trace

Benchmarking Long-Horizon

Current stateDevelopingstatus changed Oct 26

Latest change

FOCUS (Oct 26) proposes active critic learning to fix credit assignment under sparse outcome rewards, moving the thread from measuring long-horizon agents to training them.

Earlier contextThe story so far

Doc2Harness opened the thread on Oct 14 by measuring how much the harness around a model, not just the model, drives long-horizon coding-agent results. Over the next week, papers added search-state learning (ClueFlow), error-recovery tests for computer-use agents (CUA-Recovery) and a continual-learning exam.

editor-curated · source-linked

Arc

Oct 14Oct 26 · now
BASELINE · Oct 14
Doc2Harness benchmarks harness engineering for coding agents
1 source · show source ▾
WIDENING · Oct 19–20
Search state and computer-use error recovery get their own tests
2 sources · show sources ▾
NOW · Oct 25–26
Continual learning and critic training extend the focus to improvement over time
2 sources · show sources ▾

What to watch — open questions

  • Does any of these benchmarks get adopted by a major lab or independent evaluator?
  • Do the reported FOCUS gains hold on a third-party long-horizon benchmark?
  • Will CUA-Recovery results show shipped computer-use agents detecting silent state corruption?
How this thread was built
editor wrote the arc · 3 beatswatcher 1 status change

Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.