KLEAR: Knowledge-Matched LLM Evaluation in Korean across Application Requirements via Controlled Benchmark Transformation
Language-specific case: rewrites Korean benchmarks so a wrong answer can be traced to missing knowledge.
3 items · 3 sources · 2 days
Operational story trace
Follow in this browser to see new updates on your Live feed.
Latest change
DECODE (Nov 13) brings controlled evaluation to sequential decision-making, using decomposable environments to test which change to a system actually improves its decisions.
This is a research trend, not one developing event: three unrelated groups published benchmarks that change one variable at a time so a failure can be traced to a cause. On Nov 6, KLEAR did this for Korean LLM knowledge questions and AllenAI's ToyBlocks did it for robot learning in a simulated block world.
Arc
Language-specific case: rewrites Korean benchmarks so a wrong answer can be traced to missing knowledge.
Embodied case: AllenAI's controlled block world for robot learning and evaluation.
Extends the approach to sequential decisions, testing which system change helps.
Language-specific case: rewrites Korean benchmarks so a wrong answer can be traced to missing knowledge.
Embodied case: AllenAI's controlled block world for robot learning and evaluation.
Extends the approach to sequential decisions, testing which system change helps.
What to watch — open questions
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.