CalibraRL: Confidence-Calibrated Reinforcement Learning for Mitigating Hallucinations in RAG
Opens the reward-design strand: RL that rewards correct answers, penalizes wrong ones and allows abstention, aimed at RAG hallucinations.
AI Storyline
27 items · 5 sources · 13 days
Operational story trace
Reinforcement Learning
Follow in this browser to see new updates on your Live feed.
Latest change
A Nov 12 preprint targets batch makespan for agent workloads, RL rollouts included, by coordinating LLM inference with environment execution, the third rollout-scheduling paper in this thread since Oct 27.
Earlier contextThe story so far
This thread is a topical grouping of 27 arXiv preprints since Oct 26, not one event: what they share is reinforcement learning as a post-training method for LLMs and agents. Most of them work on one of three problems: what the reward should measure, how to train multi-turn tool-using agents, and how to pay for the rollouts.
editor-curated · source-linked
Arc
Oct 26Nov 11 · now
REWARD SIGNAL · Oct 26 – Nov 10Outcome-only rewards draw six papers on calibration, credit and guidance6 sources · show sources ▾
When Fine-Grained Credit Assignment Helps and Hurts: Multimodal Agentic Reinforcement Learning through Video Editing
Asks when fine-grained credit assignment helps a multimodal tool-using agent and when it does not, using video editing as the test task.
Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Uses RL to push models toward coupled knowledge acquisition and reasoning in unfamiliar domains, beyond retrieval and aggregation.
SCARL: Sequence Confidence-Aligned Reinforcement Learning
Names the calibration gap in RLVR: the verifier rewards passing answers, but nothing ties the policy's confidence to correctness.
Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning
Addresses on-policy RL stalling when the base model cannot reach the reward, using guidance drawn from textual reward sources such as judge models.
AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models
Adds AI-chosen training-time hints to RLVR, on the argument that an outcome reward reduces each attempt to a single number.
AGENT TRAINING · Oct 26 – Nov 9RL moves onto the agent stack: harnesses, skills, memory and sub-agents10 sources · show sources ▾
MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors
Applies error-targeted learning to terminal agents: find specific misbehaviors, train against them, check that they are fixed.
CoHARL: Co-Evolving Agent Policies and Executable Harnesses with Reinforcement Learning
Argues a harness tuned for a fixed policy goes stale as the policy learns, and trains the two together.
TalentRL: Strategy-Guided Reinforcement Learning for Hierarchical Skill Routing
Replaces flat retrieval from a skill bank with routing that models the high-level strategy first.
EffComp: Efficient Prompt Compression via Hybrid Reinforcement & Supervised Learning
Trains a prompt compressor with a mix of RL and supervised learning, aimed at cheaper and faster LLM calls.
Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning
Uses RL to close the gap between HTML that renders and HTML that works and looks right under real interaction.
JHiMA-WebRL: A Joint Hierarchical Multi-Agent Reinforcement Learning Framework for Long-Horizon Web Tasks
Hierarchical multi-agent RL for long-horizon, multi-step tasks in GUI web agents.
MDSKILL: Learning to Internalize Skill Memory through Multi-View Distillation for Efficient Dynamic Agent Self-Evolution
Moves skill memory out of the prompt and into the model by combining RL with on-policy self-distillation.
Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning
Multi-agent RL for sub-agents that explore in parallel with isolated context windows.
What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
The thread's diagnostic paper: multi-turn RL gains are highly sensitive to sampling breadth and interaction depth.
MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning
Trains incremental summarization with RL so bounded agent memory is optimized for downstream task utility.
ROLLOUT COST · Oct 27 – Nov 12Three papers treat rollout scheduling as the bottleneck3 sources · show sources ▾
EnvSched: Environment-Aware Scheduling for Rollout-Efficient Agent Reinforcement Learning
First rollout-cost paper in the thread: environment-aware scheduling under a finite rollout budget.
Adaptive Scheduling Pipeline for Multi-Instance Asynchronous Reinforcement Learning
Second scheduling paper: an offline rollout scheduler for multi-instance asynchronous RL of LLMs that targets rollout underutilization such as padding from uneven lengths.
Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination
Latest item: coordinates LLM inference with environment execution to cut batch makespan for evals, synthetic data and RL rollouts.
ADJACENT · Oct 30 – Nov 11Eight clustered papers sit outside LLM agent post-training8 sources · show sources ▾
Optimizing Stochastic Policy Geometry for Few-Step Flow Reinforcement Learning
Off the agent track: how to inject stochasticity so GRPO can train few-step flow models.
CausalXRL: Explainable Reinforcement Learning through Causal Graph Reasoning
Classical RL interpretability: explains agent policies through causal graph reasoning.
An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems
Theory entry: an axiomatic basis for RL when only a subset of agents is controllable, with smart grids as the motivating domain.
FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning
Diffusion counterpart to the rollout-cost theme: RL fine-tuning of few-step models, where many-step rollouts otherwise dominate training cost.
Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning
Model-based RL entry on encoding memory in world models at test time.
Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation
A benchmark with no RL method: task coordination in full-duplex voice agents, where users interrupt and redirect mid-speech.
Unprompted coordination through shared memory in LLM agents: a locked two-implementation replication and extension
A replication study with no RL method: whether LLM agents coordinate through shared memory without being told to.
ReShap: Interpretable Reinforcement Learning for High-Dimensional Multi-Action Settings
Classical RL interpretability for high-dimensional multi-action policies.
Day 1 Monday, Oct 26, 2026
CalibraRL: Confidence-Calibrated Reinforcement Learning for Mitigating Hallucinations in RAG
Opens the reward-design strand: RL that rewards correct answers, penalizes wrong ones and allows abstention, aimed at RAG hallucinations.
When Fine-Grained Credit Assignment Helps and Hurts: Multimodal Agentic Reinforcement Learning through Video Editing
Asks when fine-grained credit assignment helps a multimodal tool-using agent and when it does not, using video editing as the test task.
MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors
Applies error-targeted learning to terminal agents: find specific misbehaviors, train against them, check that they are fixed.
Day 2 Tuesday, Oct 27, 2026
CoHARL: Co-Evolving Agent Policies and Executable Harnesses with Reinforcement Learning
Argues a harness tuned for a fixed policy goes stale as the policy learns, and trains the two together.
EnvSched: Environment-Aware Scheduling for Rollout-Efficient Agent Reinforcement Learning
First rollout-cost paper in the thread: environment-aware scheduling under a finite rollout budget.
Day 3 Thursday, Oct 29, 2026
Incentivizing Self-Learning in Large Language Models via Reinforcement Learning
Uses RL to push models toward coupled knowledge acquisition and reasoning in unfamiliar domains, beyond retrieval and aggregation.
SCARL: Sequence Confidence-Aligned Reinforcement Learning
Names the calibration gap in RLVR: the verifier rewards passing answers, but nothing ties the policy's confidence to correctness.
Day 4 Friday, Oct 30, 2026
TalentRL: Strategy-Guided Reinforcement Learning for Hierarchical Skill Routing
Replaces flat retrieval from a skill bank with routing that models the high-level strategy first.
Optimizing Stochastic Policy Geometry for Few-Step Flow Reinforcement Learning
Off the agent track: how to inject stochasticity so GRPO can train few-step flow models.
CausalXRL: Explainable Reinforcement Learning through Causal Graph Reasoning
Classical RL interpretability: explains agent policies through causal graph reasoning.
Day 5 Monday, Nov 2, 2026
EffComp: Efficient Prompt Compression via Hybrid Reinforcement & Supervised Learning
Trains a prompt compressor with a mix of RL and supervised learning, aimed at cheaper and faster LLM calls.
An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems
Theory entry: an axiomatic basis for RL when only a subset of agents is controllable, with smart grids as the motivating domain.
Day 6 Tuesday, Nov 3, 2026
Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning
Uses RL to close the gap between HTML that renders and HTML that works and looks right under real interaction.
JHiMA-WebRL: A Joint Hierarchical Multi-Agent Reinforcement Learning Framework for Long-Horizon Web Tasks
Hierarchical multi-agent RL for long-horizon, multi-step tasks in GUI web agents.
Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning
Addresses on-policy RL stalling when the base model cannot reach the reward, using guidance drawn from textual reward sources such as judge models.
MDSKILL: Learning to Internalize Skill Memory through Multi-View Distillation for Efficient Dynamic Agent Self-Evolution
Moves skill memory out of the prompt and into the model by combining RL with on-policy self-distillation.
Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning
Multi-agent RL for sub-agents that explore in parallel with isolated context windows.
Day 7 Wednesday, Nov 4, 2026
What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?
The thread's diagnostic paper: multi-turn RL gains are highly sensitive to sampling breadth and interaction depth.
Day 8 Thursday, Nov 5, 2026
FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning
Diffusion counterpart to the rollout-cost theme: RL fine-tuning of few-step models, where many-step rollouts otherwise dominate training cost.
Day 9 Friday, Nov 6, 2026
Adaptive Scheduling Pipeline for Multi-Instance Asynchronous Reinforcement Learning
Second scheduling paper: an offline rollout scheduler for multi-instance asynchronous RL of LLMs that targets rollout underutilization such as padding from uneven lengths.
Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning
Model-based RL entry on encoding memory in world models at test time.
Day 10 Monday, Nov 9, 2026
Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation
A benchmark with no RL method: task coordination in full-duplex voice agents, where users interrupt and redirect mid-speech.
Unprompted coordination through shared memory in LLM agents: a locked two-implementation replication and extension
A replication study with no RL method: whether LLM agents coordinate through shared memory without being told to.
MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning
Trains incremental summarization with RL so bounded agent memory is optimized for downstream task utility.
Day 11 Tuesday, Nov 10, 2026
AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models
Adds AI-chosen training-time hints to RLVR, on the argument that an outcome reward reduces each attempt to a single number.
Day 12 Wednesday, Nov 11, 2026
ReShap: Interpretable Reinforcement Learning for High-Dimensional Multi-Action Settings
Classical RL interpretability for high-dimensional multi-action policies.
Day 13 Thursday, Nov 12, 2026
Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination
Latest item: coordinates LLM inference with environment execution to cut batch makespan for evals, synthetic data and RL rollouts.
What to watch — open questions
- Do any of the three rollout-scheduling approaches ship as open-source code or get adopted in a public training stack?
- Does an independent group reproduce the finding that multi-turn RL gains depend on sampling breadth and interaction depth?
- Do the confidence-calibration methods for RLVR hold on agent tasks, beyond the RAG and reasoning settings they were proposed for?
- Does co-evolving the harness with the policy beat tuning a harness for a fixed policy on a shared public benchmark?
How this thread was built
editor wrote the arc · 4 beats
Storylines are threaded mechanically from the feed: stories that share a distinctive anchor across multiple days and sources. Each item links to its original source. The evidence trace, current state, and open questions are written by the editor routine and refreshed whenever a new beat lands.