{"slug":"reinforcement-learning-2026-10-26","label":"Reinforcement Learning","item_count":27,"day_count":13,"source_count":5,"first_seen":"2026-10-26T10:11:24+00:00","last_updated":"2026-11-12T11:38:21+00:00","generated_at":"2026-11-14T00:13:06.733465+00:00","sources":["arxiv_agent_systems_research","arxiv_cs_ai","arxiv_cs_cl","arxiv_cs_lg","arxiv_llm_reliability"],"days":[{"date":"2026-10-26","items":[{"title":"CalibraRL: Confidence-Calibrated Reinforcement Learning for Mitigating Hallucinations in RAG","url":"http://arxiv.org/abs/2610.35142v1","source":"arxiv_llm_reliability","type":"paper","summary_1line":"Large language models (LLMs) often produce plausible but incorrect answers, motivating reinforcement learning methods that reward correct answers, penalize incorrect answers, and allow abstention. However, fixed outco...","why_it_matters":"Matches feed focus: eval.","sid":"c3af4f37e1c19b19","published":"2026-10-26T10:11:24+00:00","editor_note":"Opens the reward-design strand: RL that rewards correct answers, penalizes wrong ones and allows abstention, aimed at RAG hallucinations."},{"title":"When Fine-Grained Credit Assignment Helps and Hurts: Multimodal Agentic Reinforcement Learning through Video Editing","url":"http://arxiv.org/abs/2610.35497v1","source":"arxiv_agent_systems_research","type":"paper","summary_1line":"Reinforcement learning (RL) has shown promise for training multimodal agents to reason and use tools. However, it remains unclear when fine-grained credit assignment improves learning and when it fails to improve task...","why_it_matters":"Matches feed focus: agentic.","sid":"868490bacebdc880","published":"2026-10-26T14:25:40+00:00","editor_note":"Asks when fine-grained credit assignment helps a multimodal tool-using agent and when it does not, using video editing as the test task."},{"title":"MiRL: Error-Targeted Reinforcement Learning for Correcting Terminal Agent Misbehaviors","url":"http://arxiv.org/abs/2610.35697v1","source":"arxiv_cs_cl","type":"paper","summary_1line":"Error-targeted learning gives a legible way to improve a model: identify specific weaknesses of the model, train the model to fix these weaknesses, and check whether the weaknesses are fixed. We study how to apply err...","why_it_matters":"Matches feed focus: agent, eval.","sid":"c07d00b3f0a46112","published":"2026-10-26T16:25:05+00:00","editor_note":"Applies error-targeted learning to terminal agents: find specific misbehaviors, train against them, check that they are fixed."}]},{"date":"2026-10-27","items":[{"title":"CoHARL: Co-Evolving Agent Policies and Executable Harnesses with Reinforcement Learning","url":"http://arxiv.org/abs/2610.36469v1","source":"arxiv_agent_systems_research","type":"paper","summary_1line":"Large language model (LLM) agents interact with their environments through executable harnesses that govern tool use and feedback. Optimizing a harness for a fixed policy overlooks how its behavior changes during lear...","why_it_matters":"Matches feed focus: agent, harness.","sid":"d448a8af2a653453","published":"2026-10-27T01:25:26+00:00","editor_note":"Argues a harness tuned for a fixed policy goes stale as the policy learns, and trains the two together."},{"title":"EnvSched: Environment-Aware Scheduling for Rollout-Efficient Agent Reinforcement Learning","url":"http://arxiv.org/abs/2610.37609v1","source":"arxiv_cs_cl","type":"paper","summary_1line":"As executable environments for language agents scale, generating and verifying their rollouts becomes increasingly expensive. Under finite rollout budgets, this raises an important question: can data management at the...","why_it_matters":"Matches feed focus: agentic, eval.","sid":"02ad246823185a44","published":"2026-10-27T17:38:52+00:00","editor_note":"First rollout-cost paper in the thread: environment-aware scheduling under a finite rollout budget."}]},{"date":"2026-10-29","items":[{"title":"Incentivizing Self-Learning in Large Language Models via Reinforcement Learning","url":"http://arxiv.org/abs/2610.39671v1","source":"arxiv_llm_reliability","type":"paper","summary_1line":"Large language models (LLMs) draw on parametric knowledge and retrieval-augmented generation (RAG) to solve knowledge-intensive tasks. Yet problem solving in unfamiliar domains calls for tightly coupled knowledge acquisition and reasoning, beyond factual retrieval and aggregation","sid":"d571e973b6e6916d","published":"2026-10-29T09:21:26+00:00","editor_note":"Uses RL to push models toward coupled knowledge acquisition and reasoning in unfamiliar domains, beyond retrieval and aggregation."},{"title":"SCARL: Sequence Confidence-Aligned Reinforcement Learning","url":"http://arxiv.org/abs/2610.40407v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning by scoring sampled responses with a rule-based verifier and reinforcing those that pass. Where a classifier's training couples predicted probability to labels, nothing ties an RLVR policy's co","sid":"9471fde9e20a1e58","published":"2026-10-29T16:38:11+00:00","editor_note":"Names the calibration gap in RLVR: the verifier rewards passing answers, but nothing ties the policy's confidence to correctness."}]},{"date":"2026-10-30","items":[{"title":"TalentRL: Strategy-Guided Reinforcement Learning for Hierarchical Skill Routing","url":"http://arxiv.org/abs/2610.41143v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Natural-language skills enable large language model (LLM) agents to reuse procedures across tasks, yet most systems retrieve individual skills from a flat SkillBank without explicitly modeling the high-level strategy...","why_it_matters":"Matches feed focus: agentic, eval.","sid":"29493d021878283c","published":"2026-10-30T02:03:55+00:00","editor_note":"Replaces flat retrieval from a skill bank with routing that models the high-level strategy first."},{"title":"Optimizing Stochastic Policy Geometry for Few-Step Flow Reinforcement Learning","url":"http://arxiv.org/abs/2610.41949v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Few-step flow models generate samples through a handful of large denoising transitions. Applying GRPO to such models requires introducing stochasticity into the denoising transitions, and when a single transition span...","sid":"65dc6916d3b352f1","published":"2026-10-30T16:05:17+00:00","editor_note":"Off the agent track: how to inject stochasticity so GRPO can train few-step flow models."},{"title":"CausalXRL: Explainable Reinforcement Learning through Causal Graph Reasoning","url":"http://arxiv.org/abs/2610.42028v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Reinforcement learning is a powerful paradigm for training autonomous agents and has achieved impressive performance in complex environments. However, this success often comes at the cost of interpretability, diminish...","why_it_matters":"Matches feed focus: agent, evaluation.","sid":"4e3ef058b96b399d","published":"2026-10-30T16:55:46+00:00","editor_note":"Classical RL interpretability: explains agent policies through causal graph reasoning."}]},{"date":"2026-11-02","items":[{"title":"EffComp: Efficient Prompt Compression via Hybrid Reinforcement & Supervised Learning","url":"http://arxiv.org/abs/2611.02766v1","source":"arxiv_llm_reliability","type":"paper","summary_1line":"Prompt compression is aimed at reducing input prompt lengths to enable cheaper, faster, and potentially higher-quality LLM predictions. However, existing prompt compression methods are often limited by modest compress...","sid":"473a95cd7c540d1a","published":"2026-11-02T08:46:28+00:00","editor_note":"Trains a prompt compressor with a mix of RL and supervised learning, aimed at cheaper and faster LLM calls."},{"title":"An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems","url":"http://arxiv.org/abs/2611.03301v1","source":"arxiv_agent_systems_research","type":"paper","summary_1line":"Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study t...","why_it_matters":"Matches feed focus: agent.","sid":"bce25dc1c825a3e7","published":"2026-11-02T15:36:22+00:00","editor_note":"Theory entry: an axiomatic basis for RL when only a subset of agents is controllable, with smart grids as the motivating domain."}]},{"date":"2026-11-03","items":[{"title":"Catalyzing Interactive HTML Generation Capabilities via High-Fidelity Reinforcement Learning","url":"http://arxiv.org/abs/2611.05151v1","source":"arxiv_cs_ai","type":"paper","summary_1line":"Generating HTML from natural-language specifications is an important application of code large language models (LLMs). Despite producing renderable pages, current models often fail to ensure functionality and visual quality during actual interaction. Existing methods typically ta","sid":"3ac08f6799c57341","published":"2026-11-03T16:27:07+00:00","editor_note":"Uses RL to close the gap between HTML that renders and HTML that works and looks right under real interaction."},{"title":"JHiMA-WebRL: A Joint Hierarchical Multi-Agent Reinforcement Learning Framework for Long-Horizon Web Tasks","url":"http://arxiv.org/abs/2611.05263v1","source":"arxiv_agent_systems_research","type":"paper","summary_1line":"Multimodal web agents have shown promising capabilities for interacting with websites through graphical user interfaces (GUIs), yet realistic web tasks often require long-horizon, multi-step planning that remains chal...","why_it_matters":"Matches feed focus: agent, eval.","sid":"08c314bb237b4b82","published":"2026-11-03T17:49:14+00:00","editor_note":"Hierarchical multi-agent RL for long-horizon, multi-step tasks in GUI web agents."},{"title":"Stabilizing On-Policy Targeted Guidance in Agentic Reinforcement Learning","url":"http://arxiv.org/abs/2611.05266v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Naive on-policy RL is often bottlenecked by the capabilities of the base model --- insufficient coverage can result in an inability to learn from certain rewards. At the same time, rewards in modern post-training systems often come from rich textual sources, e.g. judge models wit","sid":"d97ee2168f669bce","published":"2026-11-03T17:54:33+00:00","editor_note":"Addresses on-policy RL stalling when the base model cannot reach the reward, using guidance drawn from textual reward sources such as judge models."},{"title":"MDSKILL: Learning to Internalize Skill Memory through Multi-View Distillation for Efficient Dynamic Agent Self-Evolution","url":"http://arxiv.org/abs/2611.05313v1","source":"arxiv_cs_cl","type":"paper","summary_1line":"Skill memories encode past experiences as textual prompts to improve reinforcement learning (RL) for self-evolving agents. Complementing RL, On-Policy Self-Distillation (OPSD) provides dense, fine-grained supervision...","why_it_matters":"Matches feed focus: agent, eval.","sid":"219537f3fa2b3690","published":"2026-11-03T18:19:55+00:00","editor_note":"Moves skill memory out of the prompt and into the model by combining RL with on-policy self-distillation."},{"title":"Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning","url":"http://arxiv.org/abs/2611.06956v1","source":"arxiv_agent_systems_research","type":"paper","summary_1line":"Multi-agent LLM systems explore in parallel by spawning sub-agents with isolated context windows, whereas single-agent systems append every search and verification step into one growing trajectory. However, existing m...","why_it_matters":"Matches feed focus: agent, eval.","sid":"981c5fba5d1f60bf","published":"2026-11-03T22:42:25+00:00","editor_note":"Multi-agent RL for sub-agents that explore in parallel with isolated context windows."}]},{"date":"2026-11-04","items":[{"title":"What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?","url":"http://arxiv.org/abs/2611.06483v1","source":"arxiv_cs_cl","type":"paper","summary_1line":"While multi-turn agentic reinforcement learning (RL) excels on long-horizon tasks, its performance gains remain highly sensitive to interaction constraints. Evaluating across sampling breadth $K$ and interaction depth...","why_it_matters":"Matches feed focus: agentic, eval.","sid":"4553deeec0e06574","published":"2026-11-04T13:24:54+00:00","editor_note":"The thread's diagnostic paper: multi-turn RL gains are highly sensitive to sampling breadth and interaction depth."}]},{"date":"2026-11-05","items":[{"title":"FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning","url":"http://arxiv.org/abs/2611.08240v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Reinforcement learning (RL) has become a standard post-training method for diffusion models, but most RL objectives roll out the current policy at every iteration, and with a many-step sampler these rollouts typically dominate the cost of training. Few-step models cut this cost,","sid":"4e9768611cda6532","published":"2026-11-05T16:19:52+00:00","editor_note":"Diffusion counterpart to the rollout-cost theme: RL fine-tuning of few-step models, where many-step rollouts otherwise dominate training cost."}]},{"date":"2026-11-06","items":[{"title":"Adaptive Scheduling Pipeline for Multi-Instance Asynchronous Reinforcement Learning","url":"http://arxiv.org/abs/2611.08864v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"We present the Adaptive Scheduling Pipeline (ASP), an offline rollout scheduler for multi-instance asynchronous reinforcement learning (RL) of large language models (LLMs). ASP jointly reduces three sources of rollout underutilization - padding bubbles from intra-minibatch length","sid":"4f75e58fea1157aa","published":"2026-11-06T01:47:52+00:00","editor_note":"Second scheduling paper: an offline rollout scheduler for multi-instance asynchronous RL of LLMs that targets rollout underutilization such as padding from uneven lengths."},{"title":"Encoding Memory in World Models with Test-Time Model-Based Reinforcement Learning","url":"http://arxiv.org/abs/2611.09625v1","source":"arxiv_cs_lg","type":"paper","summary_1line":"Model-based reinforcement learning (MBRL) approaches have demonstrated strong performance across diverse domains in recent years, learning accurate world models from agent trajectories. However, despite this progress,...","why_it_matters":"Matches feed focus: agent.","sid":"6fd3ac39e855936d","published":"2026-11-06T15:45:54+00:00","editor_note":"Model-based RL entry on encoding memory in world models at test time."}]},{"date":"2026-11-09","items":[{"title":"Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation","url":"http://arxiv.org/abs/2611.12118v1","source":"arxiv_agent_systems_research","type":"paper","summary_1line":"Full-duplex voice agents enable natural, hands-free interaction, allowing users to refine requests, redirect ongoing work, and seek clarification while the agent is speaking. However, existing voice-agent benchmarks i...","why_it_matters":"Matches feed focus: agent, evaluation.","sid":"f5f5bfe9b72425ef","published":"2026-11-09T09:31:55+00:00","editor_note":"A benchmark with no RL method: task coordination in full-duplex voice agents, where users interrupt and redirect mid-speech."},{"title":"Unprompted coordination through shared memory in LLM agents: a locked two-implementation replication and extension","url":"http://arxiv.org/abs/2611.12412v1","source":"arxiv_cs_cl","type":"paper","summary_1line":"Coordination between large language-model agents has become a central question for AI safety and software engineering, as such agents are deployed in groups that share infrastructure. Existing research tells the agent...","why_it_matters":"Matches feed focus: agent.","sid":"29f0e99816524f78","published":"2026-11-09T13:43:52+00:00","editor_note":"A replication study with no RL method: whether LLM agents coordinate through shared memory without being told to."},{"title":"MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning","url":"http://arxiv.org/abs/2611.12825v1","source":"arxiv_llm_reliability","type":"paper","summary_1line":"AI systems must often maintain bounded memory of unbounded interaction histories. However, optimizing the memory representation, and its updates, for downstream task utility is difficult for standard reinforcement learning (RL) due to severe credit assignment bottlenecks induced","why_it_matters":"Matches feed focus: evaluation.","sid":"84ecbe8e1def5e6c","published":"2026-11-09T18:25:42+00:00","editor_note":"Trains incremental summarization with RL so bounded agent memory is optimized for downstream task utility."}]},{"date":"2026-11-10","items":[{"title":"AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models","url":"http://arxiv.org/abs/2611.14515v1","source":"arxiv_cs_cl","type":"paper","summary_1line":"Reinforcement learning with verifiable rewards (RLVR) produces rich training experience, yet outcome rewards reduce each attempt to a single number and leave most of this information unused. Training-time scaffolds, such as strategies or hints added to the prompt during training,","why_it_matters":"Matches feed focus: evaluation.","sid":"283f892f9c4f657a","published":"2026-11-10T18:58:46+00:00","editor_note":"Adds AI-chosen training-time hints to RLVR, on the argument that an outcome reward reduces each attempt to a single number."}]},{"date":"2026-11-11","items":[{"title":"ReShap: Interpretable Reinforcement Learning for High-Dimensional Multi-Action Settings","url":"http://arxiv.org/abs/2611.15507v1","source":"arxiv_llm_reliability","type":"paper","summary_1line":"Deep Reinforcement Learning (RL) plays a crucial role in tackling complex tasks, yet its reliance on deep neural networks makes the learned policies opaque. To enhance the trustworthiness of RL policies, existing work (e.g., the SILVER framework [Li et al., 2025]) has focused on","sid":"a7b79c5f448ee3b6","published":"2026-11-11T14:02:55+00:00","editor_note":"Classical RL interpretability for high-dimensional multi-action policies."}]},{"date":"2026-11-12","items":[{"title":"Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination","url":"http://arxiv.org/abs/2611.16661v1","source":"arxiv_cs_ai","type":"paper","summary_1line":"Efficient batch execution of agent tasks is critical to LLM training and development workflows, including benchmark evaluation, synthetic-data generation, and RL rollouts. Since these tasks alternate between LLM infer...","why_it_matters":"Matches feed focus: agent, evaluation.","sid":"e372c76cd210406f","published":"2026-11-12T11:38:21+00:00","editor_note":"Latest item: coordinates LLM inference with environment execution to cut batch makespan for evals, synthetic data and RL rollouts."}]}],"editorial":{"tldr":"This thread is a topical grouping of 27 arXiv preprints since Oct 26, not one event: what they share is reinforcement learning as a post-training method for LLMs and agents. Most of them work on one of three problems: what the reward should measure, how to train multi-turn tool-using agents, and how to pay for the rollouts.","stale":false,"whats_new":"A Nov 12 preprint targets batch makespan for agent workloads, RL rollouts included, by coordinating LLM inference with environment execution, the third rollout-scheduling paper in this thread since Oct 27.","why_it_matters":"For teams that post-train their own agents, the recurring constraints in these papers are rollout cost and reward design, and three of the 27 treat rollout scheduling as an infrastructure problem in its own right.","take_for_builders":"Before adding GPUs to an agent RL or batch-eval pipeline, measure how much rollout wall-clock goes to environment execution and padding rather than inference, since that is the waste the three scheduling papers target.","beats":[{"kicker":"REWARD SIGNAL","tone":"launch","headline":"Outcome-only rewards draw six papers on calibration, credit and guidance","summary":"These papers change what the policy is rewarded for: abstaining, matching confidence to correctness, or using the textual feedback a single outcome score discards.","sids":["c3af4f37e1c19b19","868490bacebdc880","d571e973b6e6916d","9471fde9e20a1e58","d97ee2168f669bce","283f892f9c4f657a"]},{"kicker":"AGENT TRAINING","tone":"rising","headline":"RL moves onto the agent stack: harnesses, skills, memory and sub-agents","summary":"Ten papers apply RL to parts of an agent system rather than to a single-turn model, and one asks what multi-turn RL actually improves.","sids":["c07d00b3f0a46112","d448a8af2a653453","29493d021878283c","473a95cd7c540d1a","3ac08f6799c57341","08c314bb237b4b82","219537f3fa2b3690","981c5fba5d1f60bf","4553deeec0e06574","84ecbe8e1def5e6c"]},{"kicker":"ROLLOUT COST","tone":"now","headline":"Three papers treat rollout scheduling as the bottleneck","summary":"Environment-aware scheduling on Oct 27, an offline scheduler for asynchronous LLM RL on Nov 6, and inference-environment coordination on Nov 12.","sids":["02ad246823185a44","4f75e58fea1157aa","e372c76cd210406f"]},{"kicker":"ADJACENT","tone":"neutral","headline":"Eight clustered papers sit outside LLM agent post-training","summary":"Diffusion and flow fine-tuning, classical RL interpretability and theory, and two agent papers on voice benchmarks and shared-memory coordination.","sids":["65dc6916d3b352f1","4e3ef058b96b399d","bce25dc1c825a3e7","4e9768611cda6532","6fd3ac39e855936d","f5f5bfe9b72425ef","29f0e99816524f78","a7b79c5f448ee3b6"]}],"open_questions":["Do any of the three rollout-scheduling approaches ship as open-source code or get adopted in a public training stack?","Does an independent group reproduce the finding that multi-turn RL gains depend on sampling breadth and interaction depth?","Do the confidence-calibration methods for RLVR hold on agent tasks, beyond the RAG and reasoning settings they were proposed for?","Does co-evolving the harness with the policy beat tuning a harness for a fixed policy on a shared public benchmark?"],"generated_at":"2026-11-14T00:12:06Z"}}