The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
peer-reviewed survey · source date 2025-09-02 · added 2026-08-07 15:52:43 · updated 2026-08-07 15:52:43 · Open original blog
1
Problems / challenges / motivations
- Agentic RL work is spread across planning, tool use, memory, reasoning, and perception with no shared taxonomy.
- Standard LLM RL formulations assume a single-step MDP, which does not describe an agent acting over a long horizon with partial observability.
- Environments, benchmarks, and frameworks are fragmented and hard to compare.
2
Key ideas
- Synthesizes 500+ works into a twofold taxonomy of agentic RL across planning, tool use, memory, reasoning, self-improvement, and perception.
- Reframes LLM RL from single-step MDPs to long-horizon POMDPs, which is the framing most later work adopts.
- Consolidates open environments, benchmarks, and frameworks into a curated compendium maintained as a living reference.
- Published in TMLR 2026.
3
Why it matters for AI engineering
- It is the canonical, field-defining map for agentic RL and the safest grounding reference for the whole lane — most in-window RL work situates itself against this taxonomy.
- The POMDP reframing has direct engineering consequences: credit assignment, memory, and reward design all change once you stop pretending the episode is one step.
- Caveat: the original preprint predates the window and is retained as an anchor on the strength of its TMLR 2026 acceptance. As a survey it aggregates others' often self-reported results rather than producing new validation.
Comments
No comments yet.