RULER: Relative Universal LLM-Elicited Rewards
open-source tool · source date 2026-04-01 · added 2026-08-07 15:52:43 · updated 2026-08-07 15:52:43 · Open original blog
1
Problems / challenges / motivations
- RL on agent tasks needs a reward function, and most real tasks have no verifier and no labeled data.
- Hand-crafting reward functions per task is the main cost of applying RL to agents.
- Absolute LLM-judge scores are noisy and poorly calibrated across tasks.
2
Key ideas
- A general-purpose reward function that uses an LLM judge to rank N trajectories relatively rather than score them absolutely.
- This exploits a property of GRPO: it only needs relative rankings within a group, so calibration problems largely cancel out.
- Requires no labeled data and no hand-crafted reward; integration is a one-line `ruler_score_group` call.
- Reported to match or exceed hand-crafted rewards on 3 of 4 benchmarks.
3
Why it matters for AI engineering
- RULER is the heavily adopted practical answer to verifying non-verifiable agent tasks (the ART repo has roughly 9k+ stars) and is frequently cited across the reward/verifier lane.
- The insight worth stealing is matching the reward's precision to what the algorithm actually consumes — ranking, not scoring.
- Caveat: a vendor/practitioner tool, and "matches hand-crafted rewards" is self-reported on selected benchmarks. LLM-judge relative rewards inherit judge bias and reward-hacking risk (see "LLMs Gaming Verifiers," arXiv:2604.15149); pair any RLVR-style gains built on it with pass@k and spurious-reward baselines.
Comments
No comments yet.