AI & Agent Evaluation
2,151total visitsadmin
reading room / notes / ai engineering

Reading room

Short summaries of agent optimization and AI engineering work — harnesses, prompt and context optimization, orchestration, memory, and agentic RL.

$ evals.index --public
category: ai engineering
posts: 6
mode: short summaries
storage: postgres
status: listening
ai eval (43)ai engineering (28)
Sort by source dateLatest firstEarliest first

Filtering by orchestration. Clear filter.

Solve-Detect-Verify (FlexiVe): Inference-Time Scaling with a Flexible Generative Verifier

peer-reviewed paper · source date 2026-06-01 · 0 comments · original

1. Problems / challenges / motivations - Best-of-N spends the same verification compute on easy and hard candidates alike. - Generative verifiers are accurate but expensive, which limits how often they can be called. - Verifier training and verifier-guided search are usually studied separately. 2. Key ideas - FlexiVe is a generative verifier that...

verl — Volcano Engine RL Library (Agentic RL Path)

open-source framework · source date 2026-06-01 · 0 comments · original

1. Problems / challenges / motivations - Every lab reimplementing PPO/GRPO plumbing wastes effort and makes results incomparable. - Multi-turn agentic rollouts with tool calls do not fit training loops designed for single-turn completions. - Rollout throughput becomes the bottleneck without async and load balancing. 2. Key ideas - A production RL training...

Introducing Google Antigravity 2.0

product release · source date 2026-05-19 · 0 comments · original

1. Problems / challenges / motivations - IDE-centric tooling assumes a human in the editor, which is the wrong center of gravity once agents do most of the editing. - Steering long-running agents needs surfaces for both synchronous and asynchronous work. - Model and harness are usually developed by different teams and integrated late. 2. Key ideas - A...

Capable Language Models Can Outgrow the Benefits of Collaboration

peer-reviewed paper · source date 2026-05-01 · 0 comments · original

1. Problems / challenges / motivations - Multi-agent debate and collaboration results were mostly established on weaker base models and assumed to hold as models improve. - Teams have no decision rule for when to add agents versus when a single strong model suffices. - Collaboration adds cost and latency that need justification. 2. Key ideas - Quantifies...

Towards a Science of Scaling Agent Systems

research blog · source date 2026-04-01 · 0 comments · original

1. Problems / challenges / motivations - Multi-agent systems are adopted on the assumption that more agents means better results, with little controlled evidence. - Comparisons across papers confound topology, model, and token budget. - Errors propagate differently depending on how agents are wired together, and that effect is rarely measured. 2. Key...

Kimi K2.5: Visual Agentic Intelligence

technical report · source date 2026-02-01 · 0 comments · original

1. Problems / challenges / motivations - RLVR works where rewards are verifiable, which excludes most multimodal and open-ended agentic tasks. - Scaling agentic post-training needs a reward signal that does not depend on hand-written verifiers for every domain. - Single-agent rollouts limit the diversity of trajectories available to learn from. 2. Key...