engineering blog · source date 2026-06-03 · 0 comments ·
original1. Problems / challenges / motivations
- Self-service analytics fails in two directions: give an agent broad data access and metric definitions diverge across teams; lock access down and it cannot answer the business questions people actually have.
- Three specific failure modes drive wrong answers: concept-to-entity ambiguity (mapping a question to the...
open-source framework · source date 2026-06-01 · 0 comments ·
original1. Problems / challenges / motivations
- Every lab reimplementing PPO/GRPO plumbing wastes effort and makes results incomparable.
- Multi-turn agentic rollouts with tool calls do not fit training loops designed for single-turn completions.
- Rollout throughput becomes the bottleneck without async and load balancing.
2. Key ideas
- A production RL training...
product release · source date 2026-05-19 · 0 comments ·
original1. Problems / challenges / motivations
- IDE-centric tooling assumes a human in the editor, which is the wrong center of gravity once agents do most of the editing.
- Steering long-running agents needs surfaces for both synchronous and asynchronous work.
- Model and harness are usually developed by different teams and integrated late.
2. Key ideas
- A...
product docs · source date 2026-05-01 · 0 comments ·
original1. Problems / challenges / motivations
- Prompt optimizers mostly live inside one framework, so adopting them means adopting the framework.
- Optimized prompts are artifacts that need versioning, diffing, and rollback like any other deployed asset.
- Without CI integration, optimization is a one-off experiment rather than a maintained process.
2. Key...
peer-reviewed paper · source date 2026-04-24 · 0 comments ·
original1. Problems / challenges / motivations
- RL post-training for compound LLM systems is rollout-hungry: it learns from a scalar reward and throws away everything the trace actually said about the failure.
- Prompt optimization by hand does not scale across a multi-module pipeline.
- Earlier automatic optimizers (MIPROv2) search over instructions without...
open-source tool · source date 2026-04-01 · 0 comments ·
original1. Problems / challenges / motivations
- RL on agent tasks needs a reward function, and most real tasks have no verifier and no labeled data.
- Hand-crafting reward functions per task is the main cost of applying RL to agents.
- Absolute LLM-judge scores are noisy and poorly calibrated across tasks.
2. Key ideas
- A general-purpose reward function that...
open-source framework · source date 2026-03-29 · 0 comments ·
original1. Problems / challenges / motivations
- RL post-training couples a training engine to a rollout engine, and synchronous coupling leaves accelerators idle.
- Agentic rollouts (tool calls, sandboxes, multi-agent loops) are slow and variable, which makes the coupling worse.
- Agentic RL is often bolted on as a separate framework rather than treated as data...
practitioner blog · source date 2026-03-03 · 0 comments ·
original1. Problems / challenges / motivations
- Eval methodology is well documented and badly practiced, because doing it properly is tedious.
- Coding agents are already in the loop but are not taught how to build evaluation systems.
- Error analysis — the highest-value step — is the one teams skip first.
2. Key ideas
- Publishes `evals-skills`, a plug-in set of...
engineering blog · source date 2026-02-11 · 0 comments ·
original1. Problems / challenges / motivations
- Codebases are written to be legible to humans, which is not the same as being legible to a coding agent.
- When agents write most of the code, the bottleneck moves from writing to specifying, validating, and observing.
- Agent failures in a large repo are hard to diagnose without per-run observability.
2. Key...
engineering blog · source date 2026-01-22 · 0 comments ·
original1. Problems / challenges / motivations
- Building a specialized agent per domain multiplies maintenance and fragments capability.
- Stuffing all domain knowledge into a system prompt does not scale past a few domains.
- Reusable agent capability had no portable packaging format.
2. Key ideas
- Explains why Anthropic stopped building specialized agents and...
engineering blog · source date 2025-11-04 · 0 comments ·
original1. Problems / challenges / motivations
- Loading every MCP tool definition into context burns tokens before the agent does anything.
- Intermediate tool results flow through the model even when the model only needs a filtered summary.
- Tool selection degrades as the tool count grows.
2. Key ideas
- Presents MCP servers as code APIs the agent explores on a...
engineering blog · source date 2025-09-29 · 0 comments ·
original1. Problems / challenges / motivations
- Prompt engineering assumes a fixed instruction; agents assemble a different context at every step.
- Long-horizon agents run past any context window, so something has to decide what survives.
- The field lacked shared vocabulary for what that decision process is.
2. Key ideas
- Frames context engineering as the...