product docs · source date 2026-07-14 · 0 comments ·
original1. Problems / challenges / motivations
- Optimization research assumes someone will wire traces, evals, candidate generation, and deployment together by hand.
- Enterprises will not run a self-improvement loop without lineage, diffs, and rollback.
- Production traces are the best optimization signal available and usually go unused.
2. Key ideas
- A managed...
peer-reviewed paper · source date 2026-06-01 · 0 comments ·
original1. Problems / challenges / motivations
- Production agent quality is gated by the quality of the evaluation loop, and most teams' LLM judges are not calibrated against anything.
- Hand-written judge prompts drift and disagree with human reviewers, which makes every downstream decision noisy.
- Optimization research is overwhelmingly benchmark-based, with...
product docs · source date 2026-05-01 · 0 comments ·
original1. Problems / challenges / motivations
- Prompt optimizers mostly live inside one framework, so adopting them means adopting the framework.
- Optimized prompts are artifacts that need versioning, diffing, and rollback like any other deployed asset.
- Without CI integration, optimization is a one-off experiment rather than a maintained process.
2. Key...
peer-reviewed paper · source date 2026-04-24 · 0 comments ·
original1. Problems / challenges / motivations
- RL post-training for compound LLM systems is rollout-hungry: it learns from a scalar reward and throws away everything the trace actually said about the failure.
- Prompt optimization by hand does not scale across a multi-module pipeline.
- Earlier automatic optimizers (MIPROv2) search over instructions without...
peer-reviewed paper · source date 2026-01-30 · 0 comments ·
original1. Problems / challenges / motivations
- Rewriting a whole system prompt on each optimization step introduces brevity bias: the rewrite compresses away details it does not currently see a use for.
- Repeated rewrites cause context collapse, where accumulated knowledge degrades rather than compounds.
- Adapting a system to a new domain by retraining is slow...