Short summaries of agent optimization and AI engineering work — harnesses, prompt and context optimization, orchestration, memory, and agentic RL.
Filtering by coding agents. Clear filter.
1. Problems / challenges / motivations - Vertical agents fail in domain-specific ways that generic benchmarks never surface. - Practitioner corrections are the highest-value signal a deployed agent produces and usually vanish into a support queue. - "Self-improving" typically means prompt tweaks, which does not fix root causes in the surrounding system. 2....
1. Problems / challenges / motivations - IDE-centric tooling assumes a human in the editor, which is the wrong center of gravity once agents do most of the editing. - Steering long-running agents needs surfaces for both synchronous and asynchronous work. - Model and harness are usually developed by different teams and integrated late. 2. Key ideas - A...
1. Problems / challenges / motivations - Eval methodology is well documented and badly practiced, because doing it properly is tedious. - Coding agents are already in the loop but are not taught how to build evaluation systems. - Error analysis — the highest-value step — is the one teams skip first. 2. Key ideas - Publishes `evals-skills`, a plug-in set of...
1. Problems / challenges / motivations - Codebases are written to be legible to humans, which is not the same as being legible to a coding agent. - When agents write most of the code, the bottleneck moves from writing to specifying, validating, and observing. - Agent failures in a large repo are hard to diagnose without per-run observability. 2. Key...
1. Problems / challenges / motivations - Long-horizon agent work needs both strong coding and strong reasoning, which had been split across separate models. - Fire-and-forget agent runs waste time when the user can see it going wrong but cannot intervene. - Prompting guidance for agent models is under-specified relative to how much it changes behavior. 2....
1. Problems / challenges / motivations - Self-modification search scores a variant by its own benchmark performance, but the useful property is whether it leads somewhere — the two come apart. - The paper names this the Metaproductivity–Performance Mismatch: a mediocre variant can be the ancestor of the best ones. - Expanding the wrong branches burns large...
1. Problems / challenges / motivations - The classical Gödel machine requires a formal proof that a self-modification is an improvement, which is infeasible for real systems. - Agent design is currently done by hand, one scaffold change at a time. - Greedy self-modification collapses into a local optimum and loses variants that would have paid off...