Short summaries of agent optimization and AI engineering work — harnesses, prompt and context optimization, orchestration, memory, and agentic RL.
Filtering by self-improvement. Clear filter.
1. Problems / challenges / motivations - Discussion of recursive self-improvement defaults to model weights, while most of the practical gains in 2026 came from the code around the model. - The relevant literature is scattered across context engineering, scaffold search, and Gödel-machine work with no common map. - Practitioners have no shared vocabulary...
1. Problems / challenges / motivations - Self-improving agents are bounded by the evaluator that scores their modifications, so a fixed evaluator caps improvement. - In hard-to-verify domains — paper writing, proof grading — there is no benchmark to serve as the evaluator at all. - Letting the evaluator evolve freely invites the agent and evaluator to drift...
1. Problems / challenges / motivations - Vertical agents fail in domain-specific ways that generic benchmarks never surface. - Practitioner corrections are the highest-value signal a deployed agent produces and usually vanish into a support queue. - "Self-improving" typically means prompt tweaks, which does not fix root causes in the surrounding system. 2....
1. Problems / challenges / motivations - RL post-training for compound LLM systems is rollout-hungry: it learns from a scalar reward and throws away everything the trace actually said about the failure. - Prompt optimization by hand does not scale across a multi-module pipeline. - Earlier automatic optimizers (MIPROv2) search over instructions without...
1. Problems / challenges / motivations - Rewriting a whole system prompt on each optimization step introduces brevity bias: the rewrite compresses away details it does not currently see a use for. - Repeated rewrites cause context collapse, where accumulated knowledge degrades rather than compounds. - Adapting a system to a new domain by retraining is slow...
1. Problems / challenges / motivations - Self-modification search scores a variant by its own benchmark performance, but the useful property is whether it leads somewhere — the two come apart. - The paper names this the Metaproductivity–Performance Mismatch: a mediocre variant can be the ancestor of the best ones. - Expanding the wrong branches burns large...
1. Problems / challenges / motivations - Agentic RL work is spread across planning, tool use, memory, reasoning, and perception with no shared taxonomy. - Standard LLM RL formulations assume a single-step MDP, which does not describe an agent acting over a long horizon with partial observability. - Environments, benchmarks, and frameworks are fragmented and...
1. Problems / challenges / motivations - The classical Gödel machine requires a formal proof that a self-modification is an improvement, which is infeasible for real systems. - Agent design is currently done by hand, one scaffold change at a time. - Greedy self-modification collapses into a local optimum and loses variants that would have paid off...