Huxley-Gödel Machine: Human-Level Coding Agent Development
peer-reviewed paper · source date 2025-10-24 · added 2026-08-07 15:52:43 · updated 2026-08-07 15:52:43 · Open original blog
1
Problems / challenges / motivations
- Self-modification search scores a variant by its own benchmark performance, but the useful property is whether it leads somewhere — the two come apart.
- The paper names this the Metaproductivity–Performance Mismatch: a mediocre variant can be the ancestor of the best ones.
- Expanding the wrong branches burns large amounts of compute.
2
Key ideas
- Proposes Clade-Metaproductivity (CMP): score a variant by the aggregate benchmark performance of its descendants rather than its own.
- Guides tree expansion with Thompson sampling, balancing exploring new branches against deepening promising ones.
- Beats DGM and SICA on SWE-bench Verified and Polyglot using fewer CPU hours, reaching human-level on SWE-bench Lite.
- KAUST / MBZUAI (Schmidhuber group); ICLR 2026 oral.
3
Why it matters for AI engineering
- It is the most rigorous refinement of the Gödel-machine lineage's search signal, and later work (Red Queen GM) builds directly on CMP.
- The general principle applies well beyond agent evolution: when selecting among scaffolds or prompts, the metric should reflect downstream potential, not immediate score.
- Caveat: out-of-window (October 2025), and results are coding-benchmark-specific — "human-level" is relative to SWE-bench Lite. Heavily promoted by the authoring group, with limited independent replication of CMP's advantage.
Comments
No comments yet.