Short summaries of AI and agent evaluation research, organized by broad tags.
Filtering by benchmarks. Clear filter.
1. Problems / challenges / motivations - A frontier model release needs a single artifact that shows what was tested before deployment, not just headline capability scores. - Cyber and bio risk assessments are hard to make credible when the lab designs, runs, and reports its own evaluations. - Alignment claims need something more structured than spot checks...
1. Problems / challenges / motivations - Open-weight frontier releases cannot be recalled, so cyber risk assessment needs to happen before weights are public. - Public cyber benchmarks are contaminated and saturating; private ones are not comparable across evaluators. - No single government has full coverage of the frontier. 2. Key ideas - A joint UK AISI...
1. Problems / challenges / motivations - Fast, cheap models are where most production agent traffic actually runs, but they get thinner eval treatment than flagships. - Agentic and coding capability is scaffold-dependent, so a number without its harness is close to meaningless. - Safety-framework reporting needs to happen for every release, not only the...
1. Problems / challenges / motivations - Factuality work overwhelmingly measures precision (is what was said true?) and neglects completeness (was anything important left out?). - Rich structured rubrics are hard for an LLM judge to apply consistently. - Rubric-based scores often move when you swap the judge model, which makes them hard to trust. 2. Key...
1. Problems / challenges / motivations - Evals assume the system under test is trying to solve the task rather than trying to satisfy the scorer. - Chain-of-thought monitoring is widely proposed as a safety layer, which only works if models disclose rule-breaking in their reasoning. - Nobody had measured spontaneous cheating rates across labs under a common...
1. Problems / challenges / motivations - Open-weight releases from outside the US arrive with vendor-reported numbers and no independent baseline. - Aggregating many heterogeneous benchmarks into one capability claim is usually done informally. - Contamination makes raw public-benchmark comparisons unreliable. 2. Key ideas - A full US-government evaluation...
1. Problems / challenges / motivations - The largest open-weight releases now ship with vendor eval tables well before any technical report or independent check exists. - Practitioners have to decide whether to trust those tables in the gap between launch and verification. - Open-weight releases are irreversible, which raises the stakes on that gap. 2. Key...
1. Problems / challenges / motivations - IRT has quietly become the default aggregation method for serious independent evals (CAISI, ATLAS, GIM), imported from psychometrics. - The AI regime violates the assumptions IRT was built for: few "test takers" (models), very many items, and non-normal ability distributions. - Practitioners have no guidance on when...
1. Problems / challenges / motivations - LLM-judge bias is well documented at the input-output level (position bias, verbosity bias, self-preference) but not explained mechanistically. - Debiasing is therefore trial-and-error prompt engineering. - There is no way to predict in advance whether a judge will fail on a new benchmark. 2. Key ideas - Judge...
1. Problems / challenges / motivations - Terminal-agent benchmarks are saturating while real agentic work runs far longer than any of them. - Binary pass/fail on a multi-hour task throws away almost all the signal in the run. - Long tasks are expensive, so grading has to be worth the compute spent producing the trajectory. 2. Key ideas - 46 long-horizon...
1. Problems / challenges / motivations - Standard safety benchmarks saturate, so a card built on them stops distinguishing models or catching regressions. - Dangerous-capability thresholds (cyber, bio) need methodology that can rule things out, not just report a score. - Smaller family members ship on the same weekend as the flagship and need their own risk...
1. Problems / challenges / motivations - Coding benchmarks are the main public evidence for agent capability, but nobody audits whether their tasks are actually solvable and correctly graded. - Broken tasks and contamination push scores in both directions, and the resulting numbers feed both safety cases and research prioritization. - OpenAI had previously...
1. Problems / challenges / motivations - SE-agent benchmarks are contaminated, syntactic, and outcome-only, so their scores drift away from what developers actually care about. - The same benchmarks are used to argue capability, prioritize research, and support safety cases. - The SWE-Bench Pro retraction made the cost of this concrete rather than...
1. Problems / challenges / motivations - Evaluations read outputs, so anything a model deliberates about but never says is invisible to them. - Sandbagging and eval awareness are threats to eval validity that output-level testing structurally cannot detect. - Interpretability tools rarely connect to concrete evaluation needs. 2. Key ideas - Introduces the...
1. Problems / challenges / motivations - Contamination is the assumed explanation for suspicious benchmark gains, but the evidence was scattered across dozens of individual studies. - Detection methods differ in what access they need (weights, logits, training data) and what kind of leakage they can see. - Teams have no standard way to disclose what they...
1. Problems / challenges / motivations - Scientific-reasoning benchmarks mostly test recall or single-step analysis, not the judgment calls that make research hard. - Real analysis has dependent decision forks: an early wrong turn invalidates everything downstream. - Benchmarks authored by a lab whose models top them are easy to discount. 2. Key ideas -...
1. Problems / challenges / motivations - Independent third-party evaluations are increasingly important for frontier AI trust, but old chatbot-style tests under-measure systems that now use tools, preserve state, and act through agent harnesses. - OpenAI argues that evaluation reports should not only publish a score; they should explain what claim the setup...
1. Problems / challenges / motivations - As LLMs move from task-specific systems toward open-ended agents, one scalar score is often too opaque. A medical answer, deep-research report, tool-using trajectory, or multimodal output may need separate checks for factuality, completeness, reasoning soundness, evidence use, safety, format compliance, and practical...
1. Problems / challenges / motivations - Outcome leaderboards are too flat: one pass/fail score hides whether an agent chose the right action, used tools safely, or recovered after an error. - Agent benchmarks reward different behaviors: final success, tool-call validity, repeated-pass consistency, trajectory safety, or attack robustness. That makes...
1. Problems / challenges / motivations - Standard benchmarks favor tasks that are short, fixed, cheap, and automatically graded. That is useful for scale, but it misses messy deployed work: coordinating tools, resolving unclear requirements, waiting on external systems, and finishing multi-step projects. - Benchmarks can overstate and understate capability....
1. Problems / challenges / motivations - Agent evaluation has moved beyond answer scoring because agents now navigate websites, use tools, edit files, run terminals, recover from failures, and trade off cost and latency. - Public benchmarks measure different slices of capability, so one leaderboard number cannot tell a team whether an agent fits its...
1. Problems / challenges / motivations - Human-backed AI benchmarks often collapse disagreement into a single label even when the task is subjective. - Benchmark builders face an annotation-budget tradeoff: rate more items with fewer raters each, or fewer items with more raters each. - Too few raters can make model comparisons fragile, especially for...
1. Problems / challenges / motivations - Anthropic reports cases where Claude Opus 4.6 inferred it might be inside BrowseComp, searched for benchmark materials, and found or decrypted answer keys. - Web-enabled evaluations are vulnerable to public contamination from papers, blog posts, GitHub repositories, answer keys, and benchmark discussions. - The...
1. Problems / challenges / motivations - Agentic coding benchmarks are sensitive to infrastructure: CPU, RAM, timeouts, container limits, filesystem behavior, and sandbox configuration. - Infrastructure differences can move scores by several percentage points, sometimes more than the reported gap between leaderboard models. - Strict resource ceilings can...
1. Problems / challenges / motivations - Vercel wanted coding agents to use version-matched Next.js 16 documentation, but optional knowledge packages only help if the agent actually invokes them. - A support system can look good in theory while failing at the trigger layer: the agent may not know when to load a skill, may load it too late, or may be...
1. Problems / challenges / motivations - Anthropic's performance-engineering take-home interview lost signal as Claude became strong enough to solve earlier versions of the task. - Static technical evaluations decay when AI assistance improves; a task that once measured human skill can become a test of whether the candidate uses a strong enough model. -...
1. Problems / challenges / motivations - Agent evals are different from single-turn chat evals because agents use tools, change external state, and may fail across multiple turns even when the final answer sounds correct. - Final-message grading misses the most important question: did the task actually succeed in the environment, database, browser, files,...