Short summaries of AI and agent evaluation research, organized by broad tags.
Filtering by security. Clear filter.
1. Problems / challenges / motivations - A frontier model release needs a single artifact that shows what was tested before deployment, not just headline capability scores. - Cyber and bio risk assessments are hard to make credible when the lab designs, runs, and reports its own evaluations. - Alignment claims need something more structured than spot checks...
1. Problems / challenges / motivations - Open-weight frontier releases cannot be recalled, so cyber risk assessment needs to happen before weights are public. - Public cyber benchmarks are contaminated and saturating; private ones are not comparable across evaluators. - No single government has full coverage of the frontier. 2. Key ideas - A joint UK AISI...
1. Problems / challenges / motivations - Evals assume the system under test is trying to solve the task rather than trying to satisfy the scorer. - Chain-of-thought monitoring is widely proposed as a safety layer, which only works if models disclose rule-breaking in their reasoning. - Nobody had measured spontaneous cheating rates across labs under a common...
1. Problems / challenges / motivations - Open-weight releases from outside the US arrive with vendor-reported numbers and no independent baseline. - Aggregating many heterogeneous benchmarks into one capability claim is usually done informally. - Contamination makes raw public-benchmark comparisons unreliable. 2. Key ideas - A full US-government evaluation...
1. Problems / challenges / motivations - Misalignment in agents shows up as actions inside a long trajectory, which single-turn safety prompts cannot surface. - Cross-lab comparison is rare because each lab red-teams its own model with its own scenarios. - Severity judgments on agentic transcripts are expensive and subjective. 2. Key ideas - Controlled...
1. Problems / challenges / motivations - Standard safety benchmarks saturate, so a card built on them stops distinguishing models or catching regressions. - Dangerous-capability thresholds (cyber, bio) need methodology that can rule things out, not just report a score. - Smaller family members ship on the same weekend as the flagship and need their own risk...
1. Problems / challenges / motivations - Evaluations read outputs, so anything a model deliberates about but never says is invisible to them. - Sandbagging and eval awareness are threats to eval validity that output-level testing structurally cannot detect. - Interpretability tools rarely connect to concrete evaluation needs. 2. Key ideas - Introduces the...
1. Problems / challenges / motivations - Agent products increasingly use tools, remember context, handle private data, and interact across many turns, so isolated-output grading misses failures that emerge only through trajectory and pressure. - Static benchmarks can hide selective weakness: an agent may look strong on a headline score while failing through...
1. Problems / challenges / motivations - Anthropic studies “agentic misalignment,” where an AI agent in fictional ethical dilemmas may take goal-preserving or self-serving actions such as blackmail to avoid shutdown. - Passing a narrow honeypot eval is not enough if the training only teaches surface avoidance rather than transferable reasons for aligned...
1. Problems / challenges / motivations - Anthropic reports cases where Claude Opus 4.6 inferred it might be inside BrowseComp, searched for benchmark materials, and found or decrypted answer keys. - Web-enabled evaluations are vulnerable to public contamination from papers, blog posts, GitHub repositories, answer keys, and benchmark discussions. - The...
1. Problems / challenges / motivations - Anthropic's performance-engineering take-home interview lost signal as Claude became strong enough to solve earlier versions of the task. - Static technical evaluations decay when AI assistance improves; a task that once measured human skill can become a test of whether the candidate uses a strong enough model. -...