Context Rot: How Increasing Input Tokens Impacts LLM Performance
technical report · source date 2025-07-01 · added 2026-08-07 15:52:43 · updated 2026-08-07 15:52:43 · Open original blog
1
Problems / challenges / motivations
- Long context windows are marketed as capacity, and teams treat "fits in the window" as "will be used correctly."
- Needle-in-a-haystack tests use high semantic similarity between question and needle, which is the easy case.
- Without evidence of degradation, there is no case for the engineering cost of compaction and retrieval.
2
Key ideas
- Tests 18 frontier models and shows accuracy degrades as input length grows even on simple tasks, well before the context window fills.
- Degradation worsens with low needle-question semantic similarity and with distractors present.
- Coined and popularized the term "context rot," with an open repo (`chroma-core/context-rot`).
3
Why it matters for AI engineering
- Context rot is the empirical backbone cited across 2026 context-engineering writing and the motivating evidence for nearly the whole compaction and context-management wave — SelfCompact, CompactionRL, Self-Sum, ARC — and for vendor context-editing primitives.
- Practical read: budget context deliberately rather than to the window limit, and test retrieval with low-similarity, distractor-heavy queries.
- Caveat: out-of-window (July 2025) and a vendor technical report rather than a peer-reviewed study, though methodologically rigorous and widely referenced independently. Findings are model-set-specific and predate several frontier models.
Comments
No comments yet.