Context rot is an organization problem, not a model problem
TL;DR
The bigger the context window, the later context rot hits — it doesn't go away. The real solution is structuring your work into short sessions with verifiable deliverables. It's an organizational discipline, not a hardware problem to solve.
There’s a widespread belief in coding agent discussions: if the context window were bigger, context rot problems would disappear. Gemini 2.0 Pro with 2M tokens will fix everything. Claude with 200k tokens solves what GPT-4 couldn’t.
This is false. And the confusion costs time to those who share it.
What context rot actually reveals
When an agent starts producing incoherent results after 30-40 exchanges, the temptation is to blame the model or the context size. “If I had 500k tokens, this wouldn’t happen.”
In reality, context rot in a coding workflow is almost always the symptom of a poorly scoped task. If a session reaches 30 exchanges before producing something verifiable, the initial task was too large or too vague.
I had this realization in November, during a refactoring session. After 2 hours, the agent had produced code, but it had drifted from the conventions established at the beginning, contradicted an architecture decision made at exchange 5, and repeated the same explanation about the Repository pattern twice. I reread the session: the problem wasn’t the model. It was that I’d briefed a task of “refactor the payment service” with no exit criteria.
The same task, broken into three 20-minute sessions with Git checkpoints at the end of each, produced a clean result. The model hadn’t changed.
The real nature of the context window
The context window is a resource to actively manage, not a buffer to fill. The RAM analogy is more accurate than the storage one: you don’t solve a RAM problem by buying more RAM if the issue is that you’re not allocating and freeing properly.
A 1M-token context window doesn’t eliminate context rot — it delays it by a few hours. The same dynamics apply: initial instructions lose influence as the session lengthens, coherence degrades, hallucinations increase.
The real solution is to never need such a long context.
What this changes in the workflow
The discipline I adopted: every session starts with an explicit exit condition. “This session is done when existing tests pass and service X no longer has a direct dependency on Y.” Once the condition is met, you cut. No “while we’re at it, let’s also do Z.”
In practice, the median duration of an effective session is 15-25 minutes. Sessions longer than 45 minutes are almost always a sign of a poorly scoped task — either you interrupt and restart with better scoping, or you finish it knowing you’ll redo it.
This isn’t a model constraint. It’s a working method. And the same method that makes sessions shorter also makes tasks clearer, commits cleaner, and code reviews simpler.
When large contexts are actually useful
This doesn’t mean large context windows are useless. They’re useful for specific tasks: exploring an unknown codebase to produce a map, analyzing a large error log, or comparing two architectural approaches with both in context simultaneously.
But these tasks are one-off analysis sessions, not the normal structure of a daily workflow. For production work — writing code, testing it, committing it — short sessions win every time.
This scoping discipline is no longer just a personal habit: I now push it into the shared AGENTS.md and CLAUDE.md files in the repo, with the explicit exit condition written as a team convention. That’s where it survives past any one person.