Context Rot Is Not a Context Window Problem

We built an agent for a document processing client earlier this year, the kind of thing that ingests a stack of contracts, extracts obligations, cross references them against a policy database, and writes a summary. It worked beautifully in the demo. It worked beautifully for the first dozen documents. Then somewhere around document forty, on a run that had been going for a little over half an hour, it started confidently attributing clauses to the wrong contract, referencing a policy rule it had already flagged as deprecated three steps earlier, and generally behaving like a competent junior who has been in a meeting for too long and is nodding along without listening anymore. The context window was nowhere near full. We were sitting at maybe forty percent of the available tokens on a model with a two hundred thousand token window. There was no truncation, no eviction, nothing dropped. And yet the agent had quietly lost the plot. If you have shipped a long-horizon agent you have seen this, the silent degradation that shows up not as an error but as a slow accumulation of small wrong turns that compound until the output is subtly, expensively useless. The frustrating part is that it does not announce itself. There is no exception, no rate limit, no red log line. The agent just gets dumber over time in a way that correlates with wall-clock duration and step count far more than it correlates with how much of the window you have consumed.

The instinct, and it is a strong one because it feels intuitive, is that the model ran out of room and started forgetting things, so the fix must be a bigger window. Bump from 128k to 200k, or reach for one of the million token variants, and the problem goes away. Except it doesn't. A 2025 analysis of enterprise agent deployments put roughly sixty five percent of failures down to context drift during multi-step reasoning rather than raw token exhaustion, which lines up with what we kept seeing in production, that the failures happen with tons of headroom left. There is a distinct and well documented phenomenon where model performance degrades as the amount of context grows, even when everything technically fits, because attention is not free and not uniform, and the model's ability to retrieve and correctly weigh a specific fact buried in the middle of a huge context is much worse than its ability to handle that same fact in a short prompt. People call it context rot. The signal you actually care about gets diluted by the accumulated noise of every intermediate reasoning step, every tool output, every retry, every bit of conversational scaffolding the agent generated along the way. A bigger window makes this worse, not better, because you have given the model more room to fill with low-value tokens that drown out the three or four facts that actually matter for the current decision. You are optimizing the wrong variable, and the vendors selling you on window size are not incentivized to correct you.

The thing that convinced me this was a discipline problem and not a model problem was watching the degradation cluster around a roughly consistent time horizon across completely different agents on completely different models. Somewhere around the thirty five minute mark, give or take, coherence starts to fall apart, and this holds whether the window is 128k or a million. The reason is that a long-running agent doesn't just accumulate tokens, it accumulates decisions, and each decision was made in the context of the ones before it, so any small error early on gets baked into the reasoning trail and referenced as ground truth by every subsequent step. This is error propagation, and it is time-correlated because time is a proxy for step count. An agent that has run for thirty five minutes has typically taken enough steps that its own past output now dominates the context, and past output includes its mistakes, its half-formed hypotheses, its tool calls that returned garbage which it then reasoned over as if it were real. The model starts trusting its own transcript. Once that transcript is polluted, every new step inherits the pollution, and because the agent has no mechanism to distinguish a confidently stated intermediate guess from a verified fact, the rot spreads. Raising the window ceiling does absolutely nothing about this. You could give the agent infinite tokens and it would still poison its own well.

The discipline that fixes this is context engineering, which means treating the context you feed the model at every step as a deliberately curated artifact rather than an ever-growing append-only log. Three techniques carry most of the weight in production. The first is goal-state pinning, where you keep the original objective and the current sub-goal in a fixed, high-priority slot at the top of every prompt, re-injected fresh on every step, so the model is never more than a few hundred tokens away from remembering what it is actually supposed to be doing. Without this, the goal drifts to the bottom of a growing transcript and gets attention-starved. The second is hierarchical summarization, where instead of carrying the full raw history you periodically compress completed phases into terse, structured summaries, so a block of twenty tool calls that resolved a sub-task becomes a five line 'here is what we learned and here is the current state' note. You throw away the reasoning scaffolding and keep the conclusions. The third is memory offloading, where facts that need to persist go into an external store, a Postgres table, a Redis hash, a scratchpad file, whatever fits, and the agent retrieves them on demand rather than trying to hold everything in the window. This is exactly how you would design a system for a human analyst who cannot hold a hundred facts in working memory, and it works for the same reason. When we rebuilt that document agent around these ideas, pinning the goal, summarizing per-document into a structured record, and offloading extracted obligations to Postgres so the model only ever saw the handful relevant to the current comparison, the minute 35 wall disappeared. Runs that used to fall apart at forty documents held coherence past two hundred.

If you have an agent losing coherence on long tasks, stop reaching for a bigger model and start instrumenting the context. Log the exact prompt sent at every step, then go read the ones from around the point where quality drops, and I would bet money you find the original goal buried under ten thousand tokens of accumulated tool chatter and the model's own second-guessing. That is your diagnosis. Add a pinned goal block and re-inject it every step, that alone buys you a surprising amount. Then look at what your history actually contains and be ruthless about compressing anything that represents completed work into a structured summary, because the raw reasoning that got you there has negative value once the conclusion is reached, it only serves to dilute attention. Move durable facts out of the window into a store the agent queries, and design the retrieval so it pulls the few things relevant to the current step rather than dumping everything back in. None of this is exotic. It is boring engineering discipline applied to a place most teams treat as magic. We do a fair amount of this work at steezr, building document pipelines and AI automation for clients who tried the naive approach first and watched it degrade, and the pattern is always the same, the failure gets misdiagnosed as a model limitation when it is a context management problem the team never built for. Bigger windows are a convenience, not a cure. The teams shipping reliable long-horizon agents are the ones who decided the context was theirs to engineer.