Andrew Stellman responds to a question from Mike Loukides (his editor at Radar) about models tending to ignore information placed in the middle of long contexts. Stellman explains that while clearing and reloading context can sometimes help, the issue is deeper and has been characterized by researchers as a U‑shape problem.
What the research shows
Nelson Liu (Stanford) ran experiments that placed relevant answers at different positions in long inputs and measured retrieval. He found that models perform best when the information is at the beginning of the context window or at the recent end, and worst when it sits in the middle. These tendencies are called primacy bias and recency bias; together they form the U‑shape when plotted against position.
A 2025 ICML paper, "On the Emergence of Position Bias in Transformers," interprets the phenomenon as an equilibrium between two internal forces: the causal mask that amplifies early tokens (primacy) and position encodings like RoPE that weight tokens near the generation point (recency); the middle is where these effects cancel. Borun Chowdhury (Meta) in 2026 argued mathematically in "Lost in the Middle at Birth: An Exact Theory of Transformer Position Bias" that the U‑shape appears at initialization, before any training, with random weights.
This matters because larger context windows — even those reaching a million or multiple millions of tokens — have improved single‑fact retrieval but do not automatically make long‑context agent workflows reliable. A bigger window simply creates a larger middle where information can be deprioritized.
Five practical techniques to manage the U‑shape
Stellman presents five techniques he used while developing his open source Quality Playbook skill. The shared principle is that working memory is unreliable, so important state should be externalized, curated, and verified.
1) Curate, don't accumulate
Avoid relying on a single, long session to remain coherent. Create a separate "context brief" (a standalone document containing file paths, diffs, tests, etc.), then start a fresh session that reads that brief. Stellman used this during the Quality Playbook v1.5.2 release prep: the brief contained all necessary details, so a new session reading it produced reliable, focused fixes.
2) Put critical information at the edges
Because models attend more to the start and the recent end of context, place the load‑bearing information there and leave less critical material in the middle. In Claude Code, for example, the --append-system-prompt option can put a brief into the system prompt so the agent attends to it strongly throughout the session.
3) Prefer short sessions to long ones
Design workflows as many short, restartable sessions that read state from disk rather than one long session that accumulates context. Stellman describes his Haiku 4.5‑based system that indexes chat history: the first attempt tried to keep dedupe state in the model's memory and failed. The fix was a protocol that updated progress.json after each line and resumed from that cursor in a fresh session, expecting to run out of context and restarting cleanly.
4) Restate key info right before use
If the model must apply a constraint now, repeat it now and force a fresh read from the source. Stellman fixed the BUGS.md problem in Quality Playbook by telling the agent: "Before writing BUG‑NNN.md, re‑read the BUG‑NNN entry in BUGS.md... Do not paraphrase from memory." That explicit instruction forced the model to re‑read the file at the moment of action and prevented it from filling in blank templates from memory.
5) Test the middle
Don't assume the agent is using the middle content; check it deterministically. Compare what the agent claims (progress cursor, state) to the on‑disk truth (files, logs). In the Haiku summarizer protocol, each new session cross‑checked progress.json against the last line in the summary file; disagreements were flagged and the run paused. This kind of checkpointing prevents drifting state from becoming a buried bug.
The discipline behind the techniques
Stellman notes that the multi‑phase architecture he built to avoid silent compaction also reduces lost‑in‑the‑middle failures: each phase has a short, focused context with a brief at the start and an update at the end, minimizing any middle where information can fall out of attention. Both failure modes — context compaction and lost‑in‑the‑middle — stem from unreliable working memory, and the same discipline addresses both: keep the working set small, place load‑bearing info at the edges, and verify agent claims against ground truth on disk.
Context windows will continue to grow and compaction strategies will improve, and some techniques may eventually change. But because the U‑shape may be a structural property of transformer architectures, externalizing critical state, curating what stays in context, and building deterministic checks remain practical and likely long‑lived strategies for reliable long‑context agent workflows.
