The Context Window Is an Agent's Real Working Set — and We're Learning to Page It

Think of the context window as an agent's working set, and the analogy snaps into focus. A program that outgrew physical RAM didn't crash in the 1960s — the OS paged pieces to disk and pulled them back on demand. Long-running AI agents hit the exact same wall, except today most of them still crash. The transcript of every tool call, file read, and retrieval grows without bound, and when it outgrows the model's fixed context limit, the agent simply stops mid-task. Hitting that wall in a long coding or browsing session isn't a question of if, but when.

A wave of 2026 research is attacking the problem from the direction no one expected: borrow virtual memory itself.

The most concrete example is KVMem, a system described in a paper posted this September (arXiv:2609.04852). Instead of compacting old history into lossy summaries, KVMem treats the agent's accumulated context as paged key-value state spread across GPU memory, host RAM, and NVMe. Relevant "pages" are selected from compact attention-space indexes and materialized back into a bounded execution view each step. On a plain 24 GB RTX 5090 Laptop GPU, it ran a Qwen3 27B-class model and virtualized a workspace of up to one million tokens — four times the model's native 256K window — while sustaining single-session throughput around 50 tokens per second. On the DeepSWE long-context coding test, that approach lifted task success from 43.8% with compaction-only context management to 48.4%.

The wins aren't just about survival. Addressable Recall Compaction (arXiv:2607.25066) replaces dropped observations with compact, ID-addressable citations the agent can request on demand, rather than relying on fuzzy similarity retrieval. On needle-in-a-haystack recall it hit 99.40% exact-answer accuracy versus 88.12% for the best baseline. And a context-lifecycle design (arXiv:2606.11213) let a single agent session finish 89 sequential tasks across 80 million tokens of accumulated history with no measurable accuracy loss compared to running each task in isolation. Lossless beats lossy, and structurally-aware beats "drop the oldest turn."

Hermes Agent takes a sober, production-minded slice of these ideas today. Its persistent memory is deliberately bounded — roughly 1,300 tokens across two small files — and the system prompt is captured as a frozen snapshot once per session. That's not a bug; it keeps the prefix cache warm and stable, so the always-hot part of every request stays small and fast. Everything else goes to cheap, unlimited full-text search over past sessions (~20 ms FTS5 queries). Memory is the hot working set; session search is the disk. And for teams that need more, the whole thing is pluggable: swap the bundled context compressor for a context-engine plugin, or connect a richer external memory provider like Honcho or Mem0.

Nobody's shipping million-token paging on a laptop to production just yet, and ARC's authors themselves flag open questions about token overhead and generalization beyond the Qwen3 family. But the direction marker is clear. The context window is not a wall to be smashed by ever-larger limits — it's a resource to be managed, the way every operating system has managed RAM for six decades. The agents that run for days will be the ones that learned to page.