What is a context window and why does it constrain agent design?

Answer

The context window is the maximum amount of text, measured in tokens, a model can consider in a single request. It holds everything the model knows at that moment: system prompt, tool definitions, conversation history, file contents, and tool results.

It constrains agent design because agents fill it fast. Each tool result is appended, so reading files and running commands consumes the window quickly. A single verbose command output can consume more than an entire conversation.

Three consequences follow:

Something must be dropped or summarised once it fills, and whatever is dropped is knowledge the agent no longer has. It does not reliably know what it lost.

Attention degrades before the limit. Information in the middle of a long context is attended to less reliably than information at the start or end. So a large window does not mean you can fill it and expect uniform quality.

Cost scales with it. Every request bills for the whole context, not just the new message, which is why long sessions get progressively more expensive per turn.

Practical response: keep sessions short and focused, put durable knowledge in files loaded fresh each session, be specific about which files matter, and avoid connecting tools you do not use, since every tool definition occupies the window on every request.

Common wrong answer

Treating a large context window as removing the problem. Even with a very large window, attention is uneven, cost is real, and filling it with marginally relevant material measurably degrades output. Bigger windows change the threshold, not the principle.

Confusing the context window with memory is also common. The window is what the model sees now; memory is a mechanism for putting things back into it later.