Compaction and summarisation

When a conversation approaches the context limit, something has to give. Compaction is the standard answer: replace older turns with a summary and carry on.

What it does

The harness takes the earlier part of the conversation, asks a model to summarise it, and swaps the summary in for the original messages. The conversation continues with far fewer tokens.

Some harnesses do this automatically. Some providers offer it server side. Either way the mechanism is the same, and so is the trade.

What gets lost

A summary is lossy on purpose, and it is not lossy evenly.

Exact strings survive badly. File paths, function names, error text, and version numbers get paraphrased away. If the agent starts referring to a file that does not quite exist, this is often why.

Negative decisions vanish first. "We tried X and it did not work" tends to disappear, so the agent proposes X again. This is the most common and most annoying symptom.

Your constraints get generalised. "Never touch config/prod.yml" becomes "be careful with config files."

Controlling it

Compact deliberately, not accidentally. Automatic compaction fires when the window is nearly full, which is usually mid task and at the worst moment. If you can see it coming, get ahead of it.

Ask for the summary you want. Before compacting, ask explicitly:

> Summarise this session: decisions made and why, approaches we ruled out and why, exact file paths touched, and what is still open.

That prompt preserves the categories that generic summaries drop.

Put durable facts in files. Anything that must survive belongs in the repository, not in the conversation. A rules file or a short notes file is compaction proof, because it is re-read rather than remembered.

Prefer a fresh session with a handoff. Often better than compacting: finish, write down where you got to, start clean. You control exactly what carries over.

Signs it has happened

• The agent asks for something you gave it earlier.
• It suggests an approach you already rejected together.
• It refers to a file with a slightly wrong path.
• Its answers become noticeably more generic.

None of these mean the model got worse. They mean the evidence it was relying on is gone.

Design for it

For long running work, keep a short state file in the repository: current goal, decisions made, what is blocked. Have the agent update it as it goes.

That file survives compaction, survives a new session, and survives you closing your laptop. It is the most reliable memory available, precisely because it is not in the context window by default.