Main idea

A language model's context window is finite, which everyone knows. What is assumed next is false: that within that limit, everything supplied is equally available. Stanford researchers measured it in Lost in the Middle (Liu et al., 2024) — information placed at the beginning or the end of the context is retrieved far better than the same information buried in the middle.

The practical consequence bears on a very common move: loading a three-thousand-line file in the hope that the model will "understand everything". The model does not refuse, signals nothing, and answers. What degrades is not the shape of the answer, it is what the answer actually took into account — and nothing in the output says what was missed.

Filling up is therefore not neutral. Every line added to the context does not merely add to what the model knows: it pushes whatever was already there towards the middle.

Why it matters

This turns the window size a vendor advertises into a maximum capacity rather than a usable one. Doubling the window does not double what the model actually has at its disposal, and reasoning in terms of available tokens leads you to fill up to the brim.

It also justifies work that would otherwise look superfluous: splitting, ranking and selecting what you supply, even when everything would technically fit.

Nuances and limits

The effect is a property of models observed at a given moment, and its magnitude varies from one architecture and one generation to the next. What remains is the principle — filling up has a quality cost — more than the figure.

And position does not explain everything: information that is badly phrased or contradicted elsewhere in the context gets lost just as well at the top of a document.

Open questions

• How do you detect, on a plausible answer, that a piece of information supplied to the model was never mobilised?