Context overflow is a situation where too much text is put into a model's input in the hope that the answer will improve. In practice, the opposite happens. Models work more reliably with information at the start and end of the input, while data in the middle of a long block tends to get lost. At the same time, the more text there is, the greater the risk that an unrelated or contradictory passage makes it into the answer. For content creation, this effect has a specific consequence: a key claim, figure or condition should not be hidden in the middle of a long paragraph but placed right at the start of the passage. For companies building their own solutions on top of documents, it follows that a more precise selection of a smaller number of passages brings better results than inserting the whole document.
See also: Top-k retrieval, Content chunking, Grounding.
