Chunking documentsintermediate
A model can only read so much at once, so you cut the document into overlapping pieces — the overlap is what stops an answer being split in half at a boundary.
Split a document into retrievable pieces. No API key, no network — this is string handling, and it is where most RAG apps are already broken.
Chunks overlap, no chunk exceeds its size, and no sentence is ever cut in half.
Published in full, before you start — every point is one of these and there is nothing else. Each one runs your code; it is not a search for keywords.
A model can only read so much at once, so you cut the document into overlapping pieces — the overlap is what stops an answer being split in half at a boundary.
An AI answer with no sources cannot be checked, and an answer that cannot be checked cannot be trusted — showing which chunk each claim came from is the whole point of RAG.