Retrieval quality problems are usually ingestion problems.
Before a single vector is computed, something decided where your documents get cut. If that decision was a default in a tutorial — a character count and an overlap — then your tables are in halves, your code blocks end mid-function, and the chunk that contains the answer is missing the heading that said what the answer was about.
This site is about that decision. Chunk size as a trade-off rather than a setting. Structural boundaries versus fixed ones. What overlap actually buys and what it costs. The metadata that has to travel with a chunk for it to mean anything. And the document types — tables, code, transcripts — that break every general strategy and need their own.
There is no universal chunk size here, because there isn’t one. What there is: the mechanism behind each failure, the same text shown cut two ways so you can see the damage, and a procedure for finding what works on your corpus.
Latest posts
-
Why chunk size is a tradeoff, not a setting
Small chunks retrieve precisely and answer incompletely. Large chunks do the reverse. What each end actually breaks, and how to find your corpus's middle.
-
Split on structure, not on characters
A character splitter cuts where the counter runs out. Documents already have boundaries — headings, clauses, functions. Use those; count only as a fallback.
-
Overlap, and what it actually buys
Overlap insures against cutting through an idea. It also inflates your index and fills top-k with near-copies. When it earns its cost, and when it's a patch.
-
Metadata is half the chunk
A chunk with no source, date or heading path can't be filtered or cited. What to attach, what to embed with the text, and what to keep out of the vector.
-
Chunking tables, code and transcripts
Three document types that break general splitters, and the handling each needs: repeated headers, syntax-aware cuts, and topic-grouped speaker turns.