RAG Chunk Calculator

How the RAG Chunk Calculator works

How the RAG chunk calculator works

In retrieval-augmented generation, documents are split into chunks, each chunk is embedded, and at question time the k most similar chunks are pasted into the prompt. Overlap repeats the end of one chunk at the start of the next so a sentence cut at a boundary still appears whole somewhere. It costs extra tokens to embed and store, and the top-k chunks take k × chunk size tokens of the context window.

The formula for the number of chunks is ⌈(total − overlap) ÷ (size − overlap)⌉, and tokens stored = total + (chunks − 1) × overlap.

First example. 1,000,000 tokens in 512-token chunks with 51 tokens of overlap, about 10 percent, gives 2,170 chunks and 1,110,619 tokens stored, 11.1 percent overhead. At 1,536 dimensions the vectors take 13.33 MB as float32, and top-5 retrieval adds 2,560 tokens, about 1,920 words, to each prompt.

Second example. A 300-token document in 256-token chunks with 32 overlap gives 2 chunks and 332 tokens stored, 10.7 percent more than the original.

How many chunks should you retrieve? Three to ten is common; more raises recall but adds cost and noise to every prompt.

Chunks needed for a 1,000,000-token corpus by chunk size and overlap

Tips and pitfalls

Smaller chunks match questions more precisely but lose surrounding context; larger chunks keep context but dilute the embedding. Split on headings, paragraphs and sentences rather than raw token counts where you can. Test chunk sizes against real questions, measuring whether the right passage comes back, rather than picking a number by habit.

Related tools