"We need RAG" is often the first sentence of an AI feature plan, and often the wrong first step. Models now read hundreds of pages in one go. This page helps you pick the simplest option that works. Sizes and costs are rough rules of thumb, last checked 2026-11-13, and every document on it is invented.
01 · Measure
Start by measuring how much content you have.
The context window is how much text a model can read at once. Assume 200,000 tokens, roughly 400 pages. Drag to see where your content lands. The window is drawn twice: up close, where a dot is about a page, and from afar, where a square is a whole window.
Your content
200 pages
about 100,000 tokens
1 page is about 500 tokens: 1 dot
One context window, up close
200 of 400 dots
All of it, from afar
- 1 square = 1 whole window, the grid up close
- your content: 0.5 windows
Fits: uses 50% of the window.
Show as text
| Content | Tokens | In one window | Whole windows | Verdict |
|---|---|---|---|---|
| 20 pages | 10,000 | 20 of 400 dots | 0.05 | Fits: uses 5% of the window. |
| 200 pages | 100,000 | 200 of 400 dots | 0.5 | Fits: uses 50% of the window. |
| 2,000 pages | 1,000,000 | 400 of 400 dots | 5 | Does not fit: 5 times the window. 1,600 of the 2,000 pages are left out. |
| 20,000 pages | 10,000,000 | 400 of 400 dots | 50 | Does not fit: 50 times the window. 19,600 of the 20,000 pages are left out. |
Under a few hundred pages, you may not need search at all.
02 · Options
There are four ways to give a model your knowledge.
Small content goes straight into the prompt. Bigger content needs something in between that picks what the model reads: a summary made in advance, a search that finds the right pieces, or records looked up by who is asking. Follow one invented question through each option: what the model has to read for it, and what you have to build.
This page only decides between them. How a search index is built and read is drawn in fig. 1 of How vector search works. Why look-alike documents need records, and how the lookup works, is drawn in fig. 3 of When look-alike documents fool the search.
A customer asks
"Is my e-bike covered if it is stolen abroad?"
300 pages of policy terms for three bike plans, plus help articles: 150,000 tokens
What you build in between
Nothing in between
- Your software pastes every page in with the question.
- Prompt caching makes repeats cheap.
What the model reads, to scale
150,000 tokens: all 300 pages, with every question
- Use when
- Small content that fits in one window.
- Main risk
- Content outgrows the window.
1 dot = 500 tokens, as in fig. 1; the pale dots are the rest of the window.
Show as text
| Option | Use when | What you build | Cost | Main risk | The model reads, per question |
|---|---|---|---|---|---|
| Put it all in the prompt | Small content that fits in one window. | Nothing in between: Your software pastes every page in with the question. Prompt caching makes repeats cheap. | Lowest build effort. Cost per question grows with size; prompt caching makes repeats cheap. | Content outgrows the window. | All 300 pages, with every question: 150,000 tokens |
| Compress first, then prompt | Too big for the prompt as is, but mostly repetition and boilerplate. | A summary step: Summarise each document once. Rerun it for a document that changes. | A one-off summarising step per document, rerun when it changes. | The summary drops a detail someone needs. Test against the full text. | A 40-page summary of all 300: 20,000 tokens |
| Search pieces (RAG) | Content too big for one window that changes often, or has distinct topics. | A search index: Cut every document into pieces and index them. Keep the index up to date. Find the 5 closest pieces for each question. | An index to build and keep up to date; more moving parts. | Retrieval misses the right piece; needs testing with real questions. | The 5 closest pieces, about a page each: 2,500 tokens |
| Look up records | Many look-alike documents where the right one depends on who is asking. | Records with fields: Put the facts that differ into fields, one record per plan. Look up the record of whoever is asking. Give every record an owner. | Facts need to live in fields on records; work up front. | Records go stale if nobody owns them. | The asker's plan record: 300 tokens |
Each step up adds machinery to build and keep running. Climb only when you must.
03 · Decide
Four questions, one recommendation.
Answer four questions about your own content. The checks are asked in order, and the first yes decides; with three no's, it is search. How bad a wrong answer is never changes the option: it adds checks on top. Change any answer to follow the path again.
Your situation
Asked in orderThe option
-
1Many look-alike documents?no
Look up recordsRecords with fields
-
2Fits in one window?yes
Put it all in the promptNothing in between
-
3Up to 4,000 pages, and changes rarely?
Compress first, then promptA summary step
-
No to all three
Search pieces (RAG)A search index
Then: is a wrong answer costly?no
- Ask for citations
- Keep a test set of real questions
- Put a person on the risky answers
200 pages fits in one context window. Put it in the prompt, use prompt caching, and measure.
Show the decision rules as text
| If | Then |
|---|---|
| Many look-alike documents | Look up records (search only for general questions) |
| Fits in the window (up to about 400 pages), however often it changes | Put it all in the prompt (send the fresh version with every question) |
| Too big for one window, up to about 4,000 pages, and changes rarely | Compress first, then prompt |
| Too big for one window, and changes weekly or faster or is over about 4,000 pages | Search pieces (hybrid search) |
| A wrong answer is costly | Add citations, a test set and a human check on top of any option |
| Content | Changes | No look-alikes | Many look-alikes |
|---|---|---|---|
| 20 pages | Rarely | Put it all in the prompt | Look up records |
| 20 pages | Weekly | Put it all in the prompt | Look up records |
| 20 pages | Daily or faster | Put it all in the prompt | Look up records |
| 200 pages | Rarely | Put it all in the prompt | Look up records |
| 200 pages | Weekly | Put it all in the prompt | Look up records |
| 200 pages | Daily or faster | Put it all in the prompt | Look up records |
| 2,000 pages | Rarely | Compress first, then prompt | Look up records |
| 2,000 pages | Weekly | Search pieces (RAG) | Look up records |
| 2,000 pages | Daily or faster | Search pieces (RAG) | Look up records |
| 20,000 pages | Rarely | Search pieces (RAG) | Look up records |
| 20,000 pages | Weekly | Search pieces (RAG) | Look up records |
| 20,000 pages | Daily or faster | Search pieces (RAG) | Look up records |
The answer changes most with size and with look-alikes, less with anything else.