Lab / Explainer / 6 stages

No. 04Explainer8 min

From document to answer

What happens between uploading a document and getting a cited answer: reading, cutting, embedding, storing, searching and ranking.

Short version

Retrieval has two halves. Once: read the document, cut it into pieces, turn each piece into numbers and store it. Every question: turn the question into numbers, find the nearest pieces, rank them and hand the best few to the model.

"Chat with your documents" sounds like one feature. It is a pipeline of small decisions, and most quality problems come from the early, unglamorous steps. This page follows one invented heat pump manual, the X200, through the pipeline. Every result is prepared for this page; nothing is sent to a model.

01 · Overview

Retrieval has two halves: once, and every question.

The first half runs once for each document: read it, cut it into pieces, turn each piece into numbers and store it. The second half runs for every question, and can only work with what the first half stored. Ask another question and watch which half runs again.

Once per document

X200 manual, 40 pages

  1. Read40 pages become clean text
  2. Cut86 chunks
  3. Turn into numbersone list of numbers per chunk
  4. Store86 rows in the index

has run 1 time

The index

1 square is 1 chunk: 86 chunks from 40 pages

Every question

"How often should I clean the filter?"

  1. Turn into numbersthe question gets its own list
  2. Searchfinds the nearest chunks
  3. Rankkeeps the best 3
  4. Answerfrom those 3, with sources

Twice a year. Source: page 18.

has run 3 times

Show as text
HalfStepsRuns
Once per documentRead: 40 pages become clean text. Cut: 86 chunks. Turn into numbers: one list of numbers per chunk. Store: 86 rows in the index.Once for the X200 manual (40 pages, 86 chunks)
Every questionTurn into numbers: the question gets its own list. Search: finds the nearest chunks. Rank: keeps the best 3. Answer: from those 3, with sources.Once for every question
QuestionChunks found (of 86)Answer
Why is my heat pump showing E4?28, 24, 46E4 is a sensor fault: call service. Sources: pages 14 and 22.
How far from the wall should the unit be?5, 37, 65At least 30 cm. Source: page 3.
How often should I clean the filter?37, 38, 24Twice a year. Source: page 18.
fig. 1 The left half runs once for each document and fills the index. The right half runs for every question, and only reads what the left half stored.

If the first half is sloppy, no amount of clever searching in the second half will fix it.

02 · Stage 1, read

Reading the PDF is where tables go to die.

Page 14 of the manual has a small table of error codes. Inside this PDF the table happens to be stored column by column. Three ways to turn the page into text: the small numbers show the order in which each one reads the cells.

Page 14, as printed

1Code2Meaning3What to do4E15Low water pressure6Top up to 1.5 bar7E48Sensor fault9Call service10E711Defrost active12Wait 10 minutes

read row by row

The text it makes

CodeMeaningWhat to do

E1Low water pressureTop up to 1.5 bar

E4Sensor faultCall service

E7Defrost activeWait 10 minutes

E4 and "Call service": on the same row

Usable. Rows and columns survive, so each code stays next to its meaning and its action.

Show as text
ReaderText it makesVerdict
Plain textCode E1 E4 E7 Meaning Low water pressure Sensor fault Defrost active What to do Top up to 1.5 bar Call service Wait 10 minutesBroken. This PDF stores the table column by column, and a plain reader follows that order. "E4" and "Call service" end up 17 words apart.
Layout-awareCode | Meaning | What to do; E1 | Low water pressure | Top up to 1.5 bar; E4 | Sensor fault | Call service; E7 | Defrost active | Wait 10 minutesUsable. Rows and columns survive, so each code stays next to its meaning and its action.
Read by a language modelE1 means the water pressure is low: top it up to 1.5 bar. E4 is a sensor fault: call service. E7 means the unit is defrosting: wait about 10 minutes.Usable, and the sentences search well. But it is slower, costs money for every page, and can misread a number.
fig. 2 The same table, read three ways. Only a reader that keeps the rows keeps each code next to what it means and what to do.

Check what your reader makes of a table before you trust anything downstream.

03 · Stage 2, cut

Where you cut decides what can be found together.

A search can only hand back whole chunks, so the cut decides which facts travel together. Here are two pages of the manual, cut three ways. (How cutting one customer message changes its match is on the embeddings page.)

The manual, pages 3 and 14

Installationpage 3

Place the outdoor unit at least 30 cm from the wall, on a flat, stable base, on the rubber feet in the box. Keep 1 m free in front of the fan so the air can flow. Mount the indoor unit on a wall that can carry 60 kg, and leave 20 cm above it for the pipes.

Error codespage 14

E1: low water pressure, top up to 1.5 bar. E4: sensor fault, call service. E7: defrost active, wait 10 minutes.

the cut at character 300 lands inside "pressure"

What gets stored: 2 chunks

  1. X200 manual, Installation, page 3 Installation Place the outdoor unit at least 30 cm from the wall, on a flat, stable base, on the rubber feet in the box. Keep 1 m free in front of the fan so the air can flow. Mount the indoor unit on a wall that can carry 60 kg, and leave 20 cm above it for the pipes. 269 characters
  2. X200 manual, Error codes, page 14 Error codes E1: low water pressure, top up to 1.5 bar. E4: sensor fault, call service. E7: defrost active, wait 10 minutes. 123 characters

Each chunk carries where it came from, so a search for "X200 error" finds it and the answer can cite the page.

Show as text
Way to cutChunks storedVerdict
Every 300 characters1. "Installation Place the outdoor unit at least 30 cm from the wall, on a flat, stable base, on the rubber feet in the box. Keep 1 m free in front of the fan so the air can flow. Mount the indoor unit on a wall that can carry 60 kg, and leave 20 cm above it for the pipes. Error codes E1: low water pres" 2. "sure, top up to 1.5 bar. E4: sensor fault, call service. E7: defrost active, wait 10 minutes."Simple and fast, but it cut "pressure" in half and mixed installation with error codes. The E1 fact is now split over two chunks.
At each heading1. "Installation Place the outdoor unit at least 30 cm from the wall, on a flat, stable base, on the rubber feet in the box. Keep 1 m free in front of the fan so the air can flow. Mount the indoor unit on a wall that can carry 60 kg, and leave 20 cm above it for the pipes." 2. "Error codes E1: low water pressure, top up to 1.5 bar. E4: sensor fault, call service. E7: defrost active, wait 10 minutes."Each chunk is about one thing. But on its own it no longer says which manual or which page it came from.
At each heading, plus a label1. [X200 manual, Installation, page 3] "Installation Place the outdoor unit at least 30 cm from the wall, on a flat, stable base, on the rubber feet in the box. Keep 1 m free in front of the fan so the air can flow. Mount the indoor unit on a wall that can carry 60 kg, and leave 20 cm above it for the pipes." 2. [X200 manual, Error codes, page 14] "Error codes E1: low water pressure, top up to 1.5 bar. E4: sensor fault, call service. E7: defrost active, wait 10 minutes."Each chunk carries where it came from, so a search for "X200 error" finds it and the answer can cite the page.
fig. 3 A cut every 300 characters ignores the document and splits a fact in two. Cutting at the headings keeps each fact whole, and a label says where each piece came from.

Cut along the document's own structure and stamp each piece with where it came from.

04 · Stage 3, store

The index stores more than numbers.

Each chunk becomes one row. The numbers are for searching, the text is what the model will read, the source is for citing, and the labels let you filter before you search. In this index sit the manuals of three heat pump models, and each of them has an error E4. Pick a filter and run the question again.

The question

"What does E4 mean?"

Filter first

product is X200

The index: 7 rows

Numbersfor meaning searchText and sourcewhat the model reads and citesLabelsfor filtering

  1. E4: sensor fault, call service.X200 manual, page 14 X200English
  2. E4: fan blocked, clear leaves from the grille.X300 manual, page 9 X300English
  3. Warranty: E4 and other sensor faults are covered for 5 years.X200 manual, page 22 X200English
  4. E4: sensorfout, bel de service.X200 handleiding, pagina 14 X200Dutch
  5. E4: low refrigerant, call service.X100 manual, page 12 X100English
  6. When your heat pump shows a code, look it up in the manual.X200 manual, page 12 X200English
  7. Plaats de buitenunit minstens 30 cm van de muur.X200 handleiding, pagina 3 X200Dutch

Strip: 64 of the 1,536 numbers in each row, not to scale

Best 3 found

  1. E4: sensor fault, call service.X200 manual, page 140.84
  2. E4: sensorfout, bel de service.X200 handleiding, pagina 140.79
  3. When your heat pump shows a code, look it up in the manual.X200 manual, page 120.66

Only X200 rows are searched. The right E4 is on top.

Show as text
TextSourceLabelsScore for "What does E4 mean?"
E4: sensor fault, call service.X200 manual, page 14product X200, language English0.84
E4: fan blocked, clear leaves from the grille.X300 manual, page 9product X300, language English0.86
Warranty: E4 and other sensor faults are covered for 5 years.X200 manual, page 22product X200, language English0.61
E4: sensorfout, bel de service.X200 handleiding, pagina 14product X200, language Dutch0.79
E4: low refrigerant, call service.X100 manual, page 12product X100, language English0.85
When your heat pump shows a code, look it up in the manual.X200 manual, page 12product X200, language English0.66
Plaats de buitenunit minstens 30 cm van de muur.X200 handleiding, pagina 3product X200, language Dutch0.22
FilterRows searchedBest 3
No filter7 of 7X300 manual, page 9 (0.86); X100 manual, page 12 (0.85); X200 manual, page 14 (0.84). Two other heat pumps come first. Their E4 means something else.
Only the X2005 of 7X200 manual, page 14 (0.84); X200 handleiding, pagina 14 (0.79); X200 manual, page 12 (0.66). Only X200 rows are searched. The right E4 is on top.
Only the X200, in English3 of 7X200 manual, page 14 (0.84); X200 manual, page 12 (0.66); X200 manual, page 22 (0.61). The Dutch copy of the same page no longer takes a place.
fig. 4 Every row stores numbers, text, source and labels. The labels filter out other products before the search, so a look-alike page cannot win.

Labels help when the right product or customer is known. When near-identical documents still get mixed up, see when look-alike documents fool the search.

Store the labels you will want to filter on later: product, language, date, customer.

06 · Stage 5, answer

The model answers from a handful of chunks, and should say which.

Only the best three to five chunks go into the prompt, with an instruction to answer from them and cite them. If none of them answers the question, the honest reply is "I could not find that in the manual". Try the last question too.

The best 3 chunks

  1. 1 X200 manual, Error codes, page 14 ...E4: sensor fault, call service... not used
  2. 2 X200 manual, Troubleshooting, page 12 ...when your heat pump shows a code, look it up in the manual... not used
  3. 3 X200 manual, Warranty, page 22 ...E4 and other sensor faults are covered for 5 years... not used

The prompt

Answer the question using only the sources below. Put the number of each source you use after the sentence it supports. If the sources do not answer the question, say "I could not find that in the manual."

Question: "Why is my heat pump showing E4?"

Sources: 1: Error codes, page 142: Troubleshooting, page 123: Warranty, page 22

The answer

E4 means a sensor fault, so call service.1 A sensor fault is covered by the 5-year warranty.3

Source 2 was read but not needed.

pre-written for this page, nothing is sent to a model

Show as text

"Why is my heat pump showing E4?"

  1. X200 manual, Error codes, page 14: ...E4: sensor fault, call service...
  2. X200 manual, Troubleshooting, page 12: ...when your heat pump shows a code, look it up in the manual...
  3. X200 manual, Warranty, page 22: ...E4 and other sensor faults are covered for 5 years...

Answer: E4 means a sensor fault, so call service. [source 1] A sensor fault is covered by the 5-year warranty. [source 3]

"How far from the wall should the unit be?"

  1. X200 manual, Installation, page 3: ...at least 30 cm from the wall... Keep 1 m free in front of the fan...
  2. X200 manual, Maintenance, page 18: ...keep the grille clear of leaves and snow...
  3. X200 manual, Electrical, page 31: ...connect the X200 to a 230 V group of its own...

Answer: At least 30 cm from the wall, with 1 m free in front of the fan. [source 1]

"Does the X200 work with solar panels?"

  1. X200 manual, Electrical, page 31: ...connect the X200 to a 230 V group of its own...
  2. X200 manual, Energy use, page 20: ...uses about 2,400 kWh a year in an average home...
  3. X200 manual, Installation, page 3: ...at least 30 cm from the wall...

Answer: I could not find that in the manual.

fig. 6 The model gets three numbered chunks and an instruction. It answers only from them, cites the ones it used, and says so when none of them answers.

Ask for citations. They make wrong answers easy to spot and right answers easy to trust.