Private retrieval over my own papers

A folder of about forty PDFs is the whole corpus, and the question I ask of it most often is “where did I read that”. Retrieval-augmented generation answers that question well and answers almost nothing else reliably.11Lewis et al. (2020) introduced the retrieval-augmented pattern: retrieve passages by vector similarity, then generate conditioned on them. The failure modes I describe follow from the pattern, not from any one implementation.

The setup is AnythingLLM pointed at the folder, with bge-m3 producing the embeddings and a local Qwen3 32B model writing the answers, all on the workstation. Each PDF is split into chunks of a few hundred words, each chunk becomes a vector, and a question is matched against those vectors before the model sees anything. The model then writes from the matched chunks and cites them.

What it is good for is locating. Asked which of the reviews gives a threshold for how many data points a molecular model needs, it returns the right paper and the right page in a few seconds, which is faster than my own memory and much faster than reopening five PDFs. It is also good for the first draft of a reference list and for finding every place a term appears across the corpus.

What it is not good for is the number itself. A retrieved chunk can be the wrong table, a value can be transcribed with the wrong unit, and the model will state the result with the same confidence either way. The rule that makes the tool safe is older than the tool: no number, claim or quotation goes into anything I write until I have read it in the source. Retrieval tells me where to read. It does not replace the reading.

Two practical findings. The embedding model matters more than the chat model; changing from a general English embedding to bge-m3 improved the hit rate on engineering terms more than any change of chat model did. And chunk size is a trade: short chunks find a sentence, long chunks keep a table together, and tables are where chemical engineering papers keep their results, so I lean long.

Open for me is whether a corpus of forty papers is worth this machinery at all. At forty, a good folder structure and a reading note per paper do most of the work. At four hundred the retrieval would earn its keep. The reading notes in the Library are the part of the system that scales either way.

Footnotes

1. Lewis et al. (2020) introduced the retrieval-augmented pattern: retrieve passages by vector similarity, then generate conditioned on them. The failure modes I describe follow from the pattern, not from any one implementation. ↩