A guide on how to refine and fix issues encountered within RAG systems. A lot of tutorials often miss concepts such as chunking strategy, hybrid search, re-ranking, structured parsing, evaluation etc... the list goes on. This site is here to help bridge that gap.
Each value on this site came from running the project over a real corpus and measuring it.
New here? Start with the system you'll be diagnosing
The project, how it was built, and where the corpus comes from.
"It says it doesn't know, but the answer is right there. I can see it in the document."
Overlapping the chunks by more than the longest fact.
"Our search scores are perfect. It still says it can't answer."
Copying the table's key into every row of it.
"The table in our PDF has turned to nonsense. It gives me numbers, and they're the wrong ones."
Fixing the PDF reader. No search setting touches this.
"I typed the reference number that's printed in our document. It came back with something else entirely."
Blending key-word and semantic search.
"It retrieves the right document and still gets the answer wrong."
Sending less text, or moving the answer to an edge.
"It gave me two different numbers in the same sentence and made it sound like both were right."
Attaching each passage's date, and saying when they disagree.
"We spot-checked one of the figures it gave us. It was wrong."
Shrinking the pile. Nothing warns you when that isn't enough.