How to refine and fix issues encountered within RAG systems.
Each value on this site came from running the project over a real corpus and measuring it.
New here? Start with the system you'll be diagnosing
The project, how it was built, and where the corpus comes from.
Cut in half by a chunk boundary
"It says it doesn't know, but the answer is right there. I can see it in the document."
Overlapping the chunks by more than the longest fact.
The explanation that never gets retrieved
"Our search scores are perfect. It still says it can't answer."
Copying the table's legend into every row of it.
A table that stopped being a table
"The table in our PDF has turned to nonsense. It gives me numbers, and they're the wrong ones."
Fixing the PDF reader. No search setting touches this.
An identifier semantic search can't find
"I typed the reference number that's printed in our document. It came back with something else entirely."
Blending key-word and semantic search.
"It retrieves the right document and still gets the answer wrong."
Sending less text, or moving the answer to an edge.
Two sources disagreed, so it invented a reason why
"It gave me two different numbers in the same sentence and made it sound like both were right."
Attaching each passage's date, and saying when they disagree.
An answer pulled from memory, not the document
"We spot-checked one of the figures it gave us. It was wrong."
Shrinking the pile. Nothing warns you when that isn't enough.