Your RAG Eval Is Checking the Receipt, Not the Patient

A recent study highlights a critical flaw in clinical retrieval-augmented generation (RAG) systems: they can provide grounded answers, cite real sources, and pass faithfulness checks, yet still deliver incorrect information due to entity attribution failures. This issue, termed deceptive grounding, occurs when the model retrieves evidence for one drug but applies it to another, leading to potentially dangerous misinterpretations.

The authors emphasize the need for RAG evaluations to include entity checks that verify whether the cited evidence corresponds to the correct entity. This additional layer of scrutiny is essential, especially in high-stakes fields like medicine, law, and finance, where incorrect associations can have serious consequences. The study suggests that simply checking for real citations is insufficient; the evaluation must ensure the right context is attached to the right subject.