Reference quality of OpenEvidence across five medical specialties
Abstract
OpenEvidence (OE) is a retrieval-augmented clinical AI tool. We evaluated the references OE returned to 150 standardized prompts across five specialties, analyzing all 4979 citations. No reference was confirmed as fabricated; three abstracts had attribution errors. Most references were recent and high-impact, though journal concentration varied by specialty. This study assessed reference existence and characteristics, not whether references support OE’s clinical claims - a benchmark for evaluating clinical AI tools.
// Source
Authors: T.L. Lotan, Mahmud Omar, Yiftach Barash, Jonathan B. Kruskal, Muneeb Ahmed, Olga R. Brook, Eyal Klang, Alon Gorenshtein
Institutions: Harvard University, Tel Aviv University, Beth Israel Deaconess Medical Center, Academic College of Tel Aviv-Yafo