Retrieval-Augmented Generation for FDA Drug Label Search

Abstract

MedSearch AI is a retrieval-augmented generation (RAG) system designed to make FDA drug label information searchable through natural-language questions. The system processes 1,468 unique drug documents extracted from FDA Structured Product Labeling (SPL) XML files and compares four retrieval approaches: TF-IDF, BM25, dense retrieval using Latent Semantic Analysis (LSA), and hybrid retrieval. BM25 achieved the highest Top-1 retrieval accuracy of 90%, while hybrid retrieval achieved 100% Top-10 accuracy. The best-performing retriever was integrated with Llama 3.2 through Ollama to generate responses grounded in retrieved FDA label information. The study also evaluates generation faithfulness and relevance and analyzes failure modes including hallucination, generation failure, and context leakage.