Towards a structure-aware multi-hop retrieval framework for large language model reasoning over complex oncology guidelines

Abstract

Large language models (LLMs) augmented with clinical guidelines show promise in medical question answering, yet structural parsing and evidence retrieval over complex guidelines remain challenging. We propose SMART-RAG (Structure-aware Multi-hop Augmented Retrieval Technique for RAG), a structure-aware multi-hop retrieval framework for National Comprehensive Cancer Network (NCCN) guidelines. SMART-RAG integrates automated structural parsing, structured chunking with metadata, structure-aware reranking, and multi-hop retrieval to handle flowcharts, tables, annotations, and cross-references with dynamic cross-page and cross-guideline evidence tracing. We systematically evaluated SMART-RAG on both choice and free-response oncology benchmarks constructed from public and manually curated datasets. Results showed that, conventional RAG was susceptible to retrieval noise, which could lead to erroneous prediction shifts in up to 20.69% of originally correct questions. In contrast, SMART-RAG achieved more stable performance improvements, increasing accuracy by 14.02% while reducing retrieval interference by 11.17%. Ablation studies further confirmed the importance of chunking, reranking, and multi-hop retrieval.