Evaluating Retrieval-Augmented Generation Strategies for Repository-Level Code Understanding

Abstract

Retrieval-Augmented Generation (RAG) is increasingly employed to ground large language models in vast code repositories. However, performance degrades when resolving repository-level dependencies and multi-file code contexts. This study evaluates dense, sparse (BM25), and hybrid Reciprocal Rank Fusion (RRF) retrieval strategies across Python, Java, and C++ codebases using an LLM-as-a-Judge evaluation framework.