TRUST-Campus: Evidence-Aware Retrieval-Augmented Generation for Reliable University Information Access

Abstract

TRUST-Campus presents an evidence-aware Retrieval-Augmented Generation (RAG) framework designed to improve the reliability of university information access by distinguishing between retrieved evidence that is relevant to a question and evidence that is sufficient to support an answer. The framework introduces a semantic evidence-sufficiency decision layer between retrieval and generation, enabling the system to answer when supporting evidence is adequate and abstain when information is incomplete, context-dependent, or unsupported. The research evaluates four approaches: Dense RAG, Hybrid RAG, Reranked RAG, and Evidence-Aware RAG V4. Experiments use a frozen 50-question evaluation set spanning seven categories: paraphrase, compositional reasoning, negation, context-dependent questions, unsupported requests, future-oriented questions, and false premises. All four approaches achieved a document retrieval hit rate of 100%. Evidence-Aware RAG V4 achieved the highest answer/abstain decision accuracy of 82%, compared with 74% for Reranked RAG and 66% for Dense and Hybrid RAG. It also increased answer coverage to 42% and reduced the false rejection rate to 20%, compared with 18% coverage and 60% false rejection for Reranked RAG. Paired statistical testing found a significant improvement in coverage but not in decision accuracy. Evidence-Aware V4 also exhibited a higher false acceptance rate than Reranked RAG, demonstrating a tradeoff between answering more questions and avoiding unsupported answers. The findings highlight the importance of explicitly evaluating evidence sufficiency in trustworthy RAG systems, particularly for policy-oriented and context-dependent question answering. The study identifies personalized queries and missing user-specific context as important directions for future research.