Prompting Large Language Models with Partial Knowledge for Answering Questions with Unseen Entities

Abstract

Retrieval-Augmented Generation (RAG) systems are often constrained by the “relevance myopia”, primarily focusing on fully relevant external information to supply missing or outdated facts while overlooking the potential of partially relevant knowledge to activate the internal knowledge of Large Language Models (LLMs). We challenge this paradigm by proposing that partially relevant knowledge can “awaken” latent semantic information within LLMs. We provide a theoretical framework, grounded in information theory, that explains this awakening effect: partially relevant knowledge reduces the conditional entropy of the answer by increasing the mutual information between the context and the model's parametric knowledge. We thoroughly investigate the influence of different types of partially relevant knowledge by leveraging the gold reasoning path from Knowledge Graphs (KGs). Furthermore, we introduce a new problem called Unseen Entity Knowledge Graph Question Answering (UE-KGQA), where the entity mentioned in a question does not have corresponding entity in the KG. Specifically, UE-KGQA simulates the realistic RAG with incomplete KGs and evaluates the effectiveness of the partially relevant knowledge. Experiments conducted on three KGQA datasets reveal that prompting with partially relevant knowledge substantially outperforms traditional similarity-based RAG methods, often susceptible to noisy and irrelevant outcomes.