Efficient retrieval-augmented generation via multi-signal token compression

Abstract

Retrieval-augmented generation (RAG) improves question answering by grounding large language models in retrieved passages, but long contexts increase prompt-token cost and latency. We introduce HLLC, a deterministic, generator-agnostic framework that compresses corpus chunks during indexing using linguistic features, smoothed corpus-level lexical salience, and embedding-based semantic contribution. Because compression is performed before retrieval, the compressed index can be reused across queries without retraining the retriever or modifying the downstream model. HLLC is evaluated on the multiple-choice RACE and QuALITY benchmarks across publicly available open-weight language models and is compared with LLMLingua and LLMLingua-2. Moderate compression reduces retrieved-context tokens while retaining a substantial proportion of uncompressed exact-match accuracy, and indexing-stage compression avoids repeated query-time compression overhead. We additionally report ROUGE, BLEU, BGE-large cosine similarity, and named-entity retention as chunk-level preservation diagnostics. These measures describe semantic, lexical, and factual-anchor retention but do not establish open-ended answer quality or preservation of complete reasoning chains. The conclusions are therefore restricted to text-based, multiple-choice RAG under the evaluated datasets, models, and execution environment.