Chenyi Huang

Joint Optimization of Retrieval-Augmented KV Caching and Adaptive Speculative Decoding for Autoregressive Generation

Abstract Retrieval-assisted key–value (KV) reuse and speculative decoding remove different costs, but their naive composition can leave draft and verification overhead after prefill is shortened. RAKV-ASD combines exact causal prefix validation with a measured controller that sel...