Finite-Sample Risk Control for Semantic Cache Reuse Decisions: Formal Guarantees, a Self-Selection Feedback Loop, and Economics

Abstract

The main paper (Part I of this series) grounds the threshold choice for gray-zone reuse decisions in honest calibration, but that is still only a point estimate - it never answers how much confidence the resulting error-rate figure deserves given a finite calibration sample. This paper connects Conformal Risk Control (CRC; Angelopoulos et al., 2024) to that decision, producing a distribution-free risk guarantee that holds exactly at finite sample sizes, and honestly stress-tests whether that guarantee survives two realistic deployment conditions. First, whether calibration and test samples are genuinely exchangeable: when split chronologically across three datasets, only one dataset's guarantee still holds conservatively, while the other two overshoot - one statistically stable and severe (up to 80% relative overshoot), the other mild. This paper treats the cause of the former (real production drift versus a dataset-construction artifact) with caution and does not resolve it. Second, whether the reuse decision itself feeds back into future cache composition, inducing a self-selection feedback loop - a genuine online closed-loop simulation confirms this effect is real and follows a precise monotonic dose-response relationship with the dataset's direct-hit rate: the higher the direct-hit rate, the larger the harm. Online recalibration fully compensates for this harm on two datasets, but not on the dataset with the highest direct-hit rate. Finally, this paper proposes an explicit cost-ratio economic framework answering when synchronous verification is actually worth deploying: when the cost of an error clearly exceeds the cost of a miss (typically a ratio of roughly 1-9, dataset-dependent), the mechanism wins economically almost throughout; only when the two costs are close is a static threshold the better choice. This is Part III of a four-part series distilled from a single complete technical report (Zenodo DOI: 10.5281/zenodo.21703364), and a companion to Part I (Xin, C. (2026). Zenodo. https://doi.org/10.5281/zenodo.22660442) and Part II (Xin, C. (2026). Zenodo. https://doi.org/10.5281/zenodo.22661312).