The interplay between Bayesian inference and conformal prediction

Abstract

Conformal prediction (CP) has emerged as a cutting-edge methodology in statistics and machine learning, providing prediction intervals with finite-sample frequentist coverage guarantees. Yet, its interplay with Bayesian statistics-often criticized for lacking frequentist guarantees-remains underexplored. Recent work has suggested that CP can 'calibrate' Bayesian prediction regions, thereby imparting frequentist validity and motivating deeper investigation into frequentist-Bayesian hybrids. On the other side, Bayesian procedures have the potential to enhance CP with more informative intervals, towards nearly optimal solutions under a decision-theoretic framework. Thus, the two paradigms can be jointly used for a principled balance between validity and efficiency. This work provides a unified treatment of this emerging interface with open directions. After surveying existing ideas, we consolidate the literature with a Bayesian version of split CP and present a simple analysis of the binomial model, investigating priors' role, efficiency and computational complexity. This article is part of the theme issue 'Advancing uncertainty quantification in AI systems'.