Communicating the uncertainty of individual risk from clinical prediction tools with the PGower similarity measure

Abstract

Clinical prediction models offer risk estimates for individual patients that are uncertain, even when calibration and performance at the population level are good. We compute Gower’s distance between patients in the training dataset to construct a metric against which new patients can be compared, yielding p Gower , an interpretable similarity measure defined as the proportion of training patients with a larger average distance than the new patient, bounded between 0 and 1. We conducted a simulation study to illustrate how a low p Gower value is associated with approximation and model uncertainty in an individual’s predicted risk. We demonstrate the clinical relevance of p Gower in a case study ( n = 9108) using a prediction model for ovarian cancer. Patients with lower p Gower values showed greater variability and uncertainty in predicted risks, indicating reduced reliability of model outputs. Reporting p Gower alongside predicted risk may help clinicians identify patients whose predictions warrant greater caution in interpretation.