Quantifying and adjusting for verification bias in diagnostic test accuracy studies with conditional reference standards

Abstract

Abstract Background In diagnostic test accuracy studies, it is often infeasible to verify the true disease status of all participants with a gold standard. Several alternative study designs with incomplete verification exist. In a “check the negatives” or “check the positives” design, all participants first receive an imperfect reference standard, and only those testing negative or positive, respectively, undergo verification with the gold standard. We aimed to quantify bias arising from these designs and assess the extent to which this bias can be corrected using a model-based approach. Methods We algebraically quantify the bias in standard crude estimates of sensitivity and specificity from a “check the negatives” design. To address this bias, we propose a Bayesian model incorporating informative priors for the imperfect reference standard’s specificity and, if applicable, a conditional dependence parameter. Under conditional independence, we assessed model performance using simulations, including scenarios where the prior for specificity was correctly centred around the true value and scenarios where this parameter was underestimated. Algebraic results and a model-based correction for a “check the positives” design follow by symmetry—in this case, requiring prior information on the imperfect reference standard’s sensitivity. Results In a “check the negatives” study, crude specificity is unbiased and sensitivity is underestimated under conditional independence. Under positive conditional dependence, specificity is overestimated, while bias in sensitivity can be in either direction. Simulations show that the model-based correction reduced or eliminated bias when the prior for specificity was correctly centred. Overly pessimistic priors sometimes over-corrected bias, although this risk was mitigated with lower prior precision. In two applied examples, where data were insufficient to inform a prior or conditional dependence, the model-based adjustment increased estimated sensitivity under an assumption of conditional independence. Conclusions Researchers can use our algebraic results to assess potential bias before choosing a “check the negatives” or “check the positives” design. These designs are efficient, with minimal bias in some scenarios, but substantial bias in others. Bias can be adjusted for if accurate information is available on reference test specificity or sensitivity, respectively, though misspecified priors can lead to incorrect results.