A multi-factor machine learning model for predicting and characterizing clinical trial failures

Abstract

Abstract Background About 15% of clinical trials terminate prematurely (fail), causing financial losses and delaying treatment development. Methods This study utilized a subset of interventional trial records from the 471,252 studies registered in ClinicalTrials.gov until November 2023 to develop a clinical trial failure risk assessment machine learning tool and to examine factors leading to trial failure. The model incorporated trial design, participant demographics, eligibility criteria, disease categorization, and eligibility criteria complexity features. Results Compared to XGBoost, Random Forest, CatBoost and AdaBoost, the LightGBM algorithm performed best, achieving a balanced accuracy of 0.677 (ROC-AUC 0.74), with F1-scores of 0.770 for completed and 0.442 for terminated trials. For detecting terminated trials, sensitivity was 0.68 and specificity 0.67 (positive predictive value 0.33), so approximately one third of failing trials were not flagged. Performance was maintained under retrospective temporal validation (training on trials registered up to 2019 and testing on those registered from 2020 onward), and predicted risks were well calibrated after recalibration. Eligibility-criteria readability and complexity emerged among the most important and most feature-efficient predictors in the SHapley Additive exPlanations (SHAP) analysis. Conclusions Our findings demonstrate this model’s potential to identify trial failure risk and illustrate the potential role of complexity and readability of eligibility criteria in trial failure. Integrating such models into protocol design may help flag at-risk trials for closer scrutiny, but prospective validation is required before clinical recommendations can be made. This finding provides further motivation for sponsors to implement the principles of Good Clinical Practice (GCP) of building quality into the trial design, avoiding unnecessary complexity, and implementing clear, concise and operationally feasible protocols.