Development and internal validation of an EHR machine learning model for clinical deterioration in ICU linked inpatients

Abstract

Abstract Early warning scores for inpatient deterioration rely predominantly on contemporaneous vital signs and may not incorporate broader electronic health record (EHR) information. We developed and internally validated a multi-domain EHR machine-learning model using MIMIC-IV v3.1. A 6 hourly sliding-window framework generated 1,334,773 monitoring windows from 80,442 ICU-linked admissions. The composite outcome was ICU transfer, invasive mechanical ventilation, or in-hospital mortality within 24 h (prevalence 1.96%). XGBoost achieved an AUROC of 0.758 (patient-clustered 95% CI 0.753–0.763), compared with 0.568 for modified NEWS2, 0.549 for qSOFA, 0.539 for the Shock Index, and 0.533 for MEWS. Domain-level ablation identified vital signs as the dominant predictive domain, while multi-domain integration improved AUROC by 0.041 over a vitals-only model. At 90% sensitivity, the model generated 2.51 alerts per monitored patient-day with 2.8% positive predictive value. Collapsing contiguous alerts yielded 0.52 alert episodes per patient-day with 96.9% event-level sensitivity, although episode-level precision remained low at 5.1%. External validation and prospective operational evaluation are required before clinical use.