Uncertainty-inspired open-set model for identifying infantile fundus abnormalities

Abstract

Accurate and safe identification of infantile fundus abnormalities is critical for large-scale screening, yet remains challenging because of heterogeneous disease presentations and overconfident predictions by conventional artificial intelligence (AI) models. In this study, we aim to develop an uncertainty-inspired open-set (UIOS) system to address these challenges. Using a multicenter dataset from 39 centers comprising 319,998 fundus images from 15,647 infants, the UIOS-YOLOv8 (UIOS-Y) model integrates multi-class classification with uncertainty estimation to enable uncertainty-inspired prediction and triage for clinical decision-making. The UIOS-Y model achieved high performance in internal testing (AUC 0.995). The UIOS-Y+θ model, by excluding high-uncertainty prediction, achieved an AUC of 0.997 and enabled effective triage of clinically ambiguous cases for clinician review. Robust performance was maintained across five independent external datasets (AUCs ranging from 0.969 to 0.982). UIOS also effectively detected out-of-distribution images, enhancing safety in real-world deployment. Notably, it outperformed resident physicians and general-purpose AI systems, including ChatGPT and Gemini (all p < 0.001), particularly in complex or ambiguous cases. These findings underscore that dedicated uncertainty-inspired open-set modeling is critical for reliable identification of infantile fundus abnormalities despite of the availability of general-purpose AI chatbots, and highlights its value in real-world AI-assisted pediatric ophthalmic screening.