A multimodal deep learning method based on multiple medical images for Fuchs endothelial corneal dystrophy diagnosis

Abstract

To establish a multimodal deep learning network for the automated diagnosis of Fuchs endothelial corneal dystrophy (FECD). The ResNet-50 neural network was trained and validated using patients with FECD, patients with other corneal endothelial disorders, and healthy controls. At the single-eye level, single-modal and multimodal models were developed on the basis of anterior segment (AS) photographs, anterior segment optical coherence tomography (OCT) images, and IVCM images. Independent test sets were employed to assess the diagnostic performance of the models, with evaluation metrics including precision, recall, F1 score, and accuracy. At the single-eye level, the multimodal model achieved a precision of 0.9637 and a recall of 0.9872, with an F1 score of 0.9702, in distinguishing between FECD, other corneal endothelial disorders, and healthy eyes. At the class level, the precision/recall/F1 score was 0.9585/0.9843/0.9773 for FECD, 0.9852/0.9962/0.9747 for other corneal endothelial disorders, and 0.9532/0.9832/0.9641 for healthy eyes. The overall accuracy was 0.9872 (69 eyes). The diagnostic performance was better than that of single-modal models based on AS photographs (F1 score: 0.8684) and anterior segment OCT images (F1 score: 0.8407) and slightly better than that of the single-modal model based on IVCM images (F1 score: 0.9302). All reported precision, recall, and F1 scores are weighted-average metrics. This study presents the first multimodal deep learning model for diagnosing FECD, effectively distinguishing it from healthy corneal conditions and various anterior segment disorders. This multimodal AI model integrates standard slit-lamp photography, OCT and IVCM inputs to deliver instant, technician-level FECD screening without extra hardware. Embedding the ResNet-50 classifier in EMR or cloud platforms could facilitate the rapid and accurate diagnosis of FECD.