Diagnostic performance and energy consumption of deep learning models for dermoscopic image classification

Abstract

Deep-learning models for dermoscopy are usually chosen on the basis of accuracy and discrimination, with limited attention to the energy and carbon costs that accompany model development, retraining, and clinical use. If clinical AI is to be governed responsibly, these operational dimensions need to enter the discussion alongside conventional diagnostic metrics. We compared three widely used dermoscopic classification architectures on both clinical performance and environmental cost, and asked how energy-aware reporting might fit into the broader practice of clinical AI governance and algorithmovigilance. The HAM10000 dataset (10,015 dermatologist-labelled dermoscopic images from 7470 unique lesions, seven diagnostic classes) was split at the lesion level into training, validation, and test sets, so that every image of a given lesion was assigned to a single partition. ResNet-50, EfficientNet-B3, and Vision Transformer B/16 were fine-tuned in TensorFlow/Keras on a single NVIDIA Tesla P100 GPU. Each architecture was trained three times; metrics were computed separately per run and are reported as mean ± standard deviation. We measured accuracy, area under the receiver operating characteristic curve (AUROC), macro- and micro-averaged sensitivity and specificity, per-class sensitivity, and calibration (expected calibration error, Brier score), with lesion-clustered bootstrap confidence intervals and Bonferroni-corrected pairwise tests. Training and inference energy were measured directly by sampling board-level GPU power through the NVIDIA management interface; carbon emissions were re-computed under four electricity-grid scenarios. No statistically significant difference in accuracy was detected between any pair of architectures (ResNet-50 0.807 ± 0.002, EfficientNet-B3 0.817 ± 0.002, Vision Transformer B/16 0.808 ± 0.006). Vision Transformer B/16 required 3.07 times the training energy of EfficientNet-B3 (0.0630 ± 0.0038 vs 0.0205 ± 0.0010 kWh per run) and 4.32 times the inference energy (0.307 vs 0.071 Wh per 1,000 images), without a detectable advantage on accuracy (− 0.89 percentage points, 95% CI − 2.47 to + 0.64), AUROC (+ 0.0026, 95% CI − 0.0034 to + 0.0090) or melanoma sensitivity (+ 0.044, 95% CI − 0.026 to + 0.109). The comparison between the two convolutional networks was different in kind: EfficientNet-B3 consumed 43% more training energy than ResNet-50 and returned a significantly higher melanoma sensitivity (+ 0.102, 95% CI + 0.038 to +0.167). Parameter count did not predict energy use—the 10.8-million-parameter EfficientNet-B3 consumed more training energy than the 23.6-million-parameter ResNet-50. Thermal-design-power estimates of inference energy were 2.7–13.8 times the measured values. Changing the electricity grid changed absolute emissions but left the ranking of the three architectures unchanged. Energy is not a decision rule on its own, but it cannot enter a decision at all unless it is measured. Measuring it here produced two different answers rather than one: additional energy corresponded to a statistically significant and potentially clinically important absolute gain in melanoma sensitivity in one comparison, and to no detectable gain in the other. We do not claim that Vision Transformer B/16 and EfficientNet-B3 are equivalent—the study was not designed as a non-inferiority comparison and no clinical margin was pre-specified—but an advantage that cannot be demonstrated does not discharge the burden of justifying a threefold difference in energy use. We argue that energy use, carbon emissions, hardware configuration, calibration, and inference cost should be reported in clinical AI studies as a matter of course, and integrated into pre-deployment evaluation and post-market monitoring of medical imaging models.