Hierarchical grouped convolution and multi-scale vision transformer for uncertainty-aware early breast cancer detection using mammography

Abstract

Abstract Early breast cancer detection using mammography remains challenging because existing deep learning methods often emphasize either local lesion characteristics or global contextual information, while providing limited uncertainty quantification and clinically reliable prediction confidence. To address these limitations, this study proposes the Hierarchical Grouped Convolution and Multi-Scale Vision Transformer for Uncertainty-Aware Classification (HGCViT-UAC) framework. The proposed approach first applies adaptive median filtering, CLAHE enhancement, breast region segmentation, and patient-wise data partitioning, followed by hierarchical grouped convolution for multi-level feature extraction, multi-scale ViT attention, adaptive feature fusion, and Monte Carlo Dropout-based uncertainty estimation with calibration analysis. The framework was implemented using Python 3.10 and PyTorch 2.0 and evaluated on the CBIS-DDSM dataset containing 10,239 mammography images from 1,566 subjects using a patient-wise 70/15/15 split. The proposed model achieved 98.67% accuracy, 98.62% F1-score, 0.9914 AUC, 0.019 Expected Calibration Error, 0.026 Brier Score, and 98.67 ± 0.31% mean accuracy with a 95% confidence interval of 98.06–99.28%. Compared with the baseline ViT (84.32% accuracy), the proposed framework improves classification performance by 14.35% through hierarchical multi-scale feature learning, adaptive attention fusion, and calibrated uncertainty estimation, demonstrating reliable and clinically applicable breast cancer diagnosis.