Patient-level topology-aware graph transformer for breast histopathology classification

Abstract

Progress in computational pathology is limited not only by model architecture, but also by evaluation design. In breast histopathology benchmarks such as BreaKHis, multiple correlated image tiles often originate from the same patient; consequently, image-wise splits can inflate performance by allowing patient-specific staining, preparation, or morphology signatures to appear in both training and test sets. At the same time, many classifiers emphasize local texture patterns, whereas diagnostic interpretation often depends on spatial relations among glands, stroma, tumor nests, and tissue compartments. We address these two issues with a patient-level topology-aware graph transformer that preserves spatial backbone features, converts each image into a grid of regional tissue tokens, and applies dense multi-head attention as a learned weighted graph over tissue regions. Evaluation was organized into three acts. Act I reproduced the commonly used image-wise protocol and achieved high tile-level performance, with 99.6% binary accuracy and 97.1% eight-class accuracy. Act II enforced patient-wise splitting while retaining tile-level scoring; eight-class accuracy decreased to 58.6% and binary accuracy to 92.1%, demonstrating the effect of patient-disjoint evaluation. Act III preserved patient-wise folds and aggregated tile evidence into calibrated patient-level predictions. Across magnifications, patient-level multiclass accuracy ranged from 71.5% to 76.3%, macro-F1 from 50.8% to 59.8%, Top-3 accuracy from 83.5% to 87.0%, and binary patient-level accuracy from 93.8% to 97.5%. Comparisons with plain Swin aggregation, graph-removal variants, explicit k-nearest-neighbor and grid-graph baselines, multiple-instance learning baselines, RetCCL, and CTransPath showed that the full graph model provided the strongest primary patient-level classification performance among the tested configurations. These findings support a shift from image-level leaderboards toward leakage-aware, patient-level reporting on BreaKHis, while external validation and pathologist assessment of attention maps remain necessary before broader deployment claims.