Evaluating domain generalization in agricultural deep learning through multi-seed LOSO validation and explainable AI

Abstract

Abstract Conventional validation schemes may overestimate model robustness in agricultural deep learning when spatial domain shifts are not explicitly considered. This study evaluated convolutional neural networks for pasture biomass estimation using the CSIRO Image2Biomass dataset within a multi-seed Leave-One-State-Out (LOSO) framework. A total of 24 experiments combined four Australian geographical domains, two architectures (ResNet50 and EfficientNet-B0), and three random seeds (42, 123, and 2025). Performance was assessed using WRMSLE, MAE, RMSE, and R 2 . Seed-to-seed training variability was summarized using standard deviations across three random seeds, while 95% bootstrap confidence intervals for test WRMSLE were estimated by resampling the held-out test samples within each seed-level LOSO run. A metadata-based Random Forest was included as a non-image baseline. ResNet50 achieved mean test WRMSLE values ranging from 0.869 to 1.460 across held-out domains, whereas EfficientNet-B0 ranged from 0.911 to 1.519. The Random Forest baseline outperformed both CNN architectures in New South Wales and Western Australia, suggesting that structured ecological metadata can remain highly competitive when image-based models are transferred to unseen regions. To examine the visual basis of prediction errors, a systematic Grad-CAM audit was conducted using 40 activation maps derived from the five lowest- and five highest-error samples in each domain. The resulting attention patterns suggested domain-dependent model behavior and provided qualitative evidence consistent with reliance on contextual visual cues; however, the quantitative vegetation-attention measure should be interpreted as a coarse diagnostic proxy rather than a ground-truth segmentation metric. Overall, the combination of multi-seed LOSO validation, seed-level variability analysis, baseline benchmarking, and systematic XAI auditing constitutes a pre-deployment diagnostic framework for identifying generalization risks in agricultural computer-vision systems operating under spatial uncertainty.