S2DFusion: a statistical and scale difference attention network for infrared and visible image fusion

Abstract

Infrared and visible image fusion aims to integrate cross-modal complementary information for comprehensive all-weather scene perception. However, existing methods often fail to effectively decouple background noise from modality-specific salient features, leading to contrast degradation, diminished thermal target prominence, and the loss of fine texture details in the fused images. To address these limitations, we propose S2DFusion, a novel statistical and scale-differential attention network that constructs a “difference-oriented” explicit decoupling and fusion paradigm. Specifically, the infrared branch introduces a Statistical Difference Attention (SDA) module to actively separate and refine high-energy thermal targets from the background. Concurrently, the visible branch employs a Scale Difference Large Kernel (SDLK) module to capture and enhance multi-scale high-frequency textures. To prevent spatial confusion during feature integration, we design a lateral branch with a Local Attention Module (LAM) to preserve unique features, and a Common-Unique Cross Aggregation Module (CUCAM) at the bottleneck to achieve deep semantic integration. Furthermore, the network is driven by a multi-task loss function combining feature decoupling, intensity fidelity, and salient gradient boosting to maximize structural detail and contrast preservation. Extensive experiments across multiple public datasets demonstrate that S2DFusion achieves state-of-the-art performance in both subjective visual quality and objective metrics. Moreover, evaluations on downstream YOLO object detection tasks validate that the proposed method significantly enhances detection accuracy and robustness in complex environments by preserving high-fidelity thermal targets and multi-scale details. Feature-level decomposition analysis further verifies that the learned common and unique representations exhibit the intended cross-modal similarity and intra-modal orthogonality. Sensitivity and perceptual-quality analyses show that the gradient amplification factor provides a controllable trade-off between edge enhancement and perceptual naturalness, while downstream evaluations with both YOLOv5s and YOLOv8s demonstrate that the benefit is not detector-specific.