Feature-Level Cloud Guidance for Sentinel-2 Imagery Reconstruction in SAR-Optical Fusion

Abstract

Cloud contamination remains a major limitation in optical remote sensing because clouds and cloud shadows obscure or distort land-surface reflectance information.Sentinel-2 imagery provides rich multispectral information for land-cover mapping, environmental monitoring, agricultural assessment, and disaster management; however, its usability is often reduced under cloudy atmospheric conditions.Sentinel-1 synthetic aperture radar (SAR) imagery can support cloud removal because it provides structural and scattering-related information under all-weather and day-and-night conditions.Recent SAR-optical fusion models, such as the Heterogeneous Parallel Network for Cloud Removal (HPN-CR), have improved cloud-free Sentinel-2 reconstruction by combining SAR and optical features.However, many existing methods do not explicitly use pixel-level cloud-condition information during intermediate feature learning, even though clear regions, thin clouds, cloud shadows, and thick clouds have different reconstruction difficulties.This study proposes an OmniCloudMask (OCM)-guided feature modulation approach for cloud-aware SAR-optical Sentinel-2 image reconstruction.OCM is first used to generate pixel-level cloud and shadow information from cloud-contaminated Sentinel-2 images.The OCM class map is then converted into a cloud-severity representation and used as feature-level guidance inside the optical branch of HPN-CR.This allows intermediate optical features to be recalibrated according to cloud conditions before SAR-optical fusion.The proposed model is evaluated using a subset of the SEN12MS-CR dataset containing paired Sentinel-1 SAR images, cloud-contaminated Sentinel-2 images, and cloud-free Sentinel-2 reference images.Three configurations are compared: the original HPN-CR baseline, an OCM-guided weighted-loss model, and the proposed OCM-guided feature modulation model.The experimental results show that the proposed method achieves the best overall test performance.Compared with the original HPN-CR baseline on the selected SEN12MS-CR subset used in this study, the proposed model improves peak signal-to-noise ratio from 27.7652 dB to 27.9613 dB, reduces spectral angle mapper from 10.0691 to 9.7344, and reduces mean absolute error from 0.0317 to 0.0304, while maintaining nearly the same structural similarity; a class-stratified analysis further shows that reconstruction accuracy varies systematically across clear, thin-cloud, thick-cloud, and shadow regions.The proposed approach demonstrates that cloudaware feature modulation can improve SAR-optical cloud removal and provides a useful direction for future cloud-guided multimodal remote sensing reconstruction models.