From granularity to consensus: Mitigating sample defects for Remote Sensing Image–Text Retrieval

Abstract

Remote Sensing Image–Text Retrieval (RSITR) aims to retrieve remote sensing images or text descriptions when only one modality is available. Despite recent progress, existing RSITR methods often suffer from sample defects, which cause them to mistakenly treat semantically related pairs as negatives, thereby limiting the benefits of advanced feature representations. To tackle the sample defect issue, we propose a novel Dual-Granularity Consensus-guided False Negative Suppression Network (CFSN 2 ), which jointly improves cross-modal alignment and false negative (FN) learning. First, CFSN 2 extracts momentum-updated feature queues for both global and local representations, enabling temporally coherent and stable similarity estimation. Second, it models dual-granularity similarity distributions via Gaussian Mixture Models and detects potential FNs through probabilistic cross-granularity consensus. Finally, the detected FNs are dynamically reweighted based on their FN confidence, softly suppressing their gradients while preserving informative negatives for a balanced optimization process. Extensive experiments across three datasets show that CFSN 2 achieves strong overall retrieval performance, with R@1 scores of 19.12, 28.96, and 22.86 in image-to-text retrieval, and 15.02, 25.22, and 18.00 in text-to-image retrieval, respectively, compared with 26 baselines.