InterMine: A foundation model-based multi-task interaction network for remote sensing interpretation of open-pit coal mining activity

Abstract

The interpretation of open-pit coal mining activity is critical for resource management and ecological protection. It comprises two key tasks: extent extraction and fine-grained change detection. However, existing methods usually suffer from task separation, and their outputs cannot fully meet the requirements of interpretation. Therefore, this study proposes a foundation model-based multi-task interaction network (InterMine) to jointly perform open-pit mine extent extraction and fine-grained change detection. First, InterMine employs a Siamese foundation model as the encoder to enhance feature representation from images of complex open-pit coal mines and incorporates a channel-spatial feature reconstruction module to adapt the encoded features to meet the semantic and spatial requirements of the decoder. Second, a multi-task interaction mechanism is proposed to facilitate complementary information exchange between tasks. Finally, a consistency loss function based on hard-sample reweighting is designed to mitigate inconsistencies among multi-task outputs. InterMine is trained and evaluated on a self-constructed multi-temporal open-pit coal mining activity interpretation dataset (OMSet). Experimental results demonstrate that InterMine outperforms the compared methods. In particular, InterMine achieves F1-scores of 87.57 %, 83.68 %, and 79.43 % for open-pit mine extent extraction, excavation change detection, and reclamation change detection, respectively, outperforming the comparison methods by 1.89 ∼ 5.29, 3.04 ∼ 7.83, and 2.79 ∼ 7.42 percentage points, respectively. This work provides both technical and data support for open-pit coal mine monitoring. OMSet and the code of InterMine have been made publicly available at https://figshare.com/s/03c029d8e4846faef69f .