DCC-Transformer: a dense cross-connected Transformer leveraging multi-feature fusion for hyperspectral unmixing

Abstract

More and more deep learning models are being applied in the field of hyperspectral unmixing (HU), among which Transformer-based models have shown superior performance. However, Transformer-based models currently lack the ability to extract various feature information from hyperspectral images (HSIs). Additionally, how to effectively fuse these feature information remains a significant challenge. To more finely extract the various latent information of HSIs and effectively fuse the useful parts of feature information for the HU task, we propose a Dense Cross-Connected Transformer Leveraging Multi-Feature Fusion (DCC-Transformer). The model first uses a hybrid spatial and spectral feature block containing multiple 2D and 3D convolution layers to initially extract feature information from the HSI while removing redundant channels, allowing the model to focus on more useful spectral dimensions. Subsequently, the dense connect cross-Transformer for multi-feature fusion block is introduced to extract various potential spatial features of HSIs and effectively fuse them. Finally, a two-stage training method is utilized to enable the model to perform the HU task in a self-supervised manner, improving the model’s robustness. The model is tested on a simulated dataset and three real datasets, with experimental results demonstrating its superior performance.