Unified Embedding and Composable Fusion for Arbitrary Multimodal Person Re-Identification

Abstract

Driven by the advancement of intelligent security, person re-identification has evolved from single-modality imagery to complex multimodal tasks involving infrared, text, sketches, and paintings. However, most existing methods are restricted to fixed modality pairs and struggle to accommodate the flexible fusion requirements of variable inputs, making the integration of arbitrary modality combinations a critical challenge. To address this, we propose the Unified Embedding Representation Fusion Network (UERF-Net), a framework based on unified embedding representation fusion. This method first establishes a unified embedding space to map heterogeneous modalities into a shared representation, effectively narrowing the modality gap through an explicit pre-alignment strategy. It then introduces a composable fusion mechanism utilizing a joint self-attention module to facilitate deep token-level interactions across diverse modality combinations. Furthermore, a distribution alignment strategy is employed to optimize similarity distributions via Supervised Distribution Alignment (SDA), significantly enhancing feature stability and identity discriminative power. Experimental results on the five-modality ORBench benchmark demonstrate that UERF-Net achieves state-of-the-art performance in Rank-1 accuracy (R1) and mean average precision (mAP), confirming the robustness and superior adaptability of our unified embedding-based modeling in complex, multi-source retrieval scenarios.