Comparative Analysis of Multimodal GANs for Cross‐Modal Information Retrieval

Abstract

ABSTRACT Cross‐modal information systems have emerged as an important area of research that enables information retrieval in different modalities such as text, image and audio. Existing Cross‐modal retrieval methods focus on semantic consistency between modalities but rarely consider retrieval effectiveness in generative models. This study investigates the effectiveness of the Multimodal Generative Advertisement Network (MMGAN) in enhancing Cross‐modal Information Retrieval (CMIR) performance. Using benchmark datasets that include various modalities, it conducts a comprehensive analysis of MMGAN against several state‐of‐the‐art models, including traditional Generative Adversarial Networks (GAN) and Convolutional Neural Networks (CNN). The experimental evaluation considers both representation quality and cross‐modal retrieval effectiveness. Controlled unimodal benchmarks are used to examine latent representation quality, whereas MS‐COCO, containing paired image–text samples, serves as the primary benchmark for genuine cross‐modal retrieval. Retrieval performance is assessed using Mean Average Precision (mAP), Precision@K, Recall@K, and NDCG@K for image‐to‐text and text‐to‐image retrieval, while NMI, ARI, and clustering accuracy are employed as supplementary measures of representation and semantic grouping quality. The results demonstrate the effectiveness of the proposed MMGAN‐based framework in learning semantically aligned multimodal representations and improving cross‐modal retrieval performance. Furthermore, it analyses the strengths and limitations of MMGAN, informing readers of their potential applications and exploring future research directions in CMIR. It contributes to the advancement of generative models to improve Cross‐modal retrieval systems, emphasizing more efficient and effective information retrieval.