Biovid dataset and challenge: biometric user recognition on smartphone data in real-world conditions

Abstract

Abstract User authentication is more and more crucial in everyday life, with researchers developing sophisticated, yet easy-to-use, approaches to enforce secure authentication while providing a smooth user experience. This is particularly true in mobile authentication, in which the spread of AI-capable smartphones is opening up several biometric-based authentication methods, such as face and lips recognition. A more recently emerging setup consists of the matching of a spoken passphrase with lips movements. Despite being promising and well-suited for mobility authentication, the matching operation is not trivial, and the research around this topic is limited, as no publicly available dataset is available for experimental purposes. In this paper we thus introduce BioVid , a novel privacy-aware multimodal biometric dataset for user recognition from smartphone-recorded videos in real-world conditions. BioVid consists of 650 recordings from 43 participants, where each clip captures a user pronouncing predefined passphrases while simultaneously acquiring audio and lips movements. The dataset is designed to support dual-factor biometric authentication, combining speaker-unique identity (UID) and passphrase verification, and explicitly models realistic variability in device type, recording distance, background, and ambient noise. Together with the data, we provide a detailed statistical analysis of the linguistic, visual, and acoustic properties of the data, highlighting its diversity and suitability for robust biometric system evaluation. Moreover, to test the quality of the collected dataset, we organised an international challenge on multimodal user authentication leveraging BioVid . In this paper we thus also describe the challenge protocol, baseline systems, and evaluation metrics, while also summarising the best-performing submitted approaches as measured by Equal Error Rate (EER). BioVid and the accompanying challenge aim to establish a reproducible benchmark for multimodal biometric authentication on consumer-grade smartphones and to foster further research on privacy-aware, real-world biometric systems.