Statistically grounded evaluation of demographic bias in deep facial representations using FairFace and UTKFace

Abstract

Abstract Deep facial representation learning has achieved remarkable performance in computer vision applications; however, high overall accuracy does not necessarily ensure equitable performance across demographic groups. This study presents a unified framework for evaluating the quality and demographic fairness of pretrained deep facial representations. ResNet50, MobileNetV3, and the Data-efficient Image Transformer (DeiT) were evaluated using ImageNet-pretrained weights on the FairFace and UTKFace datasets, representing balanced and unconstrained demographic settings, respectively. Feature embeddings were assessed using one-nearest neighbour (1-NN) retrieval to measure the intrinsic discriminative capability of the representation space and Logistic Probe classification to evaluate the linear separability of demographic information. Fairness was examined through overall accuracy, subgroup performance, race–gender intersectional analysis, fairness gap, coefficient of variation, and statistical significance testing. Logistic Probe classification consistently outperformed 1-NN retrieval across both datasets, indicating that pretrained facial representations encode demographic information that can be effectively separated using simple linear classifiers. ResNet50 achieved the highest overall classification performance, whereas MobileNetV3 demonstrated more consistent subgroup performance in several evaluation settings. Intersectional analysis revealed persistent demographic disparities despite improvements in overall accuracy, underscoring the importance of fairness evaluation beyond aggregate metrics. The proposed framework provides a reproducible methodology for jointly assessing representation quality, demographic fairness, and statistical significance in deep facial representation learning.