Do not smile when acquiring consumer selfies for multi-attribute skin profiling and baseline deep learning evaluation

Abstract

Abstract The increasing demand for personalized skincare solutions highlights a significant gap: many consumers struggle to find suitable products without professional guidance. While the commercial potential for tailored product recommendations is vast, a key challenge remains the lack of effective methods for skin profiling via image classification. To address this challenge, this paper introduces a comprehensive benchmark dataset of 3203 standardized facial consumer selfies, annotated across eight primary cosmetic skin features. We establish a transparent baseline evaluation utilizing the open-source medical image classification framework AUCMEDI. A variety of deep learning architectures were trained to recognize and classify various skin features, including sagging skin, wrinkles, under-eye circles, redness, shine, pigment spots, acne, and pore size. The baseline model’s performance was evaluated using the mean absolute error ( e ), which appropriately accounts for the ordinal distance in cosmetic grading. Utilizing standard deep learning architectures, the baseline benchmark established promising performance in structural categories like sagging skin ( $$e = 1.71$$ ) and wrinkles ( $$e = 1.40$$ ). While achieving satisfactory results for under-eye circles ( $$e = 0.32$$ ), redness ( $$e = 2.22$$ ), and shine ( $$e = 0.23$$ ), the baseline models encountered significant architectural limitations when resolving highly localized or imbalanced features such as acne ( $$e = 2.59$$ ), pore size ( $$e = 2.57$$ ), and pigment spots ( $$e = 2.20$$ ), the latter three underperforming a trivial constant-mean predictor.