A multi-modal model for trust evaluation in industrial human–robot collaboration

Abstract

Abstract Industry 5.0 promotes collaborative manufacturing environments where humans and robots work together, leveraging their complementary strengths to improve productivity and safety. In such settings, trust is fundamental to ensuring seamless and effective interaction. As intelligent robots take on more complex roles, systems must be capable of adapting to changes in human trust to maintain safe and efficient operations. However, most existing trust assessment methods rely on post hoc questionnaires, which do not capture observable behavioral cues during interaction that influence human decision-making, workload, and task performance. This study addresses this limitation by introducing a multi-modal, data-driven machine learning model for classifying behavioral patterns associated with distinct trust states under controlled trust-inducing conditions. The framework integrates facial expression features extracted using a CNN model with body motion indicators processed through kinematic models. The model was evaluated in a chemical industry scenario, where a robotic manipulator supports operators in handing and mixing hazardous materials. It achieved 88.40% accuracy (AUC = 0.97) during the handing task and 85.30% accuracy (AUC = 0.94) during the pouring task. These findings demonstrate the effectiveness of multi-modal behavior analysis and sensor data fusion for accurate trust classification in human–robot collaboration.