Synthetic data-driven framework for estimating worker activity intensity using pose-based graph neural networks

Abstract

This paper proposes a metabolic equivalent of task (MET) class prediction framework that combines Stable Diffusion-generated synthetic images with B-ResSkelGCN, which learns from skeleton graphs, for non-contact evaluation of construction workers' workload levels. Using domain-informed prompts, synthetic images were generated, followed by worker region of interest (ROI) alignment and skeleton transformation to construct graph convolutional network (GCN) inputs. The trained B-ResSkelGCN achieved high MET class prediction performance on frames in a test environment. However, predictions from single frames in real videos fluctuated during action transitions or momentary stillness, such as walking. To address this issue, mode-based window integration aggregated predictions within each window. A 5-frame window improved classification performance and MET error metrics compared with single-frame prediction, whereas excessively long windows diluted action-transition information and increased confusion, highlighting the need for operational strategies that adjust integration intervals according to action-transition characteristics and management objectives.