Role of Language-Derived Semantics in Knowledge Distillation on Wearable Sensor Data

Abstract

Wearable sensor-based human activity recognition (HAR) has been widely studied for understanding human activities from time-series data. However, wearable sensor data exhibit substantial inter-individual variations, while the signal differences between activities can be relatively subtle. These characteristics make it challenging to learn effective representations for activity recognition. Considering these limitations, human cognition provides a useful motivation, as raw signals are interpreted together with prior knowledge and contextual expectations, suggesting that language-derived semantics can complement the information contained in time-series data. However, incorporating such semantic information into wearable systems requires careful consideration of the computational and memory constraints of edge devices. In this regard, knowledge distillation (KD) enables richer information to be exploited during training while retaining a compact student model for inference. Nevertheless, how language-derived semantics contribute to KD on wearable sensor data remains underexplored. In this study, we systematically investigate how textual semantics can be incorporated into KD through a Semantic Guidance Module (SGM), considering three different strategies: teacher-side (T-SG), student-side (S-SG), and both-side (B-SG). We examine these strategies through various evaluations, including analyses of teacher–student architectures, prompt types, and window lengths. Our findings demonstrate that language-derived semantics provide complementary knowledge for student learning and offer insights into how semantic information can be effectively incorporated into KD for wearable sensor-based HAR.