DeepPoseCare: a privacy-preserving pose-based temporal convolutional framework for real-time patient action recognition in healthcare environments

Abstract

Continuous and automated surveillance of patient activities in clinical environments holds great promise for the early identification of falls and distress in patients. However, current vision-based systems face obstacles such as privacy issues, the need for real-time processing, and complex environments that hinder their performance. We present DeepPoseCare, a privacy-preserving pose-based deep learning framework that integrates 2D skeletal keypoint extraction from RGB video with temporal convolutional modeling for real-time patient action recognition in healthcare environments. Video streams were anonymized by blurring facial areas, keeping only the skeletal keypoints (25 joints) with corresponding confidence scores. A Savitzky–Golay filter was used to smooth the joint trajectories before sequences of length \(T=32\) were fed into a four-block residual TCN, which produced per-frame action probabilities using softmax across four categories: Walk, Sit, Fall, and Distress. We tested DeepPoseCare on two public fall detection benchmarks (UPFall and URFall) and conducted five-fold cross-validation on simulated ward recordings. For UPFall, our model achieved 94.0% accuracy and 93.5% F 1 for falls, outperforming the state-of-the-art baselines by more than four percentage points. On URFall, the model achieved an accuracy of 93.5% with an F 1 score of 93.0%. The measured processing latency was 22 ms/frame on the RTX 3090 workstation. Because the symmetric five-frame Savitzky–Golay smoother requires two future frames, the buffering-adjusted streaming latency is 88.7 ms at 30 fps, which remains below the 500 ms clinical alert threshold. An ablation study showed that four TCN blocks optimized the balance between the F 1 gain and latency overhead. By incorporating effective pose-based feature extraction, temporal modeling, and strict privacy measures, DeepPoseCare achieves state-of-the-art performance in detecting falls and distress while adhering to the constraints of clinical deployment. These results highlight the potential of the system to improve patient safety through timely alerts, reduce staff workload, and seamlessly integrate with existing nurse-call systems via HL7 FHIR messaging.