Intelligent Programmable Metasurfaces for Robust Spatiotemporal 3D Human Pose Estimation

Abstract

ABSTRACT Programmable metasurfaces (PMs) provide a compelling platform for human sensing under conditions where vision‐based methods fail. However, conventional metasurface sensing relies exclusively on electromagnetic scattering data captured at the isolated and current time instant. Lacking integration with the historical temporal context, these single‐frame approaches suffer when severe indoor multipath fading distorts the transient scattering signatures, leading to estimation failures during deep fading events. To address this limitation, an intelligent PM system is proposed utilizing a multi‐frame spatiotemporal fusion approach, which treats human motion as a continuous evolution process. By exploiting the temporal dependencies across sequential frames through a Spatiotemporal Metasurface Pose Network (ST‐MetaPose), the system autonomously compensates for deeply faded instantaneous observation using kinematic context. For experimental validation, a prototype system featuring a 1‐bit PM consisting of a 30 × 30 array operating at 11.4 GHz is implemented. Experimental results demonstrate robust, continuous three‐dimensional 18‐keypoint pose estimation at 20 Hz, achieving a mean per‐joint position error of 0.078 m. This framework maintains high estimation stability even during destructive multipath fading, opening new avenues for intelligent electromagnetic surveillance and robust human‐machine interaction.