Reading trajectory analysis via multi-task gaze estimation for schizophrenia detection based on facial videos

Abstract

Schizophrenia is a serious psychiatric disorder characterized by cognitive dysfunction, abnormal reading behavior, and impaired oculomotor control. To support non-invasive and low-cost screening, this study proposes a multi-scale fusion framework that detects schizophrenia from facial videos recorded with an ordinary RGB camera during text-reading tasks. The framework introduces a multi-task attentional gaze estimator, termed MTAGaze, which jointly predicts eye state, iris key points, and gaze direction to maintain stable trajectory reconstruction under blink events and head-pose variations. OCR-guided semantic anchoring and spatio-temporal DBSCAN clustering are subsequently used to recover interpretable reading trajectories and fixation-event patterns. To further improve robustness, a variational autoencoder-based denoising module is integrated with evidence-based uncertainty estimation for multi-view decision fusion. The experimental results on MPIIGaze and EYEDIAP show that MTAGaze achieves mean 3D angular errors of 3.11° and 2.55°, respectively. On the schizophrenia reading-task dataset, the complete framework achieves an accuracy of 97.08%, indicating its potential as an interpretable and cost-effective auxiliary screening tool.