Data-driven personalization of just-in-time adaptive mental health intervention via two-stage reinforcement learning approach

Abstract

Most digital mental health interventions still deliver uniform content, without accounting for how a user’s fluctuating psychological state interacts with their stable individual characteristics. We propose a two-stage offline reinforcement learning (RL) framework, trained on data from a six-week micro-randomized trial (N=190), that jointly considers both the timing and the type of intervention delivery, with the goal of maximizing the next-day improvement in a composite score of depression, anxiety, and stress. Under fitted Q evaluation (FQE), the learned policy showed a higher estimated value than both the behavior policy and a rule-based policy, and this ordering was reproduced across six off-policy estimators and several robustness analyses. The learned policy exhibited patterns that generate hypotheses about clinically meaningful decision rules: it tended to recommend content when individuals appeared to retain psychological resources to engage with it, and, notably, it assigned a relatively high estimated value to positive-psychology content even in lower-distress states, a pattern consistent with the Broaden-and-Build theory that positive emotional experiences proactively build resilience. We therefore frame these findings as hypothesis-generating: the approach offers a data-driven and explainable route toward personalized intervention rules that warrants prospective validation.