Beyond Alliance Scores: Construct Transport and Measurement Validity in Mental Health Chatbot Research—A Systematic Scoping Review

Abstract

Therapeutic alliance is increasingly measured in mental health chatbot research, although many instruments originated in human psychotherapy. This systematic scoping review examined alliance conceptualization, operationalization, adaptation, and evaluation with artificial conversational agents. Searches identified 40 reports from 38 independent studies. We traced instrument genealogy and adaptation, mapped chatbot-context measurement evidence using a COSMIN-informed approach, and appraised construct transport with a review-specific rubric locked before independent rerating. Twenty-nine studies used Working Alliance Inventory or Working Alliance Version measures, with limited modification. Evidence was limited; no study met criteria for a sufficient positive conclusion on a mapped property. Under the locked primary construct-transport rule, none met the Adequate threshold; 25 were Questionable and 13 Inadequate, although sensitivity analyses showed the zero-Adequate boundary was not absolute. Users nevertheless reported favorable alliance-like or supportive relational experiences. Current scores indicate context-specific user-reported relational experience, not established equivalence with human-psychotherapy alliance.