Psychotherapy remission prediction models show limited cross-clinic generalizability in routine outpatient care
Abstract
Abstract Clinical prediction models for psychotherapy outcomes have the potential to assist therapists in planning a patient’s individual treatment, but their clinical utility depends on maintaining predictive performance across settings. We assessed prediction of post-treatment remission in 2,002 patients with internalizing disorders treated across eight German university outpatient clinics. Penalized logistic-regression models were evaluated within clinics, in pooled clinic data, and using leave-one-clinic-out validation. Within clinics, mean AUC was 0.67, balanced accuracy 0.62, and Brier score 0.21. In leave-one-clinic-out validation, mean AUC and balanced accuracy declined from 0.60 and 0.57 in corresponding held-out test samples to 0.53 and 0.52 in held-out clinics, respectively. The mean Brier score increased from 0.23 to 0.25, but this difference was not statistically significant (p = 0.18). Cross-clinic performance loss was therefore clearest for discrimination and threshold-dependent classification, while overall probabilistic prediction error showed a smaller change. These findings highlight the broader challenge of transporting psychotherapy outcome predictions across heterogeneous clinical settings.
// Source
Authors: Peter Weller, Kevin Hilbert, Georg W. Alpers, Stephan Bartholdy, Elisa Berger, Eva-Lotta Brakemeier, Anne‐Kathrin Bräscher, Timo Brockmeyer, Hanna Christiansen, Monika Equit, Lydia Fehm, Thomas Forkmann, Julia Anna Glombiewski, Sylvia Helbig‐Lang, Andrea Hermann, Jürgen Hoyer, Tina In-Albon, Anke Kirsch, Tim Klucken, Tania Lincoln, Wolfgang Lutz, Jürgen Margraf, Patrizia Odyniec, Anya Pedersen, Anne-Katrin Risch, Julian Rubel, Almut Rudolph, Henning Schöttke, Lars Schulze, Brian Schwartz, Rudolf Stark, Tobias Teismann, Julia Velten, Katja Werheid, Ulrike Willutzki, Gabriele Wilz, Ulrike Lueken
Institutions: Leipzig University, University of Erfurt, University of Münster, Universität Hamburg, Justus-Liebig-Universität Gießen, Freie Universität Berlin, Philipps University of Marburg, Humboldt-Universität zu Berlin, Johannes Gutenberg University Mainz, Saarland University, University Hospital Leipzig, Friedrich Schiller University Jena, University of Duisburg-Essen, Bielefeld University, University of Mannheim, Technische Universität Dresden, University of Kaiserslautern, FH Münster, University of Applied Sciences Kaiserslautern, Universität Greifswald, University Medical Center of the Johannes Gutenberg University Mainz, GFZ Helmholtz Centre for Geosciences, Trier University of Applied Sciences, Universität Trier, Christian-Albrechts-Universität zu Kiel, Osnabrück University, University of Siegen, Witten/Herdecke University, Nephrologisches Zentrum Goettingen