Can we improve prediction of psychotherapy outcomes through pretraining with simulated data?

Abstract

Objective In the context of personalized medicine, machine learning (ML) algorithms are gaining popularity. These algorithms require substantial information, which can be acquired effectively through the use of previously gathered data. Open data and the utilization of synthetization techniques have been proposed to address this need. In this article, we propose and evaluate an alternative approach that uses additional simulated data based on summary statistics published in the literature. Method The simulated data were used to pretrain random forests, which were afterward fine-tuned using a real dataset. Here, these terms describe combining trees trained separately on simulated and real data, without updating pretrained trees. We compared the predictive performance of the new approach with that of random forests trained only on real data. A Monte Carlo cross-validation (MCCV) framework with 100 iterations was employed to investigate the significance and stability of the results. Since the first study yielded inconclusive results, a second study with an improved methodology (i.e., systematic information extraction and a different prediction outcome) was conducted. Results In Study 1, some pretrained random forests descriptively outperformed the standard random forest. However, this improvement was not significant ( t (99) = 0.89, p = 0.19). Contrary to expectations, in Study 2, the random forest trained only on the real data outperformed the pretrained random forests. Conclusion We conclude with a discussion of challenges, such as the scarcity of informative publications, and recommendations for future research.