Prediction Tool for Individual Outcome Trajectories Across the Next Year in First-Episode Psychosis in Coordinated Specialty Care

OnTrackNY: A public sector learning healthcare system for youth and young adults with early psychosis

Coordinated Specialty Care is a treatment model for youth and young adults experiencing early psychosis. OnTrackNY, an internationally recognized public sector learning healthcare system, operates 31 coordinated specialty care teams throughout New York State with oversight from an intermediary organization, OnTrack Central. As part of the National Institute of Mental Health Early Psychosis Intervention Network initiative, OnTrackNY utilizes a stakeholder engagement unit and a data science unit to support quality improvement. This article describes how OnTrack Central uses the Institute of Medicine's Group Health Cooperative learning healthcare system framework and learning loop approach to enhance racial equity in OnTrackNY through a multi-component quality improvement project. Qualitative interviews (N = 70) with OnTrackNY participants, families, and providers revealed experiences with racism and shared decision making and identified stakeholder-driven modifications to OnTrack Central's training and implementation approach. Modifications included stakeholder co-created shared decision making training modules for providers and ethnoracially minoritized participants, and a provider learning collaborative to promote shared decision making with ethnoracially diverse participants and families. Evaluation of the modules and collaborative identified barriers including limited time for providers to engage with training and a lack of confidence among trainers in delivering racial equity trainings. Results highlighted the need to adjust content and develop two workforce training programs focused on enhancing shared decision making along the continuum of OnTrackNY care. This article demonstrates how a public sector learning healthcare system can use a stakeholder-partnered approach to enhance the competency of coordinated specialty care workforce and promote equitable and data-driven care.

Psychosis Prognosis Predictor: A continuous and uncertainty-aware prediction of treatment outcome in first-episode psychosis

INTRODUCTION

Machine learning models have shown promising potential in individual-level outcome prediction for patients with psychosis, but also have several limitations. To address some of these limitations, we present a model that predicts multiple outcomes, based on longitudinal patient data, while integrating prediction uncertainty to facilitate more reliable clinical decision-making.

MATERIAL AND METHODS

We devised a recurrent neural network architecture incorporating long short-term memory (LSTM) units to facilitate outcome prediction by leveraging multimodal baseline variables and clinical data collected at multiple time points. To account for model uncertainty, we employed a novel fuzzy logic approach to integrate the level of uncertainty into individual predictions. We predicted antipsychotic treatment outcomes in 446 first-episode psychosis patients in the OPTiMiSE study, for six different clinical scenarios. The treatment outcome measures assessed at both week 4 and week 10 encompassed symptomatic remission, clinical global remission, and functional remission.

RESULTS

Using only baseline predictors to predict different outcomes at week 4, leave-one-site-out validation AUC ranged from 0.62 to 0.66; performance improved when clinical data from week 1 was added (AUC = 0.66-0.71). For outcome at week 10, using only baseline variables, the models achieved AUC = 0.56-0.64; using data from more time points (weeks 1, 4, and 6) improved the performance to AUC = 0.72-0.74. After incorporating prediction uncertainties and stratifying the model decisions based on model confidence, we could achieve accuracies above 0.8 for ~50% of patients in five out of the six clinical scenarios.

CONCLUSION

We constructed prediction models utilizing a recurrent neural network architecture tailored to clinical scenarios derived from a time series dataset. One crucial aspect we incorporated was the consideration of uncertainty in individual predictions, which enhances the reliability of decision-making based on the model's output. We provided evidence showcasing the significance of leveraging time series data for achieving more accurate treatment outcome prediction in the field of psychiatry.

Accuracy of machine learning methods in predicting prognosis of patients with psychotic spectrum disorders: a systematic review

OBJECTIVES

We aimed to examine the predictive accuracy of functioning, relapse or remission among patients with psychotic disorders, using machine learning methods. We also identified specific features that were associated with these clinical outcomes.

DESIGN

The methodology of this review was guided by the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.

DATA SOURCES

CINAHL, EMBASE, PubMed, PsycINFO, Scopus and ScienceDirect were searched for relevant articles from database inception until 21 November 2024.

ELIGIBILITY CRITERIA

Studies were included if they involved the use of machine learning methods to predict functioning, relapse and/or remission among individuals with psychotic spectrum disorders.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers screened the records from the database search. Risk of bias was evaluated using the Quality Assessment of Diagnostic Accuracy Studies tool from Cochrane. Synthesised findings were presented in tables.

RESULTS

23 studies were included in the review, which were mostly conducted in the west (91%). Predictive summary area under the curve values for functioning, relapse and remission were 0.63-0.92 (poor to outstanding), 0.45-0.95 (poor to outstanding), 0.70-0.79 (acceptable), respectively. Logistic regression and random forest were the best performing algorithms. Factors influencing outcomes included demographic (age, ethnicity), illness (duration of untreated illness, types of symptoms), functioning (baseline functioning, interpersonal relationships and activity engagement), treatment variables (use of higher doses of antipsychotics, electroconvulsive therapy), data from passive sensor (call log, distance travelled, time spent in certain locations) and online activities (time of use, use of certain words, changes in search frequencies and length of queries).

CONCLUSION

Machine learning methods show promise in the prediction of prognosis (specifically functioning, relapse and remission) of mental disorders based on relevant collected variables. Future machine learning studies may want to focus on the inclusion of a broader swathe of variables including ecological momentary assessments, with a greater amount of good quality big data covering longer longitudinal illness courses and coupled with external validation of study findings.

PROSPERO REGISTRATION NUMBER

The review was registered on PROSPERO, ID: CRD42023441108.

Shaping tomorrow's support: baseline clinical characteristics predict later social functioning and quality of life in schizophrenia spectrum disorder

PURPOSE

We aimed to explore the multidimensional nature of social inclusion (mSI) among patients diagnosed with schizophrenia spectrum disorder (SSD), and to identify the predictors of 3-year mSI and the mSI prediction using traditional and data-driven approaches.

METHODS

We used the baseline and 3-year follow-up data of 1119 patients from the Genetic Risk and Outcome in Psychosis (GROUP) cohort in the Netherlands. The outcome mSI was defined as clusters derived from combined analyses of thirteen subscales from the Social Functioning Scale and the brief version of World Health Organization Quality of Life questionnaires through K-means clustering. Prediction models were built through multinomial logistic regression (ModelMLR) and random forest (ModelRF), internally validated via bootstrapping and compared by accuracy and the discriminability of mSI subgroups.

RESULTS

We identified five mSI subgroups: "very low (social functioning)/very low (quality of life)" (8.58%), "low/low" (12.87%), "high/low" (49.24%), "medium/high" (18.05%), and "high/high" (11.26%). The mSI was robustly predicted by a genetic predisposition for SSD, premorbid adjustment, positive, negative, and depressive symptoms, number of met needs, and baseline satisfaction with the environment and social life. The ModelRF (61.61% [54.90%, 68.01%]; P =0.013) was cautiously considered outperform the ModelMLR (59.16% [55.75%, 62.58%]; P =0.994).

CONCLUSION

We introduced and distinguished meaningful subgroups of mSI, which were modestly predictable from baseline clinical characteristics. A possibility for early prediction of mSI at the clinical stage may unlock the potential for faster and more impactful social support that is specifically tailored to the unique characteristics of the mSI subgroup to which a given patient belongs.

Psychotic disorders as a framework for precision psychiatry

People with psychotic disorders can show marked interindividual variations in the onset of illness, responses to treatment and relapse, but they receive broadly similar clinical care. Precision psychiatry is an approach that aims to stratify people with a given disorder according to different clinical outcomes and tailor treatment to their individual needs. At present, interindividual differences in outcomes of psychotic disorders are difficult to predict on the basis of clinical assessment alone. Therefore, current research in psychosis seeks to build models that predict outcomes by integrating clinical information with a range of biological measures. Here, we review recent progress in the application of precision psychiatry to psychotic disorders and consider the challenges associated with implementing this approach in clinical practice.