Development and Validation of a Dynamic Risk Prediction Model to Forecast Psychosis Onset in Patients at Clinical High Risk

Development and temporal validation of a clinical prediction model of transition to psychosis in individuals at ultra-high risk in the UHR 1000+ cohort

The concept of ultra-high risk for psychosis (UHR) has been at the forefront of psychiatric research for several decades, with the ultimate goal of preventing the onset of psychotic disorder in high-risk individuals. Orygen (Melbourne, Australia) has led a range of observational and intervention studies in this clinical population. These datasets have now been integrated into the UHR 1000+ cohort, consisting of a sample of 1,245 UHR individuals with a follow-up period ranging from 1 to 16.7 years. This paper describes the cohort, presents a clinical prediction model of transition to psychosis in this cohort, and examines how predictive performance is affected by changes in UHR samples over time. We analyzed transition to psychosis using a Cox proportional hazards model. Clinical predictors for transition to psychosis were investigated in the entire cohort using multiple imputation and Rubin's rule. To assess performance drift over time, data from 1995-2016 were used for initial model fitting, and models were subsequently validated on data from 2017-2020. Over the follow-up period, 220 cases (17.7%) developed a psychotic disorder. Pooled hazard ratio (HR) estimates showed that the Comprehensive Assessment of At-Risk Mental States (CAARMS) Disorganized Speech subscale severity score (HR=1.12, 95% CI: 1.02-1.24, p=0.024), the CAARMS Unusual Thought Content subscale severity score (HR=1.13, 95% CI: 1.03-1.24, p=0.009), the Scale for the Assessment of Negative Symptoms (SANS) total score (HR=1.02, 95% CI: 1.00-1.03, p=0.022), the Social and Occupational Functioning Assessment Scale (SOFAS) score (HR=0.98, 95% CI: 0.97-1.00, p=0.036), and time between onset of symptoms and entry to UHR service (log transformed) (HR=1.10, 95% CI: 1.02-1.19, p=0.013) were predictive of transition to psychosis. UHR individuals who met the brief limited intermittent psychotic symptoms (BLIPS) criteria had a higher probability of transitioning to psychosis than those who met the attenuated psychotic symptoms (APS) criteria (HR=0.48, 95% CI: 0.32-0.73, p=0.001) and those who met the Trait risk criteria (a first-degree relative with a psychotic disorder or a schizotypal personality disorder plus a significant decrease in functioning during the previous year) (HR=0.43, 95% CI: 0.22-0.83, p=0.013). Models based on data from 1995-2016 displayed good calibration at initial model fitting, but showed a drift of 20.2-35.4% in calibration when validated on data from 2017-2020. Large-scale longitudinal data such as those from the UHR 1000+ cohort are required to develop accurate psychosis prediction models. It is critical to assess existing and future risk calculators for temporal drift, that may reduce their utility in clinical practice over time.

Dynamic Prediction of Outcomes for Youth at Clinical High Risk for Psychosis: A Joint Modeling Approach

Importance

Leveraging the dynamic nature of clinical variables in the clinical high risk for psychosis (CHR-P) population has the potential to significantly improve the performance of outcome prediction models.

Objective

To improve performance of prediction models and elucidate dynamic clinical profiles using joint modeling to predict conversion to psychosis and symptom remission.

Design, Setting, and Participants

Data were collected as part of the third wave of the North American Prodrome Longitudinal Study (NAPLS 3), which is a 9-site prospective longitudinal study. Participants were individuals aged 12 to 30 years who met criteria for a psychosis-risk syndrome. Clinical, neurocognitive, and demographic variables were collected at baseline and at multiple follow-up visits, beginning at 2 months and up to 24 months. An initial feature selection process identified longitudinal clinical variables that showed differential change for each outcome group across 2 months. With these variables, a joint modeling framework was used to estimate the likelihood of eventual outcomes. Models were developed and tested in a 10-fold cross-validation framework. Clinical data were collected between February 2015 and November 2018, and data were analyzed from February 2022 to December 2023.

Main Outcomes and Measures

Prediction models were built to predict conversion to psychosis and symptom remission. Participants met criteria for conversion if their positive symptoms reached the fully psychotic range and for symptom remission if they were subprodromal on the Scale of Psychosis-Risk Symptoms for a duration of 6 months or more.

Results

Of 488 included NAPLS 3 participants, 232 (47.5%) were female, and the mean (SD) age was 18.2 (3.4) years. Joint models achieved a high level of accuracy in predicting conversion (balanced accuracy [BAC], 0.91) and remission (BAC, 0.99) compared with baseline models (conversion: BAC, 0.65; remission: BAC, 0.60). Clinical variables that showed differential change between outcome groups across a 2-month span, including measures of symptom severity and aspects of functioning, were also identified. Further, intra-individual risks for each outcome were more negatively correlated when using joint models (r = -0.92; P < .001) compared with baseline models (r = -0.50; P < .001).

Conclusions and Relevance

In this study, joint models significantly outperformed baseline models in predicting both conversion and remission, demonstrating that monitoring short-term clinical change may help to parse heterogeneous dynamic clinical trajectories in a CHR-P population. These findings could inform additional study of targeted treatment selection and could move the field closer to clinical implementation of prediction models.

Early expressions of psychopathology and risk associated with trans-diagnostic transition to mood and psychotic disorders in adolescents and young adults

OBJECTIVES

The heterogeneity and comorbidity of major mental disorders presenting in adolescents and young adults has fostered calls for trans-diagnostic research. This study examines early expressions of psychopathology and risk and trans-diagnostic caseness in a community cohort of twins and non-twin siblings.

METHODS

Using data from the Brisbane Longitudinal Twin Study, we estimated median number of self-rated psychiatric symptoms, prevalence of subthreshold syndromes, family history of mood and/or psychotic disorders, and likelihood of subsequent trans-diagnostic caseness (individuals meeting diagnostic criteria for mood and/or psychotic syndromes). Next, we used cross-validated Chi-Square Automatic Interaction Detector (CHAID) analyses to identify the nature and relative importance of individual self-rated symptoms that predicted trans-diagnostic caseness. We examined the positive and negative predictive values (PPV; NPV) and accuracy of all classifications (Area under the Curve and 95% confidence intervals: AUC; 95% CI).

RESULTS

Of 1815 participants (Female 1050, 58%; mean age 26.40), more than one in four met caseness criteria for a mood and/or psychotic disorder. Examination of individual factors indicated that the AUC was highest for subthreshold syndromes, followed by family history then self-rated psychiatric symptoms, and that NPV always exceeded PPV for caseness. In contrast, the CHAID analysis (adjusted for age, sex, twin status) generated a classification tree comprising six trans-diagnostic symptoms. Whilst the contribution of two symptoms (need for sleep; physical activity) to the model was more difficult to interpret, CHAID analysis indicated that four self-rated symptoms (sadness; feeling overwhelmed; impaired concentration; paranoia) offered the best discrimination between cases and non-cases. These four symptoms showed different associations with family history status.

CONCLUSIONS

The findings need replication in independent cohorts. However, the use of CHAID might provide a means of identifying specific subsets of trans-diagnostic symptoms representing clinical phenotypes that predict transition to caseness in individuals at risk of onset of major mental disorders.

A predictive paradigm for COVID-19 prognosis based on the longitudinal measure of biomarkers

Novel coronavirus disease 2019 (COVID-19) is an emerging, rapidly evolving crisis, and the ability to predict prognosis for individual COVID-19 patient is important for guiding treatment. Laboratory examinations were repeatedly measured during hospitalization for COVID-19 patients, which provide the possibility for the individualized early prediction of prognosis. However, previous studies mainly focused on risk prediction based on laboratory measurements at one time point, ignoring disease progression and changes of biomarkers over time. By using historical regression trees (HTREEs), a novel machine learning method, and joint modeling technique, we modeled the longitudinal trajectories of laboratory biomarkers and made dynamically predictions on individual prognosis for 1997 COVID-19 patients. In the discovery phase, based on 358 COVID-19 patients admitted between 10 January and 18 February 2020 from Tongji Hospital, HTREE model identified a set of important variables including 14 prognostic biomarkers. With the trajectories of those biomarkers through 5-day, 10-day and 15-day, the joint model had a good performance in discriminating the survived and deceased COVID-19 patients (mean AUCs of 88.81, 84.81 and 85.62% for the discovery set). The predictive model was successfully validated in two independent datasets (mean AUCs of 87.61, 87.55 and 87.03% for validation the first dataset including 112 patients, 94.97, 95.78 and 94.63% for the second validation dataset including 1527 patients, respectively). In conclusion, our study identified important biomarkers associated with the prognosis of COVID-19 patients, characterized the time-to-event process and obtained dynamic predictions at the individual level.

Using Natural Language Processing on Electronic Health Records to Enhance Detection and Prediction of Psychosis Risk

BACKGROUND

Using novel data mining methods such as natural language processing (NLP) on electronic health records (EHRs) for screening and detecting individuals at risk for psychosis.

METHOD

The study included all patients receiving a first index diagnosis of nonorganic and nonpsychotic mental disorder within the South London and Maudsley (SLaM) NHS Foundation Trust between January 1, 2008, and July 28, 2018. Least Absolute Shrinkage and Selection Operator (LASSO)-regularized Cox regression was used to refine and externally validate a refined version of a five-item individualized, transdiagnostic, clinically based risk calculator previously developed (Harrell's C = 0.79) and piloted for implementation. The refined version included 14 additional NLP-predictors: tearfulness, poor appetite, weight loss, insomnia, cannabis, cocaine, guilt, irritability, delusions, hopelessness, disturbed sleep, poor insight, agitation, and paranoia.

RESULTS

A total of 92 151 patients with a first index diagnosis of nonorganic and nonpsychotic mental disorder within the SLaM Trust were included in the derivation (n = 28 297) or external validation (n = 63 854) data sets. Mean age was 33.6 years, 50.7% were women, and 67.0% were of white race/ethnicity. Mean follow-up was 1590 days. The overall 6-year risk of psychosis in secondary mental health care was 3.4 (95% CI, 3.3-3.6). External validation indicated strong performance on unseen data (Harrell's C 0.85, 95% CI 0.84-0.86), an increase of 0.06 from the original model.

CONCLUSIONS

Using NLP on EHRs can considerably enhance the prognostic accuracy of psychosis risk calculators. This can help identify patients at risk of psychosis who require assessment and specialized care, facilitating earlier detection and potentially improving patient outcomes.

Prediction and Prevention in the Clinical High-Risk for Psychosis Paradigm: A Review of the Current Status and Recommendations for Future Directions of Inquiry

Prediction and prevention of negative clinical and functional outcomes represent the two primary objectives of research conducted within the clinical high-risk for psychosis (CHR-P) paradigm. Several multivariable "risk calculator" models have been developed to predict the likelihood of developing psychosis, although these models have not been translated to clinical use. Overall, less progress has been made in developing effective interventions. In this paper, we review the existing literature on both prediction and prevention in the CHR-P paradigm and, primarily, outline ways in which expanding and combining these paths of inquiry could lead to a greater improvement in individual outcomes for those most at risk.

Real-world digital implementation of the Psychosis Polyrisk Score (PPS): A pilot feasibility study

BACKGROUND

The Psychosis Polyrisk Score (PPS) is a potential biomarker integrating non-purely genetic risk/protective factors for psychosis that may improve identification of individuals at risk and prediction of their outcomes at the individual subject level. Biomarkers that are easy to administer are direly needed in early psychosis to facilitate clinical implementation. This study digitally implements the PPS and pilots its feasibility of use in the real world.

METHODS

The PPS was implemented digitally and prospectively piloted across individuals referred for a CHR-P assessment (n = 16) and healthy controls (n = 66). Distribution of PPS scores was further simulated in the general population.

RESULTS

98.8% of individuals referred for a CHR-P assessment and healthy controls completed the PPS assessment with only one drop-out. 96.3% of participants completed the assessment in under 15 min. Individuals referred for a CHR-P assessment had high PPS scores (mean = 6.2, SD = 7.23) than healthy controls (mean = -1.79, SD = 6.78, p < 0.001). In simulated general population data, scores were normally distributed ranging from -15 (lowest risk, RR = 0.03) to 39.5 (highest risk, RR = 8912.51).

DISCUSSION

The PPS is a promising biomarker which has been implemented digitally. The PPS can be easily administered to both healthy controls and individuals at potential risk for psychosis on a range of devices. It is feasible to use the PPS in real world settings to assess individuals with emerging mental disorders. The next phase of research should be to include the PPS in large-scale international cohort studies to evaluate its ability to refine the prognostication of outcomes.

Early Intervention Services for First Episode of Psychosis in South London and the Maudsley (SLaM): 20 Years of Care and Research for Young People

Introduction: Early Intervention for a first episode of Psychosis (EI) is essential to improve outcomes. There is limited research describing real-world implementation of EI services. Method: Analysis of service characteristics, outcomes (described through a retrospective 2007-2017 Electronic Health Record (EHR) cohort study) and clinical research relating to the first 20 years of implementation of EI services in South London and Maudsley (SLaM) Trust. Results: SLaM EI are standalone services serving 443,050 young individuals in South-London, where (2017) incidence of psychosis (58.3-71.9 cases per 100,000 person-years) is greater than the national average. From 2007-2017 (when the EHR was established), 1,200 individuals (62.67% male, mean age 24.38 years, 88.17% single; two-thirds of non-white ethnicity) received NICE-compliant EI care. Pathways to EI services came mainly (75.26%) through inpatient (39.83%) or community (19.33%) mental health services or Accident and Emergency departments (A&E) (16%). At 6 year follow-up 34.92% of patients were still being prescribed antipsychotics. The 3 month and 6 year cumulative proportions of those receiving clozapine were 0.75 and 7.33%; those compulsorily admitted to psychiatric hospitals 26.92 and 57.25%; those admitted to physical health hospitals 6.83 and 31.17%, respectively. Average 3 months and 6 year days spent in hospital were 0.82 and 1.85, respectively; mean 6 year attendance at A&E was 3.01. SLaM EI clinical research attracted £58 million grant income and numerous high-impact scientific publications. Conclusions: SLaM EI services represent one of the largest, most established services of its kind, and are a leading model for development of similar services in the UK and worldwide.

Progression from being at-risk to psychosis: next steps

Over the past 20 years there has been a great deal of research into those considered to be at risk for developing psychosis. Much has been learned and studies have been encouraging. The aim of this paper is to offer an update of the current status of research on risk for psychosis, and what the next steps might be in examining the progression from CHR to psychosis. Advances have been made in accurate prediction, yet there are some methodological issues in ascertainment, diagnosis, the use of data-driven selection methods and lack of external validation. Although there have been several high-quality treatment trials the heterogeneity of this clinical high-risk population has to be addressed so that their treatment needs can be properly met. Recommendations for the future include more collaborative research programmes, and ensuring they are accessible and harmonized with respect to criteria and outcomes so that the field can continue to move forward with the development of large collaborative consortiums as well as increased funding for multisite projects.

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful system for supporting the early detection of individuals at-risk of psychosis at a large scale, by leveraging electronic health records (EHRs). This risk calculator has been externally validated twice and is undergoing feasibility testing for clinical implementation. Integration of this risk calculator in clinical routine should be facilitated by prospective feasibility studies, which are required to address pragmatic challenges, such as missing data, and the usability of this risk calculator in a real-world and routine clinical setting. Here, we present an approach for a prospective implementation of a real-time psychosis risk detection and alerting service in a real-world EHR system. This method leverages the CogStack platform, which is an open-source, lightweight, and distributed information retrieval and text extraction system. The CogStack platform incorporates a set of services that allow for full-text search of clinical data, lifespan-inclusive, real-time calculation of psychosis risk, early risk-alerting to clinicians, and the visual monitoring of patients over time. Our method includes: 1) ingestion and synchronization of data from multiple sources into the CogStack platform, 2) implementation of a risk calculator, whose algorithm was previously developed and validated, for timely computation of a patient's risk of psychosis, 3) creation of interactive visualizations and dashboards to monitor patients' health status over time, and 4) building automated alerting systems to ensure that clinicians are notified of patients at-risk, so that appropriate actions can be pursued. This is the first ever study that has developed and implemented a similar detection and alerting system in clinical routine for early detection of psychosis.