Transdiagnostic individualized clinically-based risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: external replication in 2,430,333 US patients

Dynamic and Transdiagnostic Risk Calculator Based on Natural Language Processing for the Prediction of Psychosis in Secondary Mental Health Care: Development and Internal-External Validation Cohort Study

BACKGROUND

Automatic transdiagnostic risk calculators can improve the detection of individuals at risk of psychosis. However, they rely on assessment at a single point in time and can be refined with dynamic modeling techniques that account for changes in risk over time.

METHODS

We included 158,139 patients (5007 events) who received a first index diagnosis of a nonorganic and nonpsychotic mental disorder within electronic health records from the South London and Maudsley National Health Service Foundation Trust between January 1, 2008, and October 8, 2021. A dynamic Cox landmark model was developed to estimate the 2-year risk of developing psychosis according to the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis) statement. The dynamic model included 24 predictors extracted at 9 landmark points (baseline, 0, 6, 12, 24, 30, 36, 42, and 48 months): 3 demographic, 1 clinical, and 20 natural language processing-based symptom and substance use predictors. Performance was compared with a static Cox regression model with all predictors assessed at baseline only and indexed via discrimination (C-index), calibration (calibration plots), and potential clinical utility (decision curves) in internal-external validation.

RESULTS

The dynamic model improved discrimination performance from baseline compared with the static model (dynamic: C-index = 0.9; static: C-index = 0.87) and the final landmark point (dynamic: C-index = 0.79; static: C-index = 0.76). The dynamic model was also significantly better calibrated (calibration slope = 0.97-1.1) than the static model at later landmark points (≥24 months). Net benefit was higher for the dynamic than for the static model at later landmark points (≥24 months).

CONCLUSIONS

These findings suggest that dynamic prediction models can improve the detection of individuals at risk for psychosis in secondary mental health care settings.

A transdiagnostic prodrome for severe mental disorders: an electronic health record study

Effective prevention of severe mental disorders (SMD), including non-psychotic unipolar mood disorders (UMD), non-psychotic bipolar mood disorders (BMD), and psychotic disorders (PSY), rely on accurate knowledge of the duration, first presentation, time course and transdiagnosticity of their prodromal stages. Here we present a retrospective, real-world, cohort study using electronic health records, adhering to RECORD guidelines. Natural language processing algorithms were used to extract monthly occurrences of 65 prodromal features (symptoms and substance use), grouped into eight prodromal clusters. The duration, first presentation, and transdiagnosticity of the prodrome were compared between SMD groups with one-way ANOVA, Cohen's f and d. The time course (mean occurrences) of prodromal clusters was compared between SMD groups with linear mixed-effects models. 26,975 individuals diagnosed with ICD-10 SMD were followed up for up to 12 years (UMD = 13,422; BMD = 2506; PSY = 11,047; median[IQR] age 39.8[23.7] years; 55% female; 52% white). The duration of the UMD prodrome (18[36] months) was shorter than BMD (26[35], d = 0.21) and PSY (24[38], d = 0.18). Most individuals presented with multiple first prodromal clusters, with the most common being non-specific ('other'; 88% UMD, 85% BMD, 78% PSY). The only first prodromal cluster that showed a medium-sized difference between the three SMD groups was positive symptoms (f = 0.30). Time course analysis showed an increase in prodromal cluster occurrences approaching SMD onset. Feature occurrence across the prodromal period showed small/negligible differences between SMD groups, suggesting that most features are transdiagnostic, except for positive symptoms (e.g. paranoia, f = 0.40). Taken together, our findings show minimal differences in the duration and first presentation of the SMD prodromes as recorded in secondary mental health care. All the prodromal clusters intensified as individuals approached SMD onset, and all the prodromal features other than positive symptoms are transdiagnostic. These results support proposals to develop transdiagnostic preventive services for affective and psychotic disorders detected in secondary mental healthcare.

Using Electronic Health Records to Facilitate Precision Psychiatry

The use of clinical prediction models to produce individualized risk estimates can facilitate the implementation of precision psychiatry. As a source of data from large, clinically representative patient samples, electronic health records (EHRs) provide a platform to develop and validate clinical prediction models, as well as potentially implement them in routine clinical care. The current review describes promising use cases for the application of precision psychiatry to EHR data and considers their performance in terms of discrimination (ability to separate individuals with and without the outcome) and calibration (extent to which predicted risk estimates correspond to observed outcomes), as well as their potential clinical utility (weighing benefits and costs associated with the model compared to different approaches across different assumptions of the number needed to test). We review 4 externally validated clinical prediction models designed to predict psychosis onset, psychotic relapse, cardiometabolic morbidity, and suicide risk. We then discuss the prospects for clinically implementing these models and the potential added value of integrating data from evidence syntheses, standardized psychometric assessments, and biological data into EHRs. Clinical prediction models can utilize routinely collected EHR data in an innovative way, representing a unique opportunity to inform real-world clinical decision making. Combining data from other sources (e.g., meta-analyses) or enhancing EHR data with information from research studies (clinical and biomarker data) may enhance our abilities to improve the performance of clinical prediction models.

Recent Updates on Predicting Conversion in Youth at Clinical High Risk for Psychosis

PURPOSE OF REVIEW

This review highlights recent advances in the prediction and treatment of psychotic conversion. Over the past 25 years, research into the prodromal phase of psychotic illness has expanded with the promise of early identification of individuals at clinical high risk (CHR) for psychosis who are likely to convert to psychosis.

RECENT FINDINGS

Meta-analyses highlight conversion rates between 20 and 30% within 2-3 years using existing clinical criteria while research into more specific risk factors, biomarkers, and refinement of psychosis risk calculators has exploded, improving our ability to predict psychotic conversion with greater accuracy. Recent studies highlight risk factors and biomarkers likely to contribute to earlier identification and provide insight into neurodevelopmental abnormalities, CHR subtypes, and interventions that can target specific risk profiles linked to neural mechanisms. Ongoing initiatives that assess longer-term (> 5-10 years) outcome of CHR participants can provide valuable information about predictors of later conversion and diagnostic outcomes while large-scale international biomarker studies provide hope for precision intervention that will alter the course of early psychosis globally.

Prognostic models predicting transition to psychotic disorder using blood-based biomarkers: a systematic review and critical appraisal

Accumulating evidence suggests individuals with psychotic disorder show abnormalities in metabolic and inflammatory processes. Recently, several studies have employed blood-based predictors in models predicting transition to psychotic disorder in risk-enriched populations. A systematic review of the performance and methodology of prognostic models using blood-based biomarkers in the prediction of psychotic disorder from risk-enriched populations is warranted. Databases (PubMed, EMBASE and PsycINFO) were searched for eligible texts from 1998 to 15/05/2023, which detailed model development or validation studies. The checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was used to guide data extraction from eligible texts and the Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias and applicability of the studies. A narrative synthesis of the included studies was performed. Seventeen eligible studies were identified: 16 eligible model development studies and one eligible model validation study. A wide range of biomarkers were assessed, including nucleic acids, proteins, metabolites, and lipids. The range of C-index (area under the curve) estimates reported for the models was 0.67-1.00. No studies assessed model calibration. According to PROBAST criteria, all studies were at high risk of bias in the analysis domain. While a wide range of potentially predictive biomarkers were identified in the included studies, most studies did not account for overfitting in model performance estimates, no studies assessed calibration, and all models were at high risk of bias according to PROBAST criteria. External validation of the models is needed to provide more accurate estimates of their performance. Future studies which follow the latest available methodological and reporting guidelines and adopt strategies to accommodate required sample sizes for model development or validation will clarify the value of including blood-based biomarkers in models predicting psychosis.

Transdiagnostic risk of mental disorders in offspring of affected parents: a meta-analysis of family high-risk and registry studies

The offspring of parents with mental disorders are at increased risk for developing mental disorders themselves. The risk to offspring may extend transdiagnostically to disorders other than those present in the parents. The literature on this topic is vast but mixed. To inform targeted prevention and genetic counseling, we performed a comprehensive, PRISMA 2020-compliant meta-analysis. We systematically searched the literature published up to September 2022 to retrieve original family high-risk and registry studies reporting on the risk of mental disorders in offspring of parents with any type of mental disorder. We performed random-effects meta-analyses of the relative risk (risk ratio, RR) and absolute risk (lifetime, up to the age at assessment) of mental disorders, defined according to the ICD or DSM. Cumulative incidence by offspring age was determined using meta-analytic Kaplan-Meier curves. We measured heterogeneity with the I2 statistic, and risk of bias with the Quality In Prognosis Studies (QUIPS) tool. Sensitivity analyses addressed the impact of study design (family high-risk vs. registry) and specific vs. transdiagnostic risks. Transdiagnosticity was appraised with the TRANSD criteria. We identified 211 independent studies that reported data on 3,172,115 offspring of parents with psychotic, bipolar, depressive, disruptive, attention-deficit/hyperactivity, anxiety, substance use, eating, obsessive-compulsive, and borderline personality disorders, and 20,428,575 control offspring. The RR and lifetime risk of developing any mental disorder were 3.0 and 55% in offspring of parents with anxiety disorders; 2.6 and 17% in offspring of those with psychosis; 2.1 and 55% in offspring of those with bipolar disorder; 1.9 and 51% in offspring of those with depressive disorders; and 1.5 and 38% in offspring of those with substance use disorders. The offspring's RR and lifetime risk of developing the same mental disorder diagnosed in their parent were 8.4 and 32% for attention-deficit/hyperactivity disorder; 5.8 and 8% for psychosis; 5.1 and 5% for bipolar disorder; 2.8 and 9% for substance use disorders; 2.3 and 14% for depressive disorders; 2.3 and 1% for eating disorders; and 2.2 and 31% for anxiety disorders. There were 37 significant transdiagnostic associations between parental mental disorders and the RR of developing a different mental disorder in the offspring. In offspring of parents with psychosis, bipolar and depressive disorder, the risk of the same disorder onset emerged at 16, 5 and 6 years, and cumulated to 3%, 19% and 24% by age 18; and to 8%, 36% and 46% by age 28. Heterogeneity ranged from 0 to 0.98, and 96% of studies were at high risk of bias. Sensitivity analyses restricted to prospective family high-risk studies confirmed the pattern of findings with similar RR, but with greater absolute risks compared to analyses of all study types. This study demonstrates at a global, meta-analytic level that offspring of affected parents have strongly elevated RR and lifetime risk of developing any mental disorder as well as the same mental disorder diagnosed in the parent. The transdiagnostic risks suggest that offspring of parents with a range of mental disorders should be considered as candidates for targeted primary prevention.

Prognostic accuracy and clinical utility of psychometric instruments for individuals at clinical high-risk of psychosis: a systematic review and meta-analysis

Accurate prognostication of individuals at clinical high-risk for psychosis (CHR-P) is an essential initial step for effective primary indicated prevention. We aimed to summarise the prognostic accuracy and clinical utility of CHR-P assessments for primary indicated psychosis prevention. Web of Knowledge databases were searched until 1st January 2022 for longitudinal studies following-up individuals undergoing a psychometric or diagnostic CHR-P assessment, reporting transition to psychotic disorders in both those who meet CHR-P criteria (CHR-P + ) or not (CHR-P-). Prognostic accuracy meta-analysis was conducted following relevant guidelines. Primary outcome was prognostic accuracy, indexed by area-under-the-curve (AUC), sensitivity and specificity, estimated by the number of true positives, false positives, false negatives and true negatives at the longest available follow-up time. Clinical utility analyses included: likelihood ratios, Fagan's nomogram, and population-level preventive capacity (Population Attributable Fraction, PAF). A total of 22 studies (n = 4 966, 47.5% female, age range 12-40) were included. There were not enough meta-analysable studies on CHR-P diagnostic criteria (DSM-5 Attenuated Psychosis Syndrome) or non-clinical samples. Prognostic accuracy of CHR-P psychometric instruments in clinical samples (individuals referred to CHR-P services or diagnosed with 22q.11.2 deletion syndrome) was excellent: AUC = 0.85 (95% CI: 0.81-0.88) at a mean follow-up time of 34 months. This result was driven by outstanding sensitivity (0.93, 95% CI: 0.87-0.96) and poor specificity (0.58, 95% CI: 0.50-0.66). Being CHR-P + was associated with a small likelihood ratio LR + (2.17, 95% CI: 1.81-2.60) for developing psychosis. Being CHR-P- was associated with a large LR- (0.11, 95%CI: 0.06-0.21) for developing psychosis. Fagan's nomogram indicated a low positive (0.0017%) and negative (0.0001%) post-test risk in non-clinical general population samples. The PAF of the CHR-P state is 10.9% (95% CI: 4.1-25.5%). These findings consolidate the use of psychometric instruments for CHR-P in clinical samples for primary indicated prevention of psychosis. Future research should improve the ability to rule in psychosis risk.

Timing of antipsychotics and benzodiazepine initiation during a first episode of psychosis impacts clinical outcomes: Electronic health record cohort study

The impact of timing of antipsychotics and benzodiazepine treatment during a first episode of psychosis on clinical outcomes is unknown. We present a RECORD-compliant electronic health record cohort study including patients (n = 4,483, aged 14-35) with a primary diagnosis of any non-organic ICD-10 first episode of psychosis at SLAM-NHS between 2007 and 2017. The impact of antipsychotic timing (prescription > 1 week after a first episode of psychosis) was assessed on the primary outcome (risk of any psychiatric inpatient admission over 6 years), and secondary outcomes (cumulative duration of any psychiatric/medical/accident/emergency [A&E] admission over 6 years). The impact of prescribing benzodiazepine before antipsychotic at any point and of treatment patterns (antipsychotic alone, benzodiazepine alone, combination of antipsychotic with benzodiazepine) within the first week after a first episode of psychosis were also assessed. Survival analyses and zero-inflated negative binomial regressions, adjusted for core covariates, and complementary analyses were employed. Antipsychotic prescribed >1 week after a first episode of psychosis did not affect the risk of any psychiatric admission (HR = 1.04, 95% CI = 0.92-1.17, p = 0.557), but increased the duration of any psychiatric (22-28%), medical (78-35%) and A&E (30-34%) admission (months 12-72). Prescribing benzodiazepine before antipsychotic at any point did not affect the risk of any psychiatric admission (HR = 1.03, 95% CI = 0.94-1.13, p = 0.535), but reduced the duration of any psychiatric admission (17-24%, months 12-72), and increased the duration of medical (71-45%, months 12-72) and A&E (26-18%, months 12-36) admission. Prescribing antipsychotic combined with benzodiazepine within the first week after a first episode of psychosis showed better overall clinical outcomes than antipsychotic or benzodiazepine alone. Overall, delaying antipsychotic 1 week after a first episode of psychosis may worsen some clinical outcomes. Early benzodiazepine treatment can be considered with concomitant antipsychotic but not as standalone intervention.

Translating RDoC to Real-World Impact in Developmental Psychopathology: A Neurodevelopmental Framework for Application of Mental Health Risk Calculators

The National Institute of Mental Health Research Domain Criteria's (RDoC) has prompted a paradigm shift from categorical psychiatric disorders to considering multiple levels of vulnerability for probabilistic risk of disorder. However, the lack of neurodevelopmentally-based tools for clinical decision-making has limited RDoC's real-world impact. Integration with developmental psychopathology principles and statistical methods actualize the clinical implementation of RDoC to inform neurodevelopmental risk. In this conceptual paper, we introduce the probabilistic mental health risk calculator as an innovation for such translation and lay out a research agenda for generating an RDoC- and developmentally-informed paradigm that could be applied to predict a range of developmental psychopathologies from early childhood to young adulthood. We discuss methods that weigh the incremental utility for prediction based on intensity and burden of assessment, the addition of developmental change patterns, considerations for assessing outcomes, and integrative data approaches. Throughout, we illustrate the risk calculator approach with different neurodevelopmental pathways and phenotypes. Finally, we discuss real-world implementation of these methods for improving early identification and prevention of developmental psychopathology. We propose that mental health risk calculators can build a needed bridge between RDoC's multiple units of analysis and developmental science.

Developing and Validating an Individualized Clinical Prediction Model to Forecast Psychotic Recurrence in Acute and Transient Psychotic Disorders: Electronic Health Record Cohort Study

Acute and transient psychotic disorders (ATPDs) include short-lived psychotic episodes with a high probability of developing psychotic recurrences. Clinical care for ATPD is currently limited by the inability to predict outcomes. Real-world electronic health record (EHR)-based retrospective cohort study STROBE/RECORD compliant included all individuals accessing the South London and Maudsley NHS Trust between 2006 and 2017 and receiving a first diagnosis of ATPD (F23, ICD-10). After imputing missing data, stepwise and LASSO Cox regression methods employing a priori predictors (n = 23) were compared to develop and internally validate an individualized risk prediction model to forecast the risk of psychotic recurrences following TRIPOD guidelines. The primary outcome was prognostic accuracy (area under the curve [AUC]). 3018 ATPD individuals were included (average age = 33.75 years, 52.7% females). Over follow-up (average 1042 ± 1011 days, up to 8 years) there were 1160 psychotic recurrences (events). Stepwise (n = 12 predictors) and LASSO (n = 17 predictors) regression methods yielded comparable prognostic accuracy, with an events per variable ratio >100 for both models. Both models showed an internally validated adequate prognostic accuracy from 4 years follow-up (AUC 0.70 for both models) and good calibration. A refined model was adapted in view of the new ICD-11 criteria on 307 subjects with polymorphic ATPD, showing fair prognostic accuracy at 4 years (AUC: stepwise 0.68; LASSO 0.70). This study presents the first clinically based prediction model internally validated to adequately predict long-term psychotic recurrence in individuals with ATPD. The model can be automatable in EHRs, supporting further external validations and refinements to improve its prognostic accuracy.

Probability of Transition to Psychosis in Individuals at Clinical High Risk: An Updated Meta-analysis

IMPORTANCE

Estimating the current likelihood of transitioning from a clinical high risk for psychosis (CHR-P) to psychosis holds paramount importance for preventive care and applied research.

OBJECTIVE

To quantitatively examine the consistency and magnitude of transition risk to psychosis in individuals at CHR-P.

DATA SOURCES

PubMed and Web of Science databases until November 1, 2020. Manual search of references from previous articles.

STUDY SELECTION

Longitudinal studies reporting transition risks in individuals at CHR-P.

DATA EXTRACTION AND SYNTHESIS

Meta-analysis compliant with Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and Meta-analysis of Observational Studies in Epidemiology (MOOSE) reporting guidelines; independent data extraction, manually and through digitalization of Kaplan-Meier curves.

MAIN OUTCOME AND MEASURES

Primary effect size was cumulative risk of transition to psychosis at 0.5, 1, 1.5, 2, 2.5, 3, 4, and more than 4 years' follow-up, estimated using the numbers of individuals at CHR-P transitioning to psychosis at each time point. These analyses were complemented by meta-analytical Kaplan-Meier curves and speed of transition to psychosis (hazard rate). Random-effects meta-analysis, between-study heterogeneity analysis, study quality assessment, and meta-regressions were conducted.

RESULTS

A total of 130 studies and 9222 individuals at CHR-P were included. The mean (SD) age was 20.3 (4.4) years, and 5100 individuals (55.3%) were male. The cumulative transition risk was 0.09 (95% CI, 0.07-0.10; k = 37; n = 6485) at 0.5 years, 0.15 (95% CI, 0.13-0.16; k = 53; n = 7907) at 1 year, 0.20 (95% CI, 0.17-0.22; k = 30; n = 5488) at 1.5 years, 0.19 (95% CI, 0.17-0.22; k = 44; n = 7351) at 2 years, 0.25 (95% CI, 0.21-0.29; k = 19; n = 3114) at 2.5 years, 0.25 (95% CI, 0.22-0.29; k = 29; n = 4029) at 3 years, 0.27 (95% CI, 0.23-0.30; k = 16; n = 2926) at 4 years, and 0.28 (95% CI, 0.20-0.37; k = 14; n = 2301) at more than 4 years. The cumulative Kaplan-Meier transition risk was 0.08 (95% CI, 0.08-0.09; n = 4860) at 0.5 years, 0.14 (95% CI, 0.13-0.15; n = 3408) at 1 year, 0.17 (95% CI, 0.16-0.19; n = 2892) at 1.5 years, 0.20 (95% CI, 0.19-0.21; n = 2357) at 2 years, 0.25 (95% CI, 0.23-0.26; n = 1444) at 2.5 years, 0.27 (95% CI, 0.25-0.28; n = 1029) at 3 years, 0.28 (95% CI, 0.26-0.29; n = 808) at 3.5 years, 0.29 (95% CI, 0.27-0.30; n = 737) at 4 years, and 0.35 (95% CI, 0.32-0.38; n = 114) at 10 years. The hazard rate only plateaued at 4 years' follow-up. Meta-regressions showed that a lower proportion of female individuals (β = -0.02; 95% CI, -0.04 to -0.01) and a higher proportion of brief limited intermittent psychotic symptoms (β = 0.02; 95% CI, 0.01-0.03) were associated with an increase in transition risk. Heterogeneity across the studies was high (I2 range, 77.91% to 95.73%).

CONCLUSIONS AND RELEVANCE

In this meta-analysis, 25% of individuals at CHR-P developed psychosis within 3 years. Transition risk continued increasing in the long term. Extended clinical monitoring and preventive care may be beneficial in this patient population.

Preventive psychiatry: a blueprint for improving the mental health of young people

Preventive approaches have latterly gained traction for improving mental health in young people. In this paper, we first appraise the conceptual foundations of preventive psychiatry, encompassing the public health, Gordon's, US Institute of Medicine, World Health Organization, and good mental health frameworks, and neurodevelopmentally-sensitive clinical staging models. We then review the evidence supporting primary prevention of psychotic, bipolar and common mental disorders and promotion of good mental health as potential transformative strategies to reduce the incidence of these disorders in young people. Within indicated approaches, the clinical high-risk for psychosis paradigm has received the most empirical validation, while clinical high-risk states for bipolar and common mental disorders are increasingly becoming a focus of attention. Selective approaches have mostly targeted familial vulnerability and non-genetic risk exposures. Selective screening and psychological/psychoeducational interventions in vulnerable subgroups may improve anxiety/depressive symptoms, but their efficacy in reducing the incidence of psychotic/bipolar/common mental disorders is unproven. Selective physical exercise may reduce the incidence of anxiety disorders. Universal psychological/psychoeducational interventions may improve anxiety symptoms but not prevent depressive/anxiety disorders, while universal physical exercise may reduce the incidence of anxiety disorders. Universal public health approaches targeting school climate or social determinants (demographic, economic, neighbourhood, environmental, social/cultural) of mental disorders hold the greatest potential for reducing the risk profile of the population as a whole. The approach to promotion of good mental health is currently fragmented. We leverage the knowledge gained from the review to develop a blueprint for future research and practice of preventive psychiatry in young people: integrating universal and targeted frameworks; advancing multivariable, transdiagnostic, multi-endpoint epidemiological knowledge; synergically preventing common and infrequent mental disorders; preventing physical and mental health burden together; implementing stratified/personalized prognosis; establishing evidence-based preventive interventions; developing an ethical framework, improving prevention through education/training; consolidating the cost-effectiveness of preventive psychiatry; and decreasing inequalities. These goals can only be achieved through an urgent individual, societal, and global level response, which promotes a vigorous collaboration across scientific, health care, societal and governmental sectors for implementing preventive psychiatry, as much is at stake for young people with or at risk for emerging mental disorders.

Prediction and Prevention in the Clinical High-Risk for Psychosis Paradigm: A Review of the Current Status and Recommendations for Future Directions of Inquiry

Prediction and prevention of negative clinical and functional outcomes represent the two primary objectives of research conducted within the clinical high-risk for psychosis (CHR-P) paradigm. Several multivariable "risk calculator" models have been developed to predict the likelihood of developing psychosis, although these models have not been translated to clinical use. Overall, less progress has been made in developing effective interventions. In this paper, we review the existing literature on both prediction and prevention in the CHR-P paradigm and, primarily, outline ways in which expanding and combining these paths of inquiry could lead to a greater improvement in individual outcomes for those most at risk.