Third external replication of an individualised transdiagnostic prediction model for the automatic detection of individuals at risk of psychosis using electronic health records

Using Electronic Health Records to Facilitate Precision Psychiatry

The use of clinical prediction models to produce individualized risk estimates can facilitate the implementation of precision psychiatry. As a source of data from large, clinically representative patient samples, electronic health records (EHRs) provide a platform to develop and validate clinical prediction models, as well as potentially implement them in routine clinical care. The current review describes promising use cases for the application of precision psychiatry to EHR data and considers their performance in terms of discrimination (ability to separate individuals with and without the outcome) and calibration (extent to which predicted risk estimates correspond to observed outcomes), as well as their potential clinical utility (weighing benefits and costs associated with the model compared to different approaches across different assumptions of the number needed to test). We review 4 externally validated clinical prediction models designed to predict psychosis onset, psychotic relapse, cardiometabolic morbidity, and suicide risk. We then discuss the prospects for clinically implementing these models and the potential added value of integrating data from evidence syntheses, standardized psychometric assessments, and biological data into EHRs. Clinical prediction models can utilize routinely collected EHR data in an innovative way, representing a unique opportunity to inform real-world clinical decision making. Combining data from other sources (e.g., meta-analyses) or enhancing EHR data with information from research studies (clinical and biomarker data) may enhance our abilities to improve the performance of clinical prediction models.

Dynamic and Transdiagnostic Risk Calculator Based on Natural Language Processing for the Prediction of Psychosis in Secondary Mental Health Care: Development and Internal-External Validation Cohort Study

BACKGROUND

Automatic transdiagnostic risk calculators can improve the detection of individuals at risk of psychosis. However, they rely on assessment at a single point in time and can be refined with dynamic modeling techniques that account for changes in risk over time.

METHODS

We included 158,139 patients (5007 events) who received a first index diagnosis of a nonorganic and nonpsychotic mental disorder within electronic health records from the South London and Maudsley National Health Service Foundation Trust between January 1, 2008, and October 8, 2021. A dynamic Cox landmark model was developed to estimate the 2-year risk of developing psychosis according to the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis) statement. The dynamic model included 24 predictors extracted at 9 landmark points (baseline, 0, 6, 12, 24, 30, 36, 42, and 48 months): 3 demographic, 1 clinical, and 20 natural language processing-based symptom and substance use predictors. Performance was compared with a static Cox regression model with all predictors assessed at baseline only and indexed via discrimination (C-index), calibration (calibration plots), and potential clinical utility (decision curves) in internal-external validation.

RESULTS

The dynamic model improved discrimination performance from baseline compared with the static model (dynamic: C-index = 0.9; static: C-index = 0.87) and the final landmark point (dynamic: C-index = 0.79; static: C-index = 0.76). The dynamic model was also significantly better calibrated (calibration slope = 0.97-1.1) than the static model at later landmark points (≥24 months). Net benefit was higher for the dynamic than for the static model at later landmark points (≥24 months).

CONCLUSIONS

These findings suggest that dynamic prediction models can improve the detection of individuals at risk for psychosis in secondary mental health care settings.

Individualized pretest risk estimates to guide treatment decisions in patients with clinical high risk for psychotic disorders

INTRODUCTION

Clinical high risk for psychosis (CHR) states are associated with an increased risk of transition to psychosis. However, the predictive value of CHR screening interviews is dependent on pretest risk enrichment in referred patients. This poses a major obstacle to CHR outreach campaigns since they invariably lead to risk dilution through enhanced awareness. A potential compensatory strategy is to use estimates of individual pretest risk as a 'gatekeeper' for specialized assessment. We aimed to test a risk stratification model previously developed in London, UK (OASIS) and to train a new predictive model for the Swiss population.

METHOD

The sample was composed of 513 individuals referred for CHR assessment from six Swiss early psychosis detection services. Sociodemographic variables available at referral were used as predictors whereas the outcome variable was transition to psychosis.

RESULTS

Replication of the risk stratification model developed in OASIS resulted in poor performance (Harrel's c=0.51). Retraining resulted in moderate discrimination (Harrel's c=0.67) which significantly differentiated between different risk groups. The lowest risk group had a cumulative transition incidence of 6.4% (CI: 0-23.1%) over two years.

CONCLUSION

Failure to replicate the OASIS risk stratification model might reflect differences in the public health care systems and referral structures between Switzerland and London. Retraining resulted in a model with adequate discrimination performance. The developed model in combination with CHR assessment result, might be useful for identifying individuals with high pretest risk, who might benefit most from specialized intervention.

A transdiagnostic prodrome for severe mental disorders: an electronic health record study

Effective prevention of severe mental disorders (SMD), including non-psychotic unipolar mood disorders (UMD), non-psychotic bipolar mood disorders (BMD), and psychotic disorders (PSY), rely on accurate knowledge of the duration, first presentation, time course and transdiagnosticity of their prodromal stages. Here we present a retrospective, real-world, cohort study using electronic health records, adhering to RECORD guidelines. Natural language processing algorithms were used to extract monthly occurrences of 65 prodromal features (symptoms and substance use), grouped into eight prodromal clusters. The duration, first presentation, and transdiagnosticity of the prodrome were compared between SMD groups with one-way ANOVA, Cohen's f and d. The time course (mean occurrences) of prodromal clusters was compared between SMD groups with linear mixed-effects models. 26,975 individuals diagnosed with ICD-10 SMD were followed up for up to 12 years (UMD = 13,422; BMD = 2506; PSY = 11,047; median[IQR] age 39.8[23.7] years; 55% female; 52% white). The duration of the UMD prodrome (18[36] months) was shorter than BMD (26[35], d = 0.21) and PSY (24[38], d = 0.18). Most individuals presented with multiple first prodromal clusters, with the most common being non-specific ('other'; 88% UMD, 85% BMD, 78% PSY). The only first prodromal cluster that showed a medium-sized difference between the three SMD groups was positive symptoms (f = 0.30). Time course analysis showed an increase in prodromal cluster occurrences approaching SMD onset. Feature occurrence across the prodromal period showed small/negligible differences between SMD groups, suggesting that most features are transdiagnostic, except for positive symptoms (e.g. paranoia, f = 0.40). Taken together, our findings show minimal differences in the duration and first presentation of the SMD prodromes as recorded in secondary mental health care. All the prodromal clusters intensified as individuals approached SMD onset, and all the prodromal features other than positive symptoms are transdiagnostic. These results support proposals to develop transdiagnostic preventive services for affective and psychotic disorders detected in secondary mental healthcare.

Risk factors and prediction model for mental health in Chinese soldiers

Introduction

This study aimed to explore potential risk factors for mental health concerns, and the prediction model for mental health concerns in Chinese soldiers was constructed through combined eligible risk factors.

Methods

This cross-sectional study was performed on soldiers under direct command from Gansu, Sichuan, and Chongqing in China, and the soldiers were selected by cluster convenient sampling from 16 October 2018 to 10 December 2018. The Symptom Checklist-90 (SCL-90) and three questionnaires (Military Mental Health Status Questionnaire, Military Mental Health Ability Questionnaire, and Mental Quality Questionnaire for Army Men) were administered, including demographics, military careers, and 18 factors.

Results

Of 1,430 Chinese soldiers, 162 soldiers presented mental disorders, with a prevalence of 11.33%. A total of five risk factors were identified, including serving place (Sichuan vs. Gansu: OR, 1.846, 95% CI: 1.028-3.315, P = 0.038; Chongqing vs. Gansu: OR, 3.129, 95% CI, 1.669-5.869, P = 0.003), psychosis (OR, 1.491, 95% CI, 1.152-1.928, P = 0.002), depression (OR, 1.482, 95% CI, 1.349-1.629, P < 0.001), sleep problems (OR, 1.235, 95% CI, 1.162-1.311, P < 0.001), and frustration (OR, 1.050, 95% CI, 1.015-1.087, P = 0.005). The area under the ROC curve by combining these factors was 0.930 (95% CI: 0.907-0.952) for predicting mental disorders in Chinese soldiers.

Conclusion

The findings of this study demonstrate that mental disorders and onset in Chinese soldiers can be predicted on the basis of these three questionnaires, and the predictive value of the combined model was high.

Prognostic accuracy and clinical utility of psychometric instruments for individuals at clinical high-risk of psychosis: a systematic review and meta-analysis

Accurate prognostication of individuals at clinical high-risk for psychosis (CHR-P) is an essential initial step for effective primary indicated prevention. We aimed to summarise the prognostic accuracy and clinical utility of CHR-P assessments for primary indicated psychosis prevention. Web of Knowledge databases were searched until 1st January 2022 for longitudinal studies following-up individuals undergoing a psychometric or diagnostic CHR-P assessment, reporting transition to psychotic disorders in both those who meet CHR-P criteria (CHR-P + ) or not (CHR-P-). Prognostic accuracy meta-analysis was conducted following relevant guidelines. Primary outcome was prognostic accuracy, indexed by area-under-the-curve (AUC), sensitivity and specificity, estimated by the number of true positives, false positives, false negatives and true negatives at the longest available follow-up time. Clinical utility analyses included: likelihood ratios, Fagan's nomogram, and population-level preventive capacity (Population Attributable Fraction, PAF). A total of 22 studies (n = 4 966, 47.5% female, age range 12-40) were included. There were not enough meta-analysable studies on CHR-P diagnostic criteria (DSM-5 Attenuated Psychosis Syndrome) or non-clinical samples. Prognostic accuracy of CHR-P psychometric instruments in clinical samples (individuals referred to CHR-P services or diagnosed with 22q.11.2 deletion syndrome) was excellent: AUC = 0.85 (95% CI: 0.81-0.88) at a mean follow-up time of 34 months. This result was driven by outstanding sensitivity (0.93, 95% CI: 0.87-0.96) and poor specificity (0.58, 95% CI: 0.50-0.66). Being CHR-P + was associated with a small likelihood ratio LR + (2.17, 95% CI: 1.81-2.60) for developing psychosis. Being CHR-P- was associated with a large LR- (0.11, 95%CI: 0.06-0.21) for developing psychosis. Fagan's nomogram indicated a low positive (0.0017%) and negative (0.0001%) post-test risk in non-clinical general population samples. The PAF of the CHR-P state is 10.9% (95% CI: 4.1-25.5%). These findings consolidate the use of psychometric instruments for CHR-P in clinical samples for primary indicated prevention of psychosis. Future research should improve the ability to rule in psychosis risk.

Developing and Validating an Individualized Clinical Prediction Model to Forecast Psychotic Recurrence in Acute and Transient Psychotic Disorders: Electronic Health Record Cohort Study

Acute and transient psychotic disorders (ATPDs) include short-lived psychotic episodes with a high probability of developing psychotic recurrences. Clinical care for ATPD is currently limited by the inability to predict outcomes. Real-world electronic health record (EHR)-based retrospective cohort study STROBE/RECORD compliant included all individuals accessing the South London and Maudsley NHS Trust between 2006 and 2017 and receiving a first diagnosis of ATPD (F23, ICD-10). After imputing missing data, stepwise and LASSO Cox regression methods employing a priori predictors (n = 23) were compared to develop and internally validate an individualized risk prediction model to forecast the risk of psychotic recurrences following TRIPOD guidelines. The primary outcome was prognostic accuracy (area under the curve [AUC]). 3018 ATPD individuals were included (average age = 33.75 years, 52.7% females). Over follow-up (average 1042 ± 1011 days, up to 8 years) there were 1160 psychotic recurrences (events). Stepwise (n = 12 predictors) and LASSO (n = 17 predictors) regression methods yielded comparable prognostic accuracy, with an events per variable ratio >100 for both models. Both models showed an internally validated adequate prognostic accuracy from 4 years follow-up (AUC 0.70 for both models) and good calibration. A refined model was adapted in view of the new ICD-11 criteria on 307 subjects with polymorphic ATPD, showing fair prognostic accuracy at 4 years (AUC: stepwise 0.68; LASSO 0.70). This study presents the first clinically based prediction model internally validated to adequately predict long-term psychotic recurrence in individuals with ATPD. The model can be automatable in EHRs, supporting further external validations and refinements to improve its prognostic accuracy.

Preventive psychiatry: a blueprint for improving the mental health of young people

Preventive approaches have latterly gained traction for improving mental health in young people. In this paper, we first appraise the conceptual foundations of preventive psychiatry, encompassing the public health, Gordon's, US Institute of Medicine, World Health Organization, and good mental health frameworks, and neurodevelopmentally-sensitive clinical staging models. We then review the evidence supporting primary prevention of psychotic, bipolar and common mental disorders and promotion of good mental health as potential transformative strategies to reduce the incidence of these disorders in young people. Within indicated approaches, the clinical high-risk for psychosis paradigm has received the most empirical validation, while clinical high-risk states for bipolar and common mental disorders are increasingly becoming a focus of attention. Selective approaches have mostly targeted familial vulnerability and non-genetic risk exposures. Selective screening and psychological/psychoeducational interventions in vulnerable subgroups may improve anxiety/depressive symptoms, but their efficacy in reducing the incidence of psychotic/bipolar/common mental disorders is unproven. Selective physical exercise may reduce the incidence of anxiety disorders. Universal psychological/psychoeducational interventions may improve anxiety symptoms but not prevent depressive/anxiety disorders, while universal physical exercise may reduce the incidence of anxiety disorders. Universal public health approaches targeting school climate or social determinants (demographic, economic, neighbourhood, environmental, social/cultural) of mental disorders hold the greatest potential for reducing the risk profile of the population as a whole. The approach to promotion of good mental health is currently fragmented. We leverage the knowledge gained from the review to develop a blueprint for future research and practice of preventive psychiatry in young people: integrating universal and targeted frameworks; advancing multivariable, transdiagnostic, multi-endpoint epidemiological knowledge; synergically preventing common and infrequent mental disorders; preventing physical and mental health burden together; implementing stratified/personalized prognosis; establishing evidence-based preventive interventions; developing an ethical framework, improving prevention through education/training; consolidating the cost-effectiveness of preventive psychiatry; and decreasing inequalities. These goals can only be achieved through an urgent individual, societal, and global level response, which promotes a vigorous collaboration across scientific, health care, societal and governmental sectors for implementing preventive psychiatry, as much is at stake for young people with or at risk for emerging mental disorders.