Real-world implementation of precision psychiatry: Transdiagnostic risk calculator for the automatic detection of individuals at-risk of psychosis

Dynamic and Transdiagnostic Risk Calculator Based on Natural Language Processing for the Prediction of Psychosis in Secondary Mental Health Care: Development and Internal-External Validation Cohort Study

BACKGROUND

Automatic transdiagnostic risk calculators can improve the detection of individuals at risk of psychosis. However, they rely on assessment at a single point in time and can be refined with dynamic modeling techniques that account for changes in risk over time.

METHODS

We included 158,139 patients (5007 events) who received a first index diagnosis of a nonorganic and nonpsychotic mental disorder within electronic health records from the South London and Maudsley National Health Service Foundation Trust between January 1, 2008, and October 8, 2021. A dynamic Cox landmark model was developed to estimate the 2-year risk of developing psychosis according to the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis) statement. The dynamic model included 24 predictors extracted at 9 landmark points (baseline, 0, 6, 12, 24, 30, 36, 42, and 48 months): 3 demographic, 1 clinical, and 20 natural language processing-based symptom and substance use predictors. Performance was compared with a static Cox regression model with all predictors assessed at baseline only and indexed via discrimination (C-index), calibration (calibration plots), and potential clinical utility (decision curves) in internal-external validation.

RESULTS

The dynamic model improved discrimination performance from baseline compared with the static model (dynamic: C-index = 0.9; static: C-index = 0.87) and the final landmark point (dynamic: C-index = 0.79; static: C-index = 0.76). The dynamic model was also significantly better calibrated (calibration slope = 0.97-1.1) than the static model at later landmark points (≥24 months). Net benefit was higher for the dynamic than for the static model at later landmark points (≥24 months).

CONCLUSIONS

These findings suggest that dynamic prediction models can improve the detection of individuals at risk for psychosis in secondary mental health care settings.

A transdiagnostic prodrome for severe mental disorders: an electronic health record study

Effective prevention of severe mental disorders (SMD), including non-psychotic unipolar mood disorders (UMD), non-psychotic bipolar mood disorders (BMD), and psychotic disorders (PSY), rely on accurate knowledge of the duration, first presentation, time course and transdiagnosticity of their prodromal stages. Here we present a retrospective, real-world, cohort study using electronic health records, adhering to RECORD guidelines. Natural language processing algorithms were used to extract monthly occurrences of 65 prodromal features (symptoms and substance use), grouped into eight prodromal clusters. The duration, first presentation, and transdiagnosticity of the prodrome were compared between SMD groups with one-way ANOVA, Cohen's f and d. The time course (mean occurrences) of prodromal clusters was compared between SMD groups with linear mixed-effects models. 26,975 individuals diagnosed with ICD-10 SMD were followed up for up to 12 years (UMD = 13,422; BMD = 2506; PSY = 11,047; median[IQR] age 39.8[23.7] years; 55% female; 52% white). The duration of the UMD prodrome (18[36] months) was shorter than BMD (26[35], d = 0.21) and PSY (24[38], d = 0.18). Most individuals presented with multiple first prodromal clusters, with the most common being non-specific ('other'; 88% UMD, 85% BMD, 78% PSY). The only first prodromal cluster that showed a medium-sized difference between the three SMD groups was positive symptoms (f = 0.30). Time course analysis showed an increase in prodromal cluster occurrences approaching SMD onset. Feature occurrence across the prodromal period showed small/negligible differences between SMD groups, suggesting that most features are transdiagnostic, except for positive symptoms (e.g. paranoia, f = 0.40). Taken together, our findings show minimal differences in the duration and first presentation of the SMD prodromes as recorded in secondary mental health care. All the prodromal clusters intensified as individuals approached SMD onset, and all the prodromal features other than positive symptoms are transdiagnostic. These results support proposals to develop transdiagnostic preventive services for affective and psychotic disorders detected in secondary mental healthcare.

Predictive Utility of Irritability "In Context": Proof-of-Principle for an Early Childhood Mental Health Risk Calculator

OBJECTIVE

We provide proof-of-principle for a mental health risk calculator advancing clinical utility of the irritability construct for identification of young children at high risk for common, early onsetting syndromes.

METHOD

Data were harmonized from two longitudinal early childhood subsamples (total N = 403; 50.1% Male; 66.7% Nonwhite; Mage = 4.3 years). The independent subsamples were clinically enriched via disruptive behavior and violence (Subsample 1) and depression (Subsample 2). In longitudinal models, epidemiologic risk prediction methods for risk calculators were applied to test the utility of the transdiagnostic indicator, early childhood irritability, in the context of other developmental and social-ecological indicators to predict risk of internalizing/externalizing disorders at preadolescence (Mage = 9.9 years). Predictors were retained when they improved model discrimination (area under the receiver operating characteristic curve [AUC] and integrated discrimination index [IDI]) beyond the base demographic model.

RESULTS

Compared to the base model, the addition of early childhood irritability and adverse childhood experiences significantly improved the AUC (0.765) and IDI slope (0.192). Overall, 23% of preschoolers went on to develop a preadolescent internalizing/externalizing disorder. For preschoolers with both elevated irritability and adverse childhood experiences, the likelihood of an internalizing/externalizing disorder was 39-66%.

CONCLUSIONS

Predictive analytic tools enable personalized prediction of psychopathological risk for irritable young children, holding transformative potential for clinical translation.

Accurate Machine Learning Prediction in Psychiatry Needs the Right Kind of Information

This Viewpoint discusses the type and amount of data needed for machine learning models to accurately predict diagnoses and treatment outcomes at the individual patient level.

Identifying risk of stillbirth using machine learning

BACKGROUND

Previous predictive models using logistic regression for stillbirth do not leverage the advanced and nuanced techniques involved in sophisticated machine learning methods, such as modeling nonlinear relationships between outcomes.

OBJECTIVE

This study aimed to create and refine machine learning models for predicting stillbirth using data available before viability (22-24 weeks) and throughout pregnancy, as well as demographic, medical, and prenatal visit data, including ultrasound and fetal genetics.

STUDY DESIGN

This is a secondary analysis of the Stillbirth Collaborative Research Network, which included data from pregnancies resulting in stillborn and live-born infants delivered at 59 hospitals in 5 diverse regions across the United States from 2006 to 2009. The primary aim was the creation of a model for predicting stillbirth using data available before viability. Secondary aims included refining models with variables available throughout pregnancy and determining variable importance.

RESULTS

Among 3000 live births and 982 stillbirths, 101 variables of interest were identified. Of the models incorporating data available before viability, the random forests model had 85.1% accuracy (area under the curve) and high sensitivity (88.6%), specificity (85.3%), positive predictive value (85.3%), and negative predictive value (84.8%). A random forests model using data collected throughout pregnancy resulted in accuracy of 85.0%; this model had 92.2% sensitivity, 77.9% specificity, 84.7% positive predictive value, and 88.3% negative predictive value. Important variables in the previability model included previous stillbirth, minority race, gestational age at the earliest prenatal visit and ultrasound, and second-trimester serum screening.

CONCLUSION

Applying advanced machine learning techniques to a comprehensive database of stillbirths and live births with unique and clinically relevant variables resulted in an algorithm that could accurately identify 85% of pregnancies that would result in stillbirth, before they reached viability. Once validated in representative databases reflective of the US birthing population and then prospectively, these models may provide effective risk stratification and clinical decision-making support to better identify and monitor those at risk of stillbirth.

Detecting at-risk mental states for psychosis (ARMS) using machine learning ensembles and facial features

AIMS

Our study aimed to develop a machine learning ensemble to distinguish "at-risk mental states for psychosis" (ARMS) subjects from control individuals from the general population based on facial data extracted from video-recordings.

METHODS

58 non-help-seeking medication-naïve ARMS and 70 healthy subjects were screened from a general population sample. At-risk status was assessed with the Structured Interview for Prodromal Syndromes (SIPS), and "Subject's Overview" section was filmed (5-10 min). Several features were extracted, e.g., eye and mouth aspect ratio, Euler angles, coordinates from 51 facial landmarks. This elicited 649 facial features, which were further selected using Gradient Boosting Machines (AdaBoost combined with Random Forests). Data was split in 70/30 for training, and Monte Carlo cross validation was used.

RESULTS

Final model reached 83 % of mean F1-score, and balanced accuracy of 85 %. Mean area under the curve for the receiver operator curve classifier was 93 %. Convergent validity testing showed that two features included in the model were significantly correlated with Avolition (SIPS N2 item) and expression of emotion (SIPS N3 item).

CONCLUSION

Our model capitalized on short video-recordings from individuals recruited from the general population, effectively distinguishing between ARMS and controls. Results are encouraging for large-screening purposes in low-resource settings.

Characterization of patients with idiopathic normal pressure hydrocephalus using natural language processing within an electronic healthcare record system

OBJECTIVE

Idiopathic normal pressure hydrocephalus (iNPH) is an underdiagnosed, progressive, and disabling condition. Early treatment is associated with better outcomes and improved quality of life. In this paper, the authors aimed to identify features associated with patients with iNPH using natural language processing (NLP) to characterize this cohort, with the intention to later target the development of artificial intelligence-driven tools for early detection.

METHODS

The electronic health records of patients with shunt-responsive iNPH were retrospectively reviewed using an NLP algorithm. Participants were selected from a prospectively maintained single-center database of patients undergoing CSF diversion for probable iNPH (March 2008-July 2020). Analysis was conducted on preoperative health records including clinic letters, referrals, and radiology reports accessed through CogStack. Clinical features were extracted from these records as SNOMED CT (Systematized Nomenclature of Medicine Clinical Terms) concepts using a named entity recognition machine learning model. In the first phase, a base model was generated using unsupervised training on 1 million electronic health records and supervised training with 500 double-annotated documents. The model was fine-tuned to improve accuracy using 300 records from patients with iNPH double annotated by two blinded assessors. Thematic analysis of the concepts identified by the machine learning algorithm was performed, and the frequency and timing of terms were analyzed to describe this patient group.

RESULTS

In total, 293 eligible patients responsive to CSF diversion were identified. The median age at CSF diversion was 75 years, with a male predominance (69% male). The algorithm performed with a high degree of precision and recall (F1 score 0.92). Thematic analysis revealed the most frequently documented symptoms related to mobility, cognitive impairment, and falls or balance. The most frequent comorbidities were related to cardiovascular and hematological problems.

CONCLUSIONS

This model demonstrates accurate, automated recognition of iNPH features from medical records. Opportunities for translation include detecting patients with undiagnosed iNPH from primary care records, with the aim to ultimately improve outcomes for these patients through artificial intelligence-driven early detection of iNPH and prompt treatment.

Psychotic disorders as a framework for precision psychiatry

People with psychotic disorders can show marked interindividual variations in the onset of illness, responses to treatment and relapse, but they receive broadly similar clinical care. Precision psychiatry is an approach that aims to stratify people with a given disorder according to different clinical outcomes and tailor treatment to their individual needs. At present, interindividual differences in outcomes of psychotic disorders are difficult to predict on the basis of clinical assessment alone. Therefore, current research in psychosis seeks to build models that predict outcomes by integrating clinical information with a range of biological measures. Here, we review recent progress in the application of precision psychiatry to psychotic disorders and consider the challenges associated with implementing this approach in clinical practice.

Multi-omics data integration methods and their applications in psychiatric disorders

To study mental illness and health, in the past researchers have often broken down their complexity into individual subsystems (e.g., genomics, transcriptomics, proteomics, clinical data) and explored the components independently. Technological advancements and decreasing costs of high throughput sequencing has led to an unprecedented increase in data generation. Furthermore, over the years it has become increasingly clear that these subsystems do not act in isolation but instead interact with each other to drive mental illness and health. Consequently, individual subsystems are now analysed jointly to promote a holistic understanding of the underlying biological complexity of health and disease. Complementing the increasing data availability, current research is geared towards developing novel methods that can efficiently combine the information rich multi-omics data to discover biologically meaningful biomarkers for diagnosis, treatment, and prognosis. However, clinical translation of the research is still challenging. In this review, we summarise conventional and state-of-the-art statistical and machine learning approaches for discovery of biomarker, diagnosis, as well as outcome and treatment response prediction through integrating multi-omics and clinical data. In addition, we describe the role of biological model systems and in silico multi-omics model designs in clinical translation of psychiatric research from bench to bedside. Finally, we discuss the current challenges and explore the application of multi-omics integration in future psychiatric research. The review provides a structured overview and latest updates in the field of multi-omics in psychiatry.

Prognostic accuracy and clinical utility of psychometric instruments for individuals at clinical high-risk of psychosis: a systematic review and meta-analysis

Accurate prognostication of individuals at clinical high-risk for psychosis (CHR-P) is an essential initial step for effective primary indicated prevention. We aimed to summarise the prognostic accuracy and clinical utility of CHR-P assessments for primary indicated psychosis prevention. Web of Knowledge databases were searched until 1st January 2022 for longitudinal studies following-up individuals undergoing a psychometric or diagnostic CHR-P assessment, reporting transition to psychotic disorders in both those who meet CHR-P criteria (CHR-P + ) or not (CHR-P-). Prognostic accuracy meta-analysis was conducted following relevant guidelines. Primary outcome was prognostic accuracy, indexed by area-under-the-curve (AUC), sensitivity and specificity, estimated by the number of true positives, false positives, false negatives and true negatives at the longest available follow-up time. Clinical utility analyses included: likelihood ratios, Fagan's nomogram, and population-level preventive capacity (Population Attributable Fraction, PAF). A total of 22 studies (n = 4 966, 47.5% female, age range 12-40) were included. There were not enough meta-analysable studies on CHR-P diagnostic criteria (DSM-5 Attenuated Psychosis Syndrome) or non-clinical samples. Prognostic accuracy of CHR-P psychometric instruments in clinical samples (individuals referred to CHR-P services or diagnosed with 22q.11.2 deletion syndrome) was excellent: AUC = 0.85 (95% CI: 0.81-0.88) at a mean follow-up time of 34 months. This result was driven by outstanding sensitivity (0.93, 95% CI: 0.87-0.96) and poor specificity (0.58, 95% CI: 0.50-0.66). Being CHR-P + was associated with a small likelihood ratio LR + (2.17, 95% CI: 1.81-2.60) for developing psychosis. Being CHR-P- was associated with a large LR- (0.11, 95%CI: 0.06-0.21) for developing psychosis. Fagan's nomogram indicated a low positive (0.0017%) and negative (0.0001%) post-test risk in non-clinical general population samples. The PAF of the CHR-P state is 10.9% (95% CI: 4.1-25.5%). These findings consolidate the use of psychometric instruments for CHR-P in clinical samples for primary indicated prevention of psychosis. Future research should improve the ability to rule in psychosis risk.

Real-World Implementation of Precision Psychiatry: A Systematic Review of Barriers and Facilitators

Background: Despite significant research progress surrounding precision medicine in psychiatry, there has been little tangible impact upon real-world clinical care. Objective: To identify barriers and facilitators affecting the real-world implementation of precision psychiatry. Method: A PRISMA-compliant systematic literature search of primary research studies, conducted in the Web of Science, Cochrane Central Register of Controlled Trials, PsycINFO and OpenGrey databases. We included a qualitative data synthesis structured according to the ‘Consolidated Framework for Implementation Research’ (CFIR) key constructs. Results: Of 93,886 records screened, 28 studies were suitable for inclusion. The included studies reported 38 barriers and facilitators attributed to the CFIR constructs. Commonly reported barriers included: potential psychological harm to the service user (n = 11), cost and time investments (n = 9), potential economic and occupational harm to the service user (n = 8), poor accuracy and utility of the model (n = 8), and poor perceived competence in precision medicine amongst staff (n = 7). The most highly reported facilitator was the availability of adequate competence and skills training for staff (n = 7). Conclusions: Psychiatry faces widespread challenges in the implementation of precision medicine methods. Innovative solutions are required at the level of the individual and the wider system to fulfil the translational gap and impact real-world care.

Clinical prediction models in psychiatry: a systematic review of two decades of progress and challenges

Recent years have seen the rapid proliferation of clinical prediction models aiming to support risk stratification and individualized care within psychiatry. Despite growing interest, attempts to synthesize current evidence in the nascent field of precision psychiatry have remained scarce. This systematic review therefore sought to summarize progress towards clinical implementation of prediction modeling for psychiatric outcomes. We searched MEDLINE, PubMed, Embase, and PsychINFO databases from inception to September 30, 2020, for English-language articles that developed and/or validated multivariable models to predict (at an individual level) onset, course, or treatment response for non-organic psychiatric disorders (PROSPERO: CRD42020216530). Individual prediction models were evaluated based on three key criteria: (i) mitigation of bias and overfitting; (ii) generalizability, and (iii) clinical utility. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used to formally appraise each study's risk of bias. 228 studies detailing 308 prediction models were ultimately eligible for inclusion. 94.5% of developed prediction models were deemed to be at high risk of bias, largely due to inadequate or inappropriate analytic decisions. Insufficient internal validation efforts (within the development sample) were also observed, while only one-fifth of models underwent external validation in an independent sample. Finally, our search identified just one published model whose potential utility in clinical practice was formally assessed. Our findings illustrated significant growth in precision psychiatry with promising progress towards real-world application. Nevertheless, these efforts have been inhibited by a preponderance of bias and overfitting, while the generalizability and clinical utility of many published models has yet to be formally established. Through improved methodological rigor during initial development, robust evaluations of reproducibility via independent validation, and evidence-based implementation frameworks, future research has the potential to generate risk prediction tools capable of enhancing clinical decision-making in psychiatric care.

NeuroBlu, an electronic health record (EHR) trusted research environment (TRE) to support mental healthcare analytics with real-world data

PURPOSE

NeuroBlu is a real-world data (RWD) repository that contains deidentified electronic health record (EHR) data from US mental healthcare providers operating the MindLinc EHR system. NeuroBlu enables users to perform statistical analysis through a secure web-based interface. Structured data are available for sociodemographic characteristics, mental health service contacts, hospital admissions, International Classification of Diseases ICD-9/ICD-10 diagnosis, prescribed medications, family history of mental disorders, Clinical Global Impression-Severity and Improvement (CGI-S/CGI-I) and Global Assessment of Functioning (GAF). To further enhance the data set, natural language processing (NLP) tools have been applied to obtain mental state examination (MSE) and social/environmental data. This paper describes the development and implementation of NeuroBlu, the procedures to safeguard data integrity and security and how the data set supports the generation of real-world evidence (RWE) in mental health.

PARTICIPANTS

As of 31 July 2021, 562 940 individuals (48.9% men) were present in the data set with a mean age of 33.4 years (SD: 18.4 years). The most frequently recorded diagnoses were substance use disorders (1 52 790 patients), major depressive disorder (1 29 120 patients) and anxiety disorders (1 03 923 patients). The median duration of follow-up was 7 months (IQR: 1.3 to 24.4 months).

FINDINGS TO DATE

The data set has supported epidemiological studies demonstrating increased risk of psychiatric hospitalisation and reduced antidepressant treatment effectiveness among people with comorbid substance use disorders. It has also been used to develop data visualisation tools to support clinical decision-making, evaluate comparative effectiveness of medications, derive models to predict treatment response and develop NLP applications to obtain clinical information from unstructured EHR data.

FUTURE PLANS

The NeuroBlu data set will be further analysed to better understand factors related to poor clinical outcome, treatment responsiveness and the development of predictive analytic tools that may be incorporated into the source EHR system to support real-time clinical decision-making in the delivery of mental healthcare services.

Don't Get Lost in Translation: Integrating Developmental and Implementation Sciences to Accelerate Real-World Impact on Children's Development, Health, and Wellbeing

Translation of developmental science discoveries is impeded by numerous barriers at different stages of the research-to-practice pipeline. Actualization of the vast potential of the developmental sciences to improve children's health and development in the real world is imperative but has not yet been fully realized. In this commentary, we argue that an integrated developmental-implementation sciences framework will result in a translational mindset essential for accelerating real world impact. We delineate key principles and methods of implementation science of salience to the developmental science audience, lay out a potential synthesis between implementation and developmental sciences, provide an illustration of the Mental Health, Earlier Partnership (MHE-P), and set actionable steps for realization. Blending these approaches along with wide-spread adoption of the translational mindset has transformative potential for population-level impact of developmental science discovery.

Differential trajectories of tobacco smoking in people at ultra-high risk for psychosis: Associations with clinical outcomes

OBJECTIVE

People at ultra-high risk (UHR) for psychosis have a high prevalence of tobacco smoking, and rates are even higher among the subgroup that later develop a psychotic disorder. However, the longitudinal relationship between the course of tobacco smoking and clinical outcomes in UHR subjects is unknown.

METHODS

We investigated associations between tobacco smoking and clinical outcomes in a prospective study of UHR individuals (n = 324). Latent class mixed model analyses were used to identify trajectories of smoking severity. Mixed effects models were applied to investigate associations between smoking trajectory class and the course of attenuated psychotic symptoms (APS) and affective symptoms, as assessed using the CAARMS.

RESULTS

We identified four different classes of smoking trajectory: (i) Persistently High (n = 110), (ii) Decreasing (n = 29), (iii) Persistently Low (n = 165) and (iv) Increasing (n = 20). At two-year follow-up, there had been a greater increase in APS in the Persistently High class than for both the Persistently Low (ES = 9.77, SE = 4.87, p = 0.046) and Decreasing (ES = 18.18, SE = 7.61, p = 0.018) classes. There were no differences between smoking classes in the incidence of psychosis. There was a greater reduction in the severity of emotional disturbance and general symptoms in the Decreasing class than in the High (ES = -10.40, SE = 3.41, p = 0.003; ES = -22.36, SE = 10.07, p = 0.027), Increasing (ES = -11.35, SE = 4.55, p = 0.014; ES = -25.58, SE = 13.17, p = 0.050) and Low (ES = -11.38, SE = 3.29, p = 0.001; ES = -27.55, SE = 9.78, p = 0.005) classes, respectively.

CONCLUSIONS

These findings suggests that in UHR subjects persistent tobacco smoking is associated with an unfavorable course of psychotic symptoms, whereas decrease in the number of cigarettes smoked is associated with improvement in affective symptoms. Future research into smoking cessation interventions in the early stages of psychoses is required to shine light on the potential of modifying smoking behavior and its relation to clinical outcomes.

Translating RDoC to Real-World Impact in Developmental Psychopathology: A Neurodevelopmental Framework for Application of Mental Health Risk Calculators

The National Institute of Mental Health Research Domain Criteria's (RDoC) has prompted a paradigm shift from categorical psychiatric disorders to considering multiple levels of vulnerability for probabilistic risk of disorder. However, the lack of neurodevelopmentally-based tools for clinical decision-making has limited RDoC's real-world impact. Integration with developmental psychopathology principles and statistical methods actualize the clinical implementation of RDoC to inform neurodevelopmental risk. In this conceptual paper, we introduce the probabilistic mental health risk calculator as an innovation for such translation and lay out a research agenda for generating an RDoC- and developmentally-informed paradigm that could be applied to predict a range of developmental psychopathologies from early childhood to young adulthood. We discuss methods that weigh the incremental utility for prediction based on intensity and burden of assessment, the addition of developmental change patterns, considerations for assessing outcomes, and integrative data approaches. Throughout, we illustrate the risk calculator approach with different neurodevelopmental pathways and phenotypes. Finally, we discuss real-world implementation of these methods for improving early identification and prevention of developmental psychopathology. We propose that mental health risk calculators can build a needed bridge between RDoC's multiple units of analysis and developmental science.

Toward Generalizable and Transdiagnostic Tools for Psychosis Prediction: An Independent Validation and Improvement of the NAPLS-2 Risk Calculator in the Multisite PRONIA Cohort

BACKGROUND

Transition to psychosis is among the most adverse outcomes of clinical high-risk (CHR) syndromes encompassing ultra-high risk (UHR) and basic symptom states. Clinical risk calculators may facilitate an early and individualized interception of psychosis, but their real-world implementation requires thorough validation across diverse risk populations, including young patients with depressive syndromes.

METHODS

We validated the previously described NAPLS-2 (North American Prodrome Longitudinal Study 2) calculator in 334 patients (26 with transition to psychosis) with CHR or recent-onset depression (ROD) drawn from the multisite European PRONIA (Personalised Prognostic Tools for Early Psychosis Management) study. Patients were categorized into three risk enrichment levels, ranging from UHR, over CHR, to a broad-risk population comprising patients with CHR or ROD (CHR|ROD). We assessed how risk enrichment and different predictive algorithms influenced prognostic performance using reciprocal external validation.

RESULTS

After calibration, the NAPLS-2 model predicted psychosis with a balanced accuracy (BAC) (sensitivity, specificity) of 68% (73%, 63%) in the PRONIA-UHR cohort, 67% (74%, 60%) in the CHR cohort, and 70% (73%, 66%) in patients with CHR|ROD. Multiple model derivation in PRONIA-CHR|ROD and validation in NAPLS-2-UHR patients confirmed that broader risk definitions produced more accurate risk calculators (CHR|ROD-based vs. UHR-based performance: 67% [68%, 66%] vs. 58% [61%, 56%]). Support vector machines were superior in CHR|ROD (BAC = 71%), while ridge logistic regression and support vector machines performed similarly in CHR (BAC = 67%) and UHR cohorts (BAC = 65%). Attenuated psychotic symptoms predicted psychosis across risk levels, while younger age and reduced processing speed became increasingly relevant for broader risk cohorts.

CONCLUSIONS

Clinical-neurocognitive machine learning models operating in young patients with affective and CHR syndromes facilitate a more precise and generalizable prediction of psychosis. Future studies should investigate their therapeutic utility in large-scale clinical trials.

Developing and Validating an Individualized Clinical Prediction Model to Forecast Psychotic Recurrence in Acute and Transient Psychotic Disorders: Electronic Health Record Cohort Study

Acute and transient psychotic disorders (ATPDs) include short-lived psychotic episodes with a high probability of developing psychotic recurrences. Clinical care for ATPD is currently limited by the inability to predict outcomes. Real-world electronic health record (EHR)-based retrospective cohort study STROBE/RECORD compliant included all individuals accessing the South London and Maudsley NHS Trust between 2006 and 2017 and receiving a first diagnosis of ATPD (F23, ICD-10). After imputing missing data, stepwise and LASSO Cox regression methods employing a priori predictors (n = 23) were compared to develop and internally validate an individualized risk prediction model to forecast the risk of psychotic recurrences following TRIPOD guidelines. The primary outcome was prognostic accuracy (area under the curve [AUC]). 3018 ATPD individuals were included (average age = 33.75 years, 52.7% females). Over follow-up (average 1042 ± 1011 days, up to 8 years) there were 1160 psychotic recurrences (events). Stepwise (n = 12 predictors) and LASSO (n = 17 predictors) regression methods yielded comparable prognostic accuracy, with an events per variable ratio >100 for both models. Both models showed an internally validated adequate prognostic accuracy from 4 years follow-up (AUC 0.70 for both models) and good calibration. A refined model was adapted in view of the new ICD-11 criteria on 307 subjects with polymorphic ATPD, showing fair prognostic accuracy at 4 years (AUC: stepwise 0.68; LASSO 0.70). This study presents the first clinically based prediction model internally validated to adequately predict long-term psychotic recurrence in individuals with ATPD. The model can be automatable in EHRs, supporting further external validations and refinements to improve its prognostic accuracy.

Prediction models for child and adolescent mental health: A systematic review of methodology and reporting in recent research

Background

There has been a rapid growth in the publication of new prediction models relevant to child and adolescent mental health. However, before their implementation into clinical services, it is necessary to appraise the quality of their methods and reporting. We conducted a systematic review of new prediction models in child and adolescent mental health, and examined their development and validation.

Method

We searched five databases for studies developing or validating multivariable prediction models for individuals aged 18 years old or younger from 1 January 2018 to 18 February 2021. Quality of reporting was assessed using the Transparent Reporting of a multivariable prediction models for Individual Prognosis Or Diagnosis checklist, and quality of methodology using items based on expert guidance and the PROBAST tool.

Results

We identified 100 eligible studies: 41 developing a new prediction model, 48 validating an existing model and 11 that included both development and validation. Most publications (k = 75) reported a model discrimination measure, while 26 investigations reported calibration. Of 52 new prediction models, six (12%) were for suicidal outcomes, 18 (35%) for future diagnosis, five (10%) for child maltreatment. Other outcomes included violence, crime, and functional outcomes. Eleven new models (21%) were developed for use in high-risk populations. Of development studies, around a third were sufficiently statistically powered (k = 16%, 31%), while this was lower for validation investigations (k = 12, 25%). In terms of performance, the discrimination (as measured by the C-statistic) for new models ranged from 0.57 for a tool predicting ADHD diagnosis in an external validation sample to 0.99 for a machine learning model predicting foster care permanency.

Conclusions

Although some tools have recently been developed for child and adolescent mental health for prognosis and child maltreatment, none can be currently recommended for clinical practice due to a combination of methodological limitations and poor model performance. New work needs to use ensure sufficient sample sizes, representative samples, and testing of model calibration.

Impact of Comorbid Affective Disorders on Longitudinal Clinical Outcomes in Individuals at Ultra-high Risk for Psychosis

INTRODUCTION

Diagnoses of anxiety and/or depression are common in subjects at Ultra-High Risk for Psychosis (UHR) and associated with extensive functional impairment. Less is known about the impact of affective comorbidities on the prospective course of attenuated psychotic symptoms (APS).

METHOD

Latent class mixed modelling identified APS trajectories in 331 UHR subjects assessed at baseline, 6, 12, and 24 months follow-up. The prognostic value of past, baseline, and one-year DSM-IV depressive or anxiety disorders on trajectories was investigated using logistic regression, controlling for confounders. Cox proportional hazard analyses investigated associations with transition risk.

RESULTS

46.8% of participants fulfilled the criteria for a past depressive disorder, 33.2% at baseline, and 15.1% at one-year follow-up. Any past, baseline, or one-year anxiety disorder was diagnosed in 42.9%, 37.2%, and 27.0%, respectively. Participants were classified into one of three latent APS trajectory groups: (1) persistently low, (2) increasing, and (3) decreasing. Past depression was associated with a higher risk of belonging to the increasing trajectory group, compared to the persistently low (OR = 3.149, [95%CI: 1.298-7.642]) or decreasing group (OR = 3.137, [1.165-8.450]). In contrast, past (OR = .443, [.179-1.094]) or current (OR = .414, [.156-1.094]) anxiety disorders showed a trend-level association with a lower risk of belonging to the increasing group compared to the persistently low group. Past depression was significantly associated with a higher risk of transitioning to psychosis (HR = 2.123, [1.178-3.828]).

CONCLUSION

A past depressive episode might be a particularly relevant risk factor for an unfavorable course of APS in UHR individuals. Early affective disturbances may be used to advance detection, prognostic, and clinical strategies.

Preventive psychiatry: a blueprint for improving the mental health of young people

Preventive approaches have latterly gained traction for improving mental health in young people. In this paper, we first appraise the conceptual foundations of preventive psychiatry, encompassing the public health, Gordon's, US Institute of Medicine, World Health Organization, and good mental health frameworks, and neurodevelopmentally-sensitive clinical staging models. We then review the evidence supporting primary prevention of psychotic, bipolar and common mental disorders and promotion of good mental health as potential transformative strategies to reduce the incidence of these disorders in young people. Within indicated approaches, the clinical high-risk for psychosis paradigm has received the most empirical validation, while clinical high-risk states for bipolar and common mental disorders are increasingly becoming a focus of attention. Selective approaches have mostly targeted familial vulnerability and non-genetic risk exposures. Selective screening and psychological/psychoeducational interventions in vulnerable subgroups may improve anxiety/depressive symptoms, but their efficacy in reducing the incidence of psychotic/bipolar/common mental disorders is unproven. Selective physical exercise may reduce the incidence of anxiety disorders. Universal psychological/psychoeducational interventions may improve anxiety symptoms but not prevent depressive/anxiety disorders, while universal physical exercise may reduce the incidence of anxiety disorders. Universal public health approaches targeting school climate or social determinants (demographic, economic, neighbourhood, environmental, social/cultural) of mental disorders hold the greatest potential for reducing the risk profile of the population as a whole. The approach to promotion of good mental health is currently fragmented. We leverage the knowledge gained from the review to develop a blueprint for future research and practice of preventive psychiatry in young people: integrating universal and targeted frameworks; advancing multivariable, transdiagnostic, multi-endpoint epidemiological knowledge; synergically preventing common and infrequent mental disorders; preventing physical and mental health burden together; implementing stratified/personalized prognosis; establishing evidence-based preventive interventions; developing an ethical framework, improving prevention through education/training; consolidating the cost-effectiveness of preventive psychiatry; and decreasing inequalities. These goals can only be achieved through an urgent individual, societal, and global level response, which promotes a vigorous collaboration across scientific, health care, societal and governmental sectors for implementing preventive psychiatry, as much is at stake for young people with or at risk for emerging mental disorders.

[Cross-sectoral therapeutic concepts and innovative technologies: new opportunities for the treatment of patients with mental disorders]

Mental disorders are widespread and a major public health problem. The risk of developing a mental disorder at some point in life is around 40%. Therefore, mental disorders are among the most common diseases. Despite the introduction of newer psychotropic drugs, disorder-specific psychotherapy and stimulation techniques, many of those affected still show insufficient symptom remission and a chronic course of the disorder. Conceptual and technological progress in recent years has enabled a new, more flexible and personalized form of mental health care. Both the traditional therapeutic concepts and newer decentralized, modularly structured, track units, together with innovative digital technologies, will offer individualized therapeutic options in order to alleviate symptoms and improve quality of life of patients with mental illnesses. The primary goal of closely combining inpatient care concepts with innovative technologies is to provide comprehensive therapy and aftercare concepts for all individual needs of patients with mental disorders. Last but not least, this also ensures that specialist psychiatric treatment is available regardless of location. In twenty-first century psychiatry, modern care structures must be effectively linked to the current dynamics of digital transformation. This narrative review is dedicated to the theoretical and practical aspects of a cross-sectoral treatment system combined with innovative digital technologies in the psychiatric-psychotherapeutic field. The authors aim to illuminate these therapy modalities using the example of the Central Institute of Mental Health in Mannheim.

Establishing a clinical service to prevent psychosis: What, how and when? Systematic review

The first rate-limiting step to successfully translate prevention of psychosis in to clinical practice is to establish specialised Clinical High Risk for Psychosis (CHR-P) services. This study systematises the knowledge regarding CHR-P services and provides guidelines for translational implementation. We conducted a PRISMA/MOOSE-compliant (PROSPERO-CRD42020163640) systematic review of Web of Science to identify studies until 4/05/2020 reporting on CHR-P service configuration, outreach strategy and referrals, service user characteristics, interventions, and outcomes. Fifty-six studies (1998-2020) were included, encompassing 51 distinct CHR-P services across 15 countries and a catchment area of 17,252,666 people. Most services (80.4%) consisted of integrated multidisciplinary teams taking care of CHR-P and other patients. Outreach encompassed active (up to 97.6%) or passive (up to 63.4%) approaches: referrals came mostly (90%) from healthcare agencies. CHR-P individuals were more frequently males (57.2%). Most (70.6%) services accepted individuals aged 12-35 years, typically assessed with the CAARMS/SIPS (83.7%). Baseline comorbid mental conditions were reported in two-third (69.5%) of cases, and unemployment in one third (36.6%). Most services provided up to 2-years (72.4%), of clinical monitoring (100%), psychoeducation (81.1%), psychosocial support (73%), family interventions (73%), individual (67.6%) and group (18.9%) psychotherapy, physical health interventions (37.8%), antipsychotics (87.1%), antidepressants (74.2%), anxiolytics (51.6%), and mood stabilisers (38.7%). Outcomes were more frequently ascertained clinically (93.0%) and included: persistence of symptoms/comorbidities (67.4%), transition to psychosis (53.5%), and functional status (48.8%). We provide ten practical recommendations for implementation of CHR-P services. Health service knowledge summarised by the current study will facilitate translational efforts for implementation of CHR-P services worldwide.

Clinician Versus Nomogram Predicted Estimates of Kidney Stone Recurrence Risk

Purpose: Kidney stone recurrence rates vary between patients. A patient's risk informs the frequency and intensity of preventative interventions. Clinicians routinely use clinical experience to estimate risk. We sought to compare clinician estimated recurrence risk with the recurrence of kidney stones (ROKS) nomogram. Materials and Methods: We surveyed members of the Endourological Society with clinical expertise in kidney stones. Respondents estimated the risk of recurrence for patients in three clinical vignettes corresponding to low, intermediate, and high recurrence risk from the nomogram. Clinician estimates were compared with ROKS estimates. Results: The majority of the 318 respondents were from North America (n = 127, 40%). The most commonly estimated recurrence was 50% at 5 years. The respondents' estimates were significantly different from the ROKS predicted recurrence rate for all cases (Case 1, 50% vs 93% p < 0.0001; Case 2, 50% vs 60% p < 0.0001; Case 3, 60% vs 22% p < 0.0001). The ROKS predicted estimates ranged from 22% to 93%, whereas the median urologist-derived 5-year risk estimates for each case ranged from 50% to 60%. The median range of estimates by respondents across cases was 20%, narrower than the 71% for the ROKS nomogram. The majority of respondents (95%) do not use nomograms in practice, mostly because of lack of awareness of useful nomograms (59%). Conclusions: This study suggests that clinicians may not be able to distinguish those with high and low recurrence risk when compared with peers and when compared with a nomogram. Clinical decision support tools are needed to enable clinicians to better estimate stone recurrence risk.

Transdiagnostic individualized clinically-based risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: external replication in 2,430,333 US patients

The real-world impact of psychosis prevention is reliant on effective strategies for identifying individuals at risk. A transdiagnostic, individualized, clinically-based risk calculator to improve this has been developed and externally validated twice in two different UK healthcare trusts with convincing results. The prognostic performance of this risk calculator outside the UK is unknown. All individuals who accessed primary or secondary health care services belonging to the IBM® MarketScan® Commercial Database between January 2015 and December 2017, and received a first ICD-10 index diagnosis of nonorganic/nonpsychotic mental disorder, were included. According to the risk calculator, age, gender, ethnicity, age-by-gender, and ICD-10 cluster diagnosis at index date were used to predict development of any ICD-10 nonorganic psychotic disorder. Because patient-level ethnicity data were not available city-level ethnicity proportions were used as proxy. The study included 2,430,333 patients with a mean follow-up of 15.36 months and cumulative incidence of psychosis at two years of 1.43%. There were profound differences compared to the original development UK database in terms of case-mix, psychosis incidence, distribution of baseline predictors (ICD-10 cluster diagnoses), availability of patient-level ethnicity data, follow-up time and availability of specialized clinical services for at-risk individuals. Despite these important differences, the model retained accuracy significantly above chance (Harrell's C = 0.676, 95% CI: 0.672-0.679). To date, this is the largest international external replication of an individualized prognostic model in the field of psychiatry. This risk calculator is transportable on an international scale to improve the automatic detection of individuals at risk of psychosis.

Implementing Precision Psychiatry: A Systematic Review of Individualized Prediction Models for Clinical Practice

BACKGROUND

The impact of precision psychiatry for clinical practice has not been systematically appraised. This study aims to provide a comprehensive review of validated prediction models to estimate the individual risk of being affected with a condition (diagnostic), developing outcomes (prognostic), or responding to treatments (predictive) in mental disorders.

METHODS

PRISMA/RIGHT/CHARMS-compliant systematic review of the Web of Science, Cochrane Central Register of Reviews, and Ovid/PsycINFO databases from inception until July 21, 2019 (PROSPERO CRD42019155713) to identify diagnostic/prognostic/predictive prediction studies that reported individualized estimates in psychiatry and that were internally or externally validated or implemented. Random effect meta-regression analyses addressed the impact of several factors on the accuracy of prediction models.

FINDINGS

Literature search identified 584 prediction modeling studies, of which 89 were included. 10.4% of the total studies included prediction models internally validated (n = 61), 4.6% models externally validated (n = 27), and 0.2% (n = 1) models considered for implementation. Across validated prediction modeling studies (n = 88), 18.2% were diagnostic, 68.2% prognostic, and 13.6% predictive. The most frequently investigated condition was psychosis (36.4%), and the most frequently employed predictors clinical (69.5%). Unimodal compared to multimodal models (β = .29, P = .03) and diagnostic compared to prognostic (β = .84, p < .0001) and predictive (β = .87, P = .002) models were associated with increased accuracy.

INTERPRETATION

To date, several validated prediction models are available to support the diagnosis and prognosis of psychiatric conditions, in particular, psychosis, or to predict treatment response. Advancements of knowledge are limited by the lack of implementation research in real-world clinical practice. A new generation of implementation research is required to address this translational gap.