Machine Learning: A Primer for Child Psychiatrists

Tracing Tomorrow: young people's preferences and values related to use of personal sensing to predict mental health, using a digital game methodology

BACKGROUND

Use of personal sensing to predict mental health risk has sparked interest in adolescent psychiatry, offering a potential tool for targeted early intervention.

OBJECTIVES

We investigated the preferences and values of UK adolescents with regard to use of digital sensing information, including social media and internet searching behaviour. We also investigated the impact of risk information on adolescents' self-understanding.

METHODS

Following a Design Bioethics approach, we created and disseminated a purpose-built digital game (www.tracingtomorrow.org) that immersed the player-character in a fictional scenario in which they received a risk assessment for depression Data were collected through game choices across relevant scenarios, with decision-making supported through clickable information points.

FINDINGS

The game was played by 7337 UK adolescents aged 16-18 years. Most participants were willing to personally communicate mental health risk information to their parents or best friend. The acceptability of school involvement in risk predictions based on digital traces was mixed, due mainly to privacy concerns. Most participants indicated that risk information could negatively impact their academic self-understanding. Participants overwhelmingly preferred individual face-to-face over digital options for support.

CONCLUSIONS

The potential of digital phenotyping in supporting early intervention in mental health can only be fulfilled if data are collected, communicated and actioned in ways that are trustworthy, relevant and acceptable to young people.

CLINICAL IMPLICATIONS

To minimise the risk of ethical harms in real-world applications of preventive psychiatric technologies, it is essential to investigate young people's values and preferences as part of design and implementation processes.

Identifying women with postdelivery posttraumatic stress disorder using natural language processing of personal childbirth narratives

BACKGROUND

Maternal mental disorders are considered a leading complication of childbirth and a common contributor to maternal death. In addition to undermining maternal welfare, untreated postpartum psychopathology can result in child emotional and physical neglect and associated significant pediatric health costs. Some women may experience traumatic childbirth and develop posttraumatic stress disorder symptoms after delivery (childbirth-related posttraumatic stress disorder). Although women are routinely screened for postpartum depression in the United States, there is no recommended protocol to inform the identification of women who are likely to experience childbirth-related posttraumatic stress disorder. Advancements in computational methods of free text have shown promise in informing the diagnosis of psychiatric conditions. Although the language in narratives of stressful events has been associated with posttrauma outcomes, whether the narratives of childbirth processed via machine learning can be useful for childbirth-related posttraumatic stress disorder screening is unknown.

OBJECTIVE

This study aimed to examine the use of written narrative accounts of personal childbirth experiences for the identification of women with childbirth-related posttraumatic stress disorder. To this end, we developed a model based on natural language processing and machine learning algorithms to identify childbirth-related posttraumatic stress disorder via the classification of birth narratives.

STUDY DESIGN

Overall, 1127 eligible postpartum women who enrolled in a study survey during the COVID-19 pandemic provided short written childbirth narrative accounts in which they were instructed to focus on the most distressing aspects of their childbirth experience. They also completed a posttraumatic stress disorder symptom screen to determine childbirth-related posttraumatic stress disorder. After the exclusion criteria were applied, data from 995 participants were analyzed. A machine learning-based Sentence-Transformers natural language processing model was used to represent narratives as vectors that served as inputs for a neural network machine learning model developed in this study to identify participants with childbirth-related posttraumatic stress disorder.

RESULTS

The machine learning model derived from natural language processing of childbirth narratives achieved good performance (area under the curve, 0.75; F1 score, 0.76; sensitivity, 0.8; specificity, 0.70). Moreover, women with childbirth-related posttraumatic stress disorder generated longer narratives (t test results: t=2.30; p=.02) and used more negative emotional expressions (Wilcoxon test: sadness: p=8.90e-04; W=31,017; anger: p=1.32e-02; W=35,005.50) and death-related words (Wilcoxon test: p=3.48e-05; W=34,538) in describing their childbirth experience than those with no childbirth-related posttraumatic stress disorder.

CONCLUSION

This study provided proof of concept that personal childbirth narrative accounts generated in the early postpartum period and analyzed via advanced computational methods can detect with relatively high accuracy women who are likely to endorse childbirth-related posttraumatic stress disorder and those at low risk. This suggests that birth narratives could be promising for informing low-cost, noninvasive tools for maternal mental health screening, and more research that used machine learning to predict early signs of maternal psychiatric morbidity is warranted.

Identifying Women with Post-Delivery Posttraumatic Stress Disorder using Natural Language Processing of Personal Childbirth Narratives

Background

Maternal mental disorders are considered a leading complication of childbirth and a common contributor to maternal death. In addition to undermining maternal welfare, untreated postpartum psychopathology can result in child emotional and physical neglect, and associated significant pediatric health costs. Some women may experience a traumatic childbirth and develop posttraumatic stress disorder (PTSD) symptoms following delivery (CB-PTSD). Although women are routinely screened for postpartum depression in the U.S., there is no recommended protocol to inform the identification of women who are likely to experience CB-PTSD. Advancements in computational methods of free text has shown promise in informing diagnosis of psychiatric conditions. Although the language in narratives of stressful events has been associated with post-trauma outcomes, whether the narratives of childbirth processed via machine learning can be useful for CB-PTSD screening is unknown.

Objective

This study examined the utility of written narrative accounts of personal childbirth experience for the identification of women with provisional CB-PTSD. To this end, we developed a model based on natural language processing (NLP) and machine learning (ML) algorithms to identify CB-PTSD via classification of birth narratives.

Study Design

A total of 1,127 eligible postpartum women who enrolled in a study survey during the COVID-19 era provided short written childbirth narrative accounts in which they were instructed to focus on the most distressing aspects of their childbirth experience. They also completed a PTSD symptom screen to determine provisional CB-PTSD. After exclusion criteria were applied, data from 995 participants was analyzed. An ML-based Sentence-Transformer NLP model was used to represent narratives as vectors that served as inputs for a neural network ML model developed in this study to identify participants with provisional CB-PTSD.

Results

The ML model derived from NLP of childbirth narratives achieved good performance: AUC 0.75, F1-score 0.76, sensitivity 0.8, and specificity 0.70. Moreover, women with provisional CB-PTSD generated longer narratives (t-test results: t=2 . 30, p=0 . 02 ) and used more negative emotional expressions (Wilcoxon test: 'sadness': p=8 . 90e- ⁰⁴ , W=31,017 ; 'anger': p=1 . 32e- ⁰² , W=35,005 . 50 ) and death-related words (Wilcoxon test: p=3 . 48e- ⁰⁵ , W=34,538 ) in describing their childbirth experience than those with no CB-PTSD.

Conclusions

This study provides proof of concept that personal childbirth narrative accounts generated in the early postpartum period and analyzed via advanced computational methods can detect with relatively high accuracy women who are likely to endorse CB-PTSD and those at low risk. This suggests that birth narratives could be promising for informing low-cost, non-invasive tools for maternal mental health screening, and more research that utilizes ML to predict early signs of maternal psychiatric morbidity is warranted.

Machine Learning Methods for Predicting Postpartum Depression: Scoping Review

BACKGROUND

Machine learning (ML) offers vigorous statistical and probabilistic techniques that can successfully predict certain clinical conditions using large volumes of data. A review of ML and big data research analytics in maternal depression is pertinent and timely, given the rapid technological developments in recent years.

OBJECTIVE

This study aims to synthesize the literature on ML and big data analytics for maternal mental health, particularly the prediction of postpartum depression (PPD).

METHODS

We used a scoping review methodology using the Arksey and O'Malley framework to rapidly map research activity in ML for predicting PPD. Two independent researchers searched PsycINFO, PubMed, IEEE Xplore, and the ACM Digital Library in September 2020 to identify relevant publications in the past 12 years. Data were extracted from the articles' ML model, data type, and study results.

RESULTS

A total of 14 studies were identified. All studies reported the use of supervised learning techniques to predict PPD. Support vector machine and random forest were the most commonly used algorithms in addition to Naive Bayes, regression, artificial neural network, decision trees, and XGBoost (Extreme Gradient Boosting). There was considerable heterogeneity in the best-performing ML algorithm across the selected studies. The area under the receiver operating characteristic curve values reported for different algorithms were support vector machine (range 0.78-0.86), random forest method (0.88), XGBoost (0.80), and logistic regression (0.93).

CONCLUSIONS

ML algorithms can analyze larger data sets and perform more advanced computations, which can significantly improve the detection of PPD at an early stage. Further clinical research collaborations are required to fine-tune ML algorithms for prediction and treatment. ML might become part of evidence-based practice in addition to clinical knowledge and existing research evidence.

Maternal Parenting Stress in the Face of Early Regulatory Disorders in Infancy: A Machine Learning Approach to Identify What Matters Most

Objective: Early regulatory disorders (ERD) in infancy are typically associated with high parenting stress (PS). Theoretical and empirical literature suggests a wide range of factors that may contribute to PS related to ERD. The aim of this study was to identify key predictors of maternal PS within a large predictor data set in a sample of N = 135 mothers of infants diagnosed with ERD. Methods: We used machine learning to identify relevant predictors. Maternal PS was assessed with the Parenting Stress Index. The multivariate dataset assessed cross-sectionally consisted of 464 self-reported and clinically rated variables covering mother-reported psychological distress, maternal self-efficacy, parental reflective functioning, socio-demographics, each parent's history of illness, recent significant life events, former miscarriage/abortion, pregnancy, obstetric history, infants' medical history, development, and social environment. Variables were drawn from behavioral diaries on regulatory symptoms and parental co-regulative behavior as well as a clinical interview which was utilized to diagnose ERD and to assess clinically rated regulatory symptoms, quality of parent-infant relationship, organic/biological and psychosocial risks, and social-emotional functioning. Results: The final prediction model identified 11 important variables summing up to the areas maternal self-efficacy, psychological distress (particularly depression and anger-hostility), infant regulatory symptoms (particularly duration of fussing/crying), and age-appropriate physical development. The RMSE (i.e., prediction accuracy) of the final model applied to the test set was 21.72 (R ² = 0.58). Conclusions: This study suggests that among behavioral, environmental, developmental, parent-infant relationship, and mental health variables, a mother's higher self-efficacy, psychological distress symptoms particularly depression and anger symptoms, symptoms in the child particularly fussing/crying symptoms, and age-inappropriate physical development are associated with higher maternal PS. With these factors identified, clinicians may more efficiently assess a mother's PS related to ERD in a low-risk help-seeking sample.

Machine Learning for Differential Diagnosis Between Clinical Conditions With Social Difficulty: Autism Spectrum Disorder, Early Psychosis, and Social Anxiety Disorder

Differential diagnosis in adult cohorts with social difficulty is confounded by comorbid mental health conditions, common etiologies, and shared phenotypes. Identifying shared and discriminating profiles can facilitate intervention and remediation strategies. The objective of the study was to identify salient features of a composite test battery of cognitive and mood measures using a machine learning paradigm in clinical cohorts with social interaction difficulties. We recruited clinical participants who met standardized diagnostic criteria for autism spectrum disorder (ASD: n = 62), early psychosis (EP: n = 48), or social anxiety disorder (SAD: N = 83) and compared them with a neurotypical comparison group (TYP: N = 43). Using five machine-learning algorithms and repeated cross-validation, we trained and tested classification models using measures of cognitive and executive function, lower- and higher-order social cognition and mood severity. Performance metrics were the area under the curve (AUC) and Brier Scores. Sixteen features successfully differentiated between the groups. The control versus social impairment cohorts (ASD, EP, SAD) were differentiated by social cognition, visuospatial memory and mood measures. Importantly, a distinct profile cluster drawn from social cognition, visual learning, executive function and mood, distinguished the neurodevelopmental cohort (EP and ASD) from the SAD group. The mean AUC range was between 0.891 and 0.916 for social impairment versus control cohorts and, 0.729 to 0.781 for SAD vs neurodevelopmental cohorts. This is the first study that compares an extensive battery of neuropsychological and self-report measures using a machine learning protocol in clinical and neurodevelopmental cohorts characterized by social impairment. Findings are relevant for diagnostic, intervention and remediation strategies for these groups.

Digital Phenotyping With Mobile and Wearable Devices: Advanced Symptom Measurement in Child and Adolescent Depression

With an estimated 75% of all mental disorders beginning in the first two decades of life,¹ childhood and adolescence are crucial developmental periods to identify and intercept the unfolding of mental health problems, their relationships with physical health, and the multiple, interwoven connections to the surrounding environment.² Because an individual's mental health is best conceptualized, captured, and treated by taking into account the network of physiological and social functions that constitute the context of individual experience, accessing and analyzing data on multiple health indicators simultaneously can accelerate prediction of disease progression. With the advent of new technologies, dense and extensive amounts of biopsychosocial readouts that can be translated into clinically relevant information have become available in real time, with the potential to revolutionize the practice of medicine. However, challenges to this more ecological and comprehensive approach to mental health measurement include the actual capacity of capturing, safely storing, and analyzing dense data sets (encompassing, for example, mood, cognitions, physical activity, sleep, social interactions) from multiple synchronized sources, and identifying which among multiple indicators ultimately prove useful to improve prediction of a deterioration in symptoms and of initiating early intervention. In this Translations article, we focus on digital phenotyping (DP), which relates to the capturing of the aforementioned relevant biopsychosocial data. This concept is rapidly growing and gaining relevance to child and adolescent psychiatry, and is connected with overarching data science themes of "big data" (extremely large data sets, including data from electronic medical records, imaging, genomics, and patients' smartphones),^3,4 in addition to "machine learning" (the science of getting computers to act without being explicitly programmed)⁵ and "precision medicine" (the practice of custom tailoring treatments to a patient's disease processes),⁶ which have all received attention in this journal. We will describe principles and current applications of DP, together with its potential to facilitate improved outcomes and its limits, using depression in children and adolescents as an illustrative example.

Identification of suicidal behavior among psychiatrically hospitalized adolescents using natural language processing and machine learning of electronic health records

Objective

The rapid proliferation of machine learning research using electronic health records to classify healthcare outcomes offers an opportunity to address the pressing public health problem of adolescent suicidal behavior. We describe the development and evaluation of a machine learning algorithm using natural language processing of electronic health records to identify suicidal behavior among psychiatrically hospitalized adolescents.

Methods

Adolescents hospitalized on a psychiatric inpatient unit in a community health system in the northeastern United States were surveyed for history of suicide attempt in the past 12 months. A total of 73 respondents had electronic health records available prior to the index psychiatric admission. Unstructured clinical notes were downloaded from the year preceding the index inpatient admission. Natural language processing identified phrases from the notes associated with the suicide attempt outcome. We enriched this group of phrases with a clinically focused list of terms representing known risk and protective factors for suicide attempt in adolescents. We then applied the random forest machine learning algorithm to develop a classification model. The model performance was evaluated using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy.

Results

The final model had a sensitivity of 0.83, specificity of 0.22, AUC of 0.68, a PPV of 0.42, NPV of 0.67, and an accuracy of 0.47. The terms mostly highly associated with suicide attempt clustered around terms related to suicide, family members, psychiatric disorders, and psychotropic medications.

Conclusion

This analysis demonstrates modest success of a natural language processing and machine learning approach to identifying suicide attempt among a small sample of hospitalized adolescents in a psychiatric setting.

Prediction of outcome in internet-delivered cognitive behaviour therapy for paediatric obsessive-compulsive disorder: A machine learning approach

BACKGROUND

There are no consistent predictors of treatment outcome in paediatric obsessive-compulsive disorder (OCD). One reason for this might be the use of suboptimal statistical methodology. Machine learning is an approach to efficiently analyse complex data. Machine learning has been widely used within other fields, but has rarely been tested in the prediction of paediatric mental health treatment outcomes.

OBJECTIVE

To test four different machine learning methods in the prediction of treatment response in a sample of paediatric OCD patients who had received Internet-delivered cognitive behaviour therapy (ICBT).

METHODS

Participants were 61 adolescents (12-17 years) who enrolled in a randomized controlled trial and received ICBT. All clinical baseline variables were used to predict strictly defined treatment response status three months after ICBT. Four machine learning algorithms were implemented. For comparison, we also employed a traditional logistic regression approach.

RESULTS

Multivariate logistic regression could not detect any significant predictors. In contrast, all four machine learning algorithms performed well in the prediction of treatment response, with 75 to 83% accuracy.

CONCLUSIONS

The results suggest that machine learning algorithms can successfully be applied to predict paediatric OCD treatment outcome. Validation studies and studies in other disorders are warranted.