Developing algorithms to predict adult onset internalizing disorders: An ensemble learning approach

Performance Metrics, Algorithms, and Applications of Artificial Intelligence in Vascular and Interventional Neurology: A Review of Basic Elements

Artificial intelligence (AI) is currently being used as a routine tool for day-to-day activity. Medicine is not an exception to the growing usage of AI in various scientific fields. Vascular and interventional neurology deal with diseases that require early diagnosis and appropriate intervention, which are crucial to saving patients' lives. In these settings, AI can be an extra pair of hands for physicians or in conditions where there is a shortage of clinical experts. In this article, the authors have reviewed the common metrics used in interpreting the performance of models and common algorithms used in this field.

Predicting adolescent depression and anxiety from multi-wave longitudinal data using machine learning

BACKGROUND

This study leveraged machine learning to evaluate the contribution of information from multiple developmental stages to prospective prediction of depression and anxiety in mid-adolescence.

METHODS

A community sample (N = 374; 53.5% male) of children and their families completed tri-annual assessments across ages 3-15. The feature set included several important risk factors spanning psychopathology, temperament/personality, family environment, life stress, interpersonal relationships, neurocognitive, hormonal, and neural functioning, and parental psychopathology and personality. We used canonical correlation analysis (CCA) to reduce the large feature set to a lower dimensional space while preserving the longitudinal structure of the data. Ablation analysis was conducted to evaluate the relative contributions to prediction of information gathered at different developmental periods and relative to previous disorder status (i.e. age 12 depression or anxiety) and demographics (sex, race, ethnicity).

RESULTS

CCA components from individual waves predicted age 15 disorder status better than chance across ages 3, 6, 9, and 12 for anxiety and 9 and 12 for depression. Only the components from age 12 for depression, and ages 9 and 12 for anxiety, improved prediction over prior disorder status and demographics.

CONCLUSIONS

These findings suggest that screening for risk of adolescent depression can be successful as early as age 9, while screening for risk of adolescent anxiety can be successful as early as age 3. Assessing additional risk factors at age 12 for depression, and going back to age 9 for anxiety, can improve screening for risk at age 15 beyond knowing standard demographics and disorder history.

Predicting Future Depressive Episodes from Resting-State fMRI with Generative Embedding

After a first episode of major depressive disorder (MDD), there is substantial risk for a long-term remitting-relapsing course. Prevention and early interventions are thus critically important. Various studies have examined the feasibility of detecting at-risk individuals based on out-of-sample predictions about the future occurrence of depression. However, functional magnetic resonance imaging (MRI) has received very little attention for this purpose so far. Here, we explored the utility of generative models (i.e. different dynamic causal models, DCMs) as well as functional connectivity (FC) for predicting future episodes of depression in never-depressed adults, using a large dataset (N=906) of task-free ("resting state") fMRI data from the UK Biobank. Connectivity analyses were conducted using timeseries from pre-computed spatially independent components of different dimensionalities. Over a three year period, 50% of participants showed indications of at least one depressive episode, while the other 50% did not. Using nested cross-validation for training and a held-out test set (80/20 split), we systematically examined the combination of 8 connectivity feature sets and 17 classifiers. We found that a generative embedding procedure based on combining regression DCM (rDCM) with a support vector machine (SVM) enabled the best predictions, both on the training set (0.63 accuracy, 0.66 area under the curve, AUC) and the test set (0.62 accuracy, 0.64 AUC; p<0.001). However, on the test set, rDCM was only slightly superior to predictions based on FC (0.59 accuracy, 0.61 AUC). Interpreting model predictions based on SHAP (SHapley Additive exPlanations) values suggested that the most predictive connections were widely distributed and not confined to specific networks. Overall, our analyses suggest (i) ways of improving future fMRI-based generative embedding approaches for the early detection of individuals at-risk for depression and that (ii) achieving accuracies of clinical utility may require combination of fMRI with other data modalities.

Predicting the presence of depressive symptoms in the HIV-HCV co-infected population in Canada using supervised machine learning

BACKGROUND

Depression is common in the human immunodeficiency virus (HIV)-hepatitis C virus (HCV) co-infected population. Demographic, behavioural, and clinical data collected in research settings may be of help in identifying those at risk for clinical depression. We aimed to predict the presence of depressive symptoms indicative of a risk of depression and identify important classification predictors using supervised machine learning.

METHODS

We used data from the Canadian Co-infection Cohort, a multicentre prospective cohort, and its associated sub-study on Food Security (FS). The Center for Epidemiologic Studies Depression Scale-10 (CES-D-10) was administered in the FS sub-study; participants were classified as being at risk for clinical depression if scores ≥ 10. We developed two random forest algorithms using the training data (80%) and tenfold cross validation to predict the CES-D-10 classes-1. Full algorithm with all candidate predictors (137 predictors) and 2. Reduced algorithm using a subset of predictors based on expert opinion (46 predictors). We evaluated the algorithm performances in the testing data using area under the receiver operating characteristic curves (AUC) and generated predictor importance plots.

RESULTS

We included 1,934 FS sub-study visits from 717 participants who were predominantly male (73%), white (76%), unemployed (73%), and high school educated (52%). At the first visit, median age was 49 years (IQR:43-54) and 53% reported presence of depressive symptoms with CES-D-10 scores ≥ 10. The full algorithm had an AUC of 0.82 (95% CI:0.78-0.86) and the reduced algorithm of 0.76 (95% CI:0.71-0.81). Employment, HIV clinical stage, revenue source, body mass index, and education were the five most important predictors.

CONCLUSION

We developed a prediction algorithm that could be instrumental in identifying individuals at risk for depression in the HIV-HCV co-infected population in research settings. Development of such machine learning algorithms using research data with rich predictor information can be useful for retrospective analyses of unanswered questions regarding impact of depressive symptoms on clinical and patient-centred outcomes among vulnerable populations.

Prediction of depression cases, incidence, and chronicity in a large occupational cohort using machine learning techniques: an analysis of the ELSA-Brasil study

Abstract.

BACKGROUND

Depression is highly prevalent and marked by a chronic and recurrent course. Despite being a major cause of disability worldwide, little is known regarding the determinants of its heterogeneous course. Machine learning techniques present an opportunity to develop tools to predict diagnosis and prognosis at an individual level.

METHODS

We examined baseline (2008-2010) and follow-up (2012-2014) data of the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil), a large occupational cohort study. We implemented an elastic net regularization analysis with a 10-fold cross-validation procedure using socioeconomic and clinical factors as predictors to distinguish at follow-up: (1) depressed from non-depressed participants, (2) participants with incident depression from those who did not develop depression, and (3) participants with chronic (persistent or recurrent) depression from those without depression.

RESULTS

We assessed 15 105 and 13 922 participants at waves 1 and 2, respectively. The elastic net regularization model distinguished outcome levels in the test dataset with an area under the curve of 0.79 (95% CI 0.76-0.82), 0.71 (95% CI 0.66-0.77), 0.90 (95% CI 0.86-0.95) for analyses 1, 2, and 3, respectively.

CONCLUSIONS

Diagnosis and prognosis related to depression can be predicted at an individual subject level by integrating low-cost variables, such as demographic and clinical data. Future studies should assess longer follow-up periods and combine biological predictors, such as genetics and blood biomarkers, to build more accurate tools to predict depression course.

Common and specific determinants of 9-year depression and anxiety course-trajectories: A machine-learning investigation in the Netherlands Study of Depression and Anxiety (NESDA)

BACKGROUND

Given the strong relationship between depression and anxiety, there is an urge to investigate their shared and specific long-term course determinants. The current study aimed to identify and compare the main determinants of the 9-year trajectories of combined and pure depression and anxiety symptom severity.

METHODS

Respondents with a 6-month depression and/or anxiety diagnosis (n=1,701) provided baseline data on 152 sociodemographic, clinical and biological variables. Depression and anxiety symptom severity assessed at baseline, 2-, 4-, 6- and 9-year follow-up, were used to identify data-driven course-trajectory subgroups for general psychological distress, pure depression, and pure anxiety severity scores. For each outcome (class-probability), a Superlearner (SL) algorithm identified an optimally weighted (minimum mean squared error) combination of machine-learning prediction algorithms. For each outcome, the top determinants in the SL were identified by determining variable-importance and correlations between each SL-predicted and observed outcome (ρ_pred) were calculated.

RESULTS

Low to high prediction correlations (ρ_pred: 0.41-0.91, median=0.73) were found. In the SL, important determinants of psychological distress were age, young age of onset, respiratory rate, participation disability, somatic disease, low income, minor depressive disorder and mastery score. For course of pure depression and anxiety symptom severity, similar determinants were found. Specific determinants of pure depression included several types of healthcare-use, and of pure-anxiety course included somatic arousal and psychological distress.

LIMITATIONS

Limited sample size for machine learning.

CONCLUSIONS

The determinants of depression- and anxiety-severity course are mostly shared. Domain-specific exceptions are healthcare use for depression and somatic arousal and distress for anxiety-severity course.

A Machine Learning Approach to Predicting New-onset Depression in a Military Population

OBJECTIVE

Depression is one of the most common mental disorders in the United States in both civilian and military populations, but few prospective studies assess a wide range of predictors across multiple domains for new-onset (incident) depression in adulthood. Supervised machine learning methods can identify predictors of incident depression out of many different candidate variables, without some of the assumptions and constraints that underlie traditional regression analyses. The objectives of this study were to identify predictors of incident depression across 5 years of follow-up using machine learning, and to assess prediction accuracy of the algorithms.

METHODS

Data were from a cohort of Army National Guard members free of history of depression at baseline (n = 1951 men and 298 women), interviewed once per year for probable depression. Classification trees and random forests were constructed and cross-validated, using 84 candidate predictors from the baseline interviews.

RESULTS

Stressors and traumas such as emotional mistreatment and adverse childhood experiences, demographics such as being a parent or student, and military characteristics including paygrade and deployment location were predictive of probable depression. Cross-validated random forest algorithms were moderately accurate (68% for women and 73% for men).

CONCLUSIONS

Events and characteristics throughout the life course, both in and outside of deployment, predict incident depression in adulthood among military personnel. Although replication studies are needed, these results may help inform potential intervention targets to reduce depression incidence among military personnel. Future research should further refine and explore interactions between identified variables.

Development of Digitally Obtainable 10-Year Risk Scores for Depression and Anxiety in the General Population

The burden of depression and anxiety in the world is rising. Identification of individuals at increased risk of developing these conditions would help to target them for prevention and ultimately reduce the healthcare burden. We developed a 10-year predictive algorithm for depression and anxiety using the full cohort of over 400,000 UK Biobank (UKB) participants without pre-existing depression or anxiety using digitally obtainable information. From the initial 167 variables selected from UKB, processed into 429 features, iterative backward elimination using Cox proportional hazards model was performed to select predictors which account for the majority of its predictive capability. Baseline and reduced models were then trained for depression and anxiety using both Cox and DeepSurv, a deep neural network approach to survival analysis. The baseline Cox model achieved concordance of 0.7772 and 0.7720 on the validation dataset for depression and anxiety, respectively. For the DeepSurv model, respective concordance indices were 0.7810 and 0.7728. After feature selection, the depression model contained 39 predictors and the concordance index was 0.7769 for Cox and 0.7772 for DeepSurv. The reduced anxiety model, with 53 predictors, achieved concordance of 0.7699 for Cox and 0.7710 for DeepSurv. The final models showed good discrimination and calibration in the test datasets. We developed predictive risk scores with high discrimination for depression and anxiety using the UKB cohort, incorporating predictors which are easily obtainable via smartphone. If deployed in a digital solution, it would allow individuals to track their risk, as well as provide some pointers to how to decrease it through lifestyle changes.

Can machine learning be useful as a screening tool for depression in primary care?

Depression is a widespread disease with a high economic burden and a complex pathophysiology disease that is still not wholly clarified, not to mention it usually is associated as a risk factor for absenteeism at work and suicide. Just 50% of patients with depression are diagnosed in primary care, and only 15% receive treatment. Stigmatization, the coexistence of somatic symptoms, and the need to remember signs in the past two weeks can contribute to explaining this situation. In this context, tools that can serve as diagnostic screening are of great value, as they can reduce the number of undiagnosed patients. Besides, Artificial Intelligence (AI) has enabled several fruitful applications in medicine, particularly in psychiatry. This study aims to evaluate the performance of Machine Learning (ML) algorithms in the detection of depressive patients from the clinical, laboratory, and sociodemographic data obtained from the Brazilian National Network for Research on Cardiovascular Diseases from June 2016 to July 2018. The results obtained are promising. In one of them, Random Forests, the accuracy, sensibility, and area under the receiver operating characteristic curve were, respectively, 0.89, 0.90, and 0.87.

Prediction of depression among medical check-ups of 433,190 patients: A nationwide population-based study

Depression is a mental illness that causes significant disturbances in daily life. Depression is commonly associated with low mood, severe health problems, and substantial socioeconomic burden; hence, it is necessary to be able to detect depression earlier. We utilized the medical check-up cohort database of the National Health Insurance Sharing Service in Korea. We split the total dataset into training (70%) and test (30%) sets. Subsequently, five-fold cross validation was performed in the training set. The holdout test set was only used in the last step to evaluate the performance of the predictive model. Random forest algorithm was used for the predictive model. The analysis included 433,190 individuals who had a national medical check-up from 2009-2015, which included 10,824 (2.56%) patients in the depression group. The area under the receiver-operating curve was 0.849. Other performance metrics included a sensitivity of 0.737, specificity of 0.824, positive predictive value of 0.097, negative predictive value of 0.992, and accuracy of 0.780. Our predictive model could contribute to proactively reducing depression prevalence by administering interventions to prevent depression in patients receiving medical check-up. Future studies are needed to prospectively validate the predictability of this model.

Supervised Machine Learning: A Brief Primer

Machine learning is increasingly used in mental health research and has the potential to advance our understanding of how to characterize, predict, and treat mental disorders and associated adverse health outcomes (e.g., suicidal behavior). Machine learning offers new tools to overcome challenges for which traditional statistical methods are not well-suited. This paper provides an overview of machine learning with a specific focus on supervised learning (i.e., methods that are designed to predict or classify an outcome of interest). Several common supervised learning methods are described, along with applied examples from the published literature. We also provide an overview of supervised learning model building, validation, and performance evaluation. Finally, challenges in creating robust and generalizable machine learning algorithms are discussed.