A Systematic Evaluation of Machine Learning-Based Biomarkers for Major Depressive Disorder

Importance

Biological psychiatry aims to understand mental disorders in terms of altered neurobiological pathways. However, for one of the most prevalent and disabling mental disorders, major depressive disorder (MDD), no informative biomarkers have been identified.

Objective

To evaluate whether machine learning (ML) can identify a multivariate biomarker for MDD.

Design, Setting, and Participants

This study used data from the Marburg-Münster Affective Disorders Cohort Study, a case-control clinical neuroimaging study. Patients with acute or lifetime MDD and healthy controls aged 18 to 65 years were recruited from primary care and the general population in Münster and Marburg, Germany, from September 11, 2014, to September 26, 2018. The Münster Neuroimaging Cohort (MNC) was used as an independent partial replication sample. Data were analyzed from April 2022 to June 2023.

Exposure

Patients with MDD and healthy controls.

Main Outcome and Measure

Diagnostic classification accuracy was quantified on an individual level using an extensive ML-based multivariate approach across a comprehensive range of neuroimaging modalities, including structural and functional magnetic resonance imaging and diffusion tensor imaging as well as a polygenic risk score for depression.

Results

Of 1801 included participants, 1162 (64.5%) were female, and the mean (SD) age was 36.1 (13.1) years. There were a total of 856 patients with MDD (47.5%) and 945 healthy controls (52.5%). The MNC replication sample included 1198 individuals (362 with MDD [30.1%] and 836 healthy controls [69.9%]). Training and testing a total of 4 million ML models, mean (SD) accuracies for diagnostic classification ranged between 48.1% (3.6%) and 62.0% (4.8%). Integrating neuroimaging modalities and stratifying individuals based on age, sex, treatment, or remission status does not enhance model performance. Findings were replicated within study sites and also observed in structural magnetic resonance imaging within MNC. Under simulated conditions of perfect reliability, performance did not significantly improve. Analyzing model errors suggests that symptom severity could be a potential focus for identifying MDD subgroups.

Conclusion and Relevance

Despite the improved predictive capability of multivariate compared with univariate neuroimaging markers, no informative individual-level MDD biomarker-even under extensive ML optimization in a large sample of diagnosed patients-could be identified.

Reduced fractional anisotropy in bipolar disorder v. major depressive disorder independent of current symptoms

BACKGROUND

Patients with bipolar disorder (BD) show reduced fractional anisotropy (FA) compared to patients with major depressive disorder (MDD). Little is known about whether these differences are mood state-independent or influenced by acute symptom severity. Therefore, the aim of this study was (1) to replicate abnormalities in white matter microstructure in BD v. MDD and (2) to investigate whether these vary across depressed, euthymic, and manic mood.

METHODS

In this cross-sectional diffusion tensor imaging study, n = 136 patients with BD were compared to age- and sex-matched MDD patients and healthy controls (HC) (n = 136 each). Differences in FA were investigated using tract-based spatial statistics. Using interaction models, the influence of acute symptom severity and mood state on the differences between patient groups were tested.

RESULTS

Analyses revealed a main effect of diagnosis on FA across all three groups (ptfce-FWE = 0.003). BD patients showed reduced FA compared to both MDD (ptfce-FWE = 0.005) and HC (ptfce-FWE < 0.001) in large bilateral clusters. These consisted of several white matter tracts previously described in the literature, including commissural, association, and projection tracts. There were no significant interaction effects between diagnosis and symptom severity or mood state (all ptfce-FWE > 0.704).

CONCLUSIONS

Results indicated that the difference between BD and MDD was independent of depressive and manic symptom severity and mood state. Disruptions in white matter microstructure in BD might be a trait effect of the disorder. The potential of FA values to be used as a biomarker to differentiate BD from MDD should be further addressed in future studies using longitudinal designs.

Genetic, individual, and familial risk correlates of brain network controllability in major depressive disorder

Many therapeutic interventions in psychiatry can be viewed as attempts to influence the brain's large-scale, dynamic network state transitions. Building on connectome-based graph analysis and control theory, Network Control Theory is emerging as a powerful tool to quantify network controllability-i.e., the influence of one brain region over others regarding dynamic network state transitions. If and how network controllability is related to mental health remains elusive. Here, from Diffusion Tensor Imaging data, we inferred structural connectivity and inferred calculated network controllability parameters to investigate their association with genetic and familial risk in patients diagnosed with major depressive disorder (MDD, n = 692) and healthy controls (n = 820). First, we establish that controllability measures differ between healthy controls and MDD patients while not varying with current symptom severity or remission status. Second, we show that controllability in MDD patients is associated with polygenic scores for MDD and psychiatric cross-disorder risk. Finally, we provide evidence that controllability varies with familial risk of MDD and bipolar disorder as well as with body mass index. In summary, we show that network controllability is related to genetic, individual, and familial risk in MDD patients. We discuss how these insights into individual variation of network controllability may inform mechanistic models of treatment response prediction and personalized intervention-design in mental health.

Towards a network control theory of electroconvulsive therapy response

Electroconvulsive Therapy (ECT) is arguably the most effective intervention for treatment-resistant depression. While large interindividual variability exists, a theory capable of explaining individual response to ECT remains elusive. To address this, we posit a quantitative, mechanistic framework of ECT response based on Network Control Theory (NCT). Then, we empirically test our approach and employ it to predict ECT treatment response. To this end, we derive a formal association between Postictal Suppression Index (PSI)-an ECT seizure quality index-and whole-brain modal and average controllability, NCT metrics based on white-matter brain network architecture, respectively. Exploiting the known association of ECT response and PSI, we then hypothesized an association between our controllability metrics and ECT response mediated by PSI. We formally tested this conjecture in N = 50 depressive patients undergoing ECT. We show that whole-brain controllability metrics based on pre-ECT structural connectome data predict ECT response in accordance with our hypotheses. In addition, we show the expected mediation effects via PSI. Importantly, our theoretically motivated metrics are at least on par with extensive machine learning models based on pre-ECT connectome data. In summary, we derived and tested a control-theoretic framework capable of predicting ECT response based on individual brain network architecture. It makes testable, quantitative predictions regarding individual therapeutic response, which are corroborated by strong empirical evidence. Our work might constitute a starting point for a comprehensive, quantitative theory of personalized ECT interventions rooted in control theory.

Quantifying Deviations of Brain Structure and Function in Major Depressive Disorder Across Neuroimaging Modalities

IMPORTANCE

Identifying neurobiological differences between patients with major depressive disorder (MDD) and healthy individuals has been a mainstay of clinical neuroscience for decades. However, recent meta-analyses have raised concerns regarding the replicability and clinical relevance of brain alterations in depression.

OBJECTIVE

To quantify the upper bounds of univariate effect sizes, estimated predictive utility, and distributional dissimilarity of healthy individuals and those with depression across structural magnetic resonance imaging (MRI), diffusion-tensor imaging, and functional task-based as well as resting-state MRI, and to compare results with an MDD polygenic risk score (PRS) and environmental variables.

DESIGN, SETTING, AND PARTICIPANTS

This was a cross-sectional, case-control clinical neuroimaging study. Data were part of the Marburg-Münster Affective Disorders Cohort Study. Patients with depression and healthy controls were recruited from primary care and the general population in Münster and Marburg, Germany. Study recruitment was performed from September 11, 2014, to September 26, 2018. The sample comprised patients with acute and chronic MDD as well as healthy controls in the age range of 18 to 65 years. Data were analyzed from October 29, 2020, to April 7, 2022.

MAIN OUTCOMES AND MEASURES

Primary analyses included univariate partial effect size (η2), classification accuracy, and distributional overlapping coefficient for healthy individuals and those with depression across neuroimaging modalities, controlling for age, sex, and additional modality-specific confounding variables. Secondary analyses included patient subgroups for acute or chronic depressive status.

RESULTS

A total of 1809 individuals (861 patients [47.6%] and 948 controls [52.4%]) were included in the analysis (mean [SD] age, 35.6 [13.2] years; 1165 female patients [64.4%]). The upper bound of the effect sizes of the single univariate measures displaying the largest group difference ranged from partial η2 of 0.004 to 0.017, and distributions overlapped between 87% and 95%, with classification accuracies ranging between 54% and 56% across neuroimaging modalities. This pattern remained virtually unchanged when considering either only patients with acute or chronic depression. Differences were comparable with those found for PRS but substantially smaller than for environmental variables.

CONCLUSIONS AND RELEVANCE

Results of this case-control study suggest that even for maximum univariate biological differences, deviations between patients with MDD and healthy controls were remarkably small, single-participant prediction was not possible, and similarity between study groups dominated. Biological psychiatry should facilitate meaningful outcome measures or predictive approaches to increase the potential for a personalization of the clinical practice.

Recommendations for machine learning benchmarks in neuroimaging

The field of neuroimaging has embraced methods from machine learning in a variety of ways. Although an increasing number of initiatives have published open-access neuroimaging datasets, specifically designed benchmarks are rare in the field. In this article, we first describe how benchmarks in computer science and biomedical imaging have fostered methodological progress in machine learning. Second, we identify the special characteristics of neuroimaging data and outline what researchers have to ensure when establishing a neuroimaging benchmark, how datasets should be composed and how adequate evaluation criteria can be chosen. Based on lessons learned from machine learning benchmarks, we argue for an extended evaluation procedure that, next to applying suitable performance metrics, focuses on scientifically relevant aspects such as explainability, robustness, uncertainty, computational efficiency and code quality. Lastly, we envision a collaborative neuroimaging benchmarking platform that combines the discussed aspects in a collaborative and agile framework, allowing researchers across disciplines to work together on the key predictive problems of the field of neuroimaging and psychiatry.

An uncertainty-aware, shareable, and transparent neural network architecture for brain-age modeling

The deviation between chronological age and age predicted from neuroimaging data has been identified as a sensitive risk marker of cross-disorder brain changes, growing into a cornerstone of biological age research. However, machine learning models underlying the field do not consider uncertainty, thereby confounding results with training data density and variability. Also, existing models are commonly based on homogeneous training sets, often not independently validated, and cannot be shared because of data protection issues. Here, we introduce an uncertainty-aware, shareable, and transparent Monte Carlo dropout composite quantile regression (MCCQR) Neural Network trained on N = 10,691 datasets from the German National Cohort. The MCCQR model provides robust, distribution-free uncertainty quantification in high-dimensional neuroimaging data, achieving lower error rates compared with existing models. In two examples, we demonstrate that it prevents spurious associations and increases power to detect deviant brain aging. We make the pretrained model and code publicly available.

Identification of transdiagnostic psychiatric disorder subtypes using unsupervised learning

Psychiatric disorders show heterogeneous symptoms and trajectories, with current nosology not accurately reflecting their molecular etiology and the variability and symptomatic overlap within and between diagnostic classes. This heterogeneity impedes timely and targeted treatment. Our study aimed to identify psychiatric patient clusters that share clinical and genetic features and may profit from similar therapies. We used high-dimensional data clustering on deep clinical data to identify transdiagnostic groups in a discovery sample (N = 1250) of healthy controls and patients diagnosed with depression, bipolar disorder, schizophrenia, schizoaffective disorder, and other psychiatric disorders. We observed five diagnostically mixed clusters and ordered them based on severity. The least impaired cluster 0, containing most healthy controls, showed general well-being. Clusters 1-3 differed predominantly regarding levels of maltreatment, depression, daily functioning, and parental bonding. Cluster 4 contained most patients diagnosed with psychotic disorders and exhibited the highest severity in many dimensions, including medication load. Depressed patients were present in all clusters, indicating that we captured different disease stages or subtypes. We replicated all but the smallest cluster 1 in an independent sample (N = 622). Next, we analyzed genetic differences between clusters using polygenic scores (PGS) and the psychiatric family history. These genetic variables differed mainly between clusters 0 and 4 (prediction area under the receiver operating characteristic curve (AUC) = 81%; significant PGS: cross-disorder psychiatric risk, schizophrenia, and educational attainment). Our results confirm that psychiatric disorders consist of heterogeneous subtypes sharing molecular factors and symptoms. The identification of transdiagnostic clusters advances our understanding of the heterogeneity of psychiatric disorders and may support the development of personalized treatments.

Brain aging in major depressive disorder: results from the ENIGMA major depressive disorder working group

Major depressive disorder (MDD) is associated with an increased risk of brain atrophy, aging-related diseases, and mortality. We examined potential advanced brain aging in adult MDD patients, and whether this process is associated with clinical characteristics in a large multicenter international dataset. We performed a mega-analysis by pooling brain measures derived from T1-weighted MRI scans from 19 samples worldwide. Healthy brain aging was estimated by predicting chronological age (18-75 years) from 7 subcortical volumes, 34 cortical thickness and 34 surface area, lateral ventricles and total intracranial volume measures separately in 952 male and 1236 female controls from the ENIGMA MDD working group. The learned model coefficients were applied to 927 male controls and 986 depressed males, and 1199 female controls and 1689 depressed females to obtain independent unbiased brain-based age predictions. The difference between predicted "brain age" and chronological age was calculated to indicate brain-predicted age difference (brain-PAD). On average, MDD patients showed a higher brain-PAD of +1.08 (SE 0.22) years (Cohen's d = 0.14, 95% CI: 0.08-0.20) compared with controls. However, this difference did not seem to be driven by specific clinical characteristics (recurrent status, remission status, antidepressant medication use, age of onset, or symptom severity). This highly powered collaborative effort showed subtle patterns of age-related structural brain abnormalities in MDD. Substantial within-group variance and overlap between groups were observed. Longitudinal studies of MDD and somatic health outcomes are needed to further assess the clinical value of these brain-PAD estimates.

Systematic misestimation of machine learning performance in neuroimaging studies of depression

We currently observe a disconcerting phenomenon in machine learning studies in psychiatry: While we would expect larger samples to yield better results due to the availability of more data, larger machine learning studies consistently show much weaker performance than the numerous small-scale studies. Here, we systematically investigated this effect focusing on one of the most heavily studied questions in the field, namely the classification of patients suffering from Major Depressive Disorder (MDD) and healthy controls based on neuroimaging data. Drawing upon structural MRI data from a balanced sample of N = 1868 MDD patients and healthy controls from our recent international Predictive Analytics Competition (PAC), we first trained and tested a classification model on the full dataset which yielded an accuracy of 61%. Next, we mimicked the process by which researchers would draw samples of various sizes (N = 4 to N = 150) from the population and showed a strong risk of misestimation. Specifically, for small sample sizes (N = 20), we observe accuracies of up to 95%. For medium sample sizes (N = 100) accuracies up to 75% were found. Importantly, further investigation showed that sufficiently large test sets effectively protect against performance misestimation whereas larger datasets per se do not. While these results question the validity of a substantial part of the current literature, we outline the relatively low-cost remedy of larger test sets, which is readily available in most cases.

FreeSurfer ‐based segmentation of hippocampal subfields: A review of methods and applications, with a novel quality control procedure for ENIGMA studies and other collaborative efforts

Structural hippocampal abnormalities are common in many neurological and psychiatric disorders, and variation in hippocampal measures is related to cognitive performance and other complex phenotypes such as stress sensitivity. Hippocampal subregions are increasingly studied, as automated algorithms have become available for mapping and volume quantification. In the context of the Enhancing Neuro Imaging Genetics through Meta Analysis Consortium, several Disease Working Groups are using the FreeSurfer software to analyze hippocampal subregion (subfield) volumes in patients with neurological and psychiatric conditions along with data from matched controls. In this overview, we explain the algorithm's principles, summarize measurement reliability studies, and demonstrate two additional aspects (subfield autocorrelation and volume/reliability correlation) with illustrative data. We then explain the rationale for a standardized hippocampal subfield segmentation quality control (QC) procedure for improved pipeline harmonization. To guide researchers to make optimal use of the algorithm, we discuss how global size and age effects can be modeled, how QC steps can be incorporated and how subfields may be aggregated into composite volumes. This discussion is based on a synopsis of 162 published neuroimaging studies (01/2013–12/2019) that applied the FreeSurfer hippocampal subfield segmentation in a broad range of domains including cognition and healthy aging, brain development and neurodegeneration, affective disorders, psychosis, stress regulation, neurotoxicity, epilepsy, inflammatory disease, childhood adversity and posttraumatic stress disorder, and candidate and whole genome (epi‐)genetics. Finally, we highlight points where FreeSurfer‐based hippocampal subfield studies may be optimized.

Predicting intelligence from brain gray matter volume

A positive association between brain size and intelligence is firmly established, but whether region-specific anatomical differences contribute to general intelligence remains an open question. Results from voxel-based morphometry (VBM) - one of the most widely used morphometric methods - have remained inconclusive so far. Here, we applied cross-validated machine learning-based predictive modeling to test whether out-of-sample prediction of individual intelligence scores is possible on the basis of voxel-wise gray matter volume. Features were derived from structural magnetic resonance imaging data (N = 308) using (a) a purely data-driven method (principal component analysis) and (b) a domain knowledge-based approach (atlas parcellation). When using relative gray matter (corrected for total brain size), only the atlas-based approach provided significant prediction, while absolute gray matter (uncorrected) allowed for above-chance prediction with both approaches. Importantly, in all significant predictions, the absolute error was relatively high, i.e., greater than ten IQ points, and in the atlas-based models, the predicted IQ scores varied closely around the sample mean. This renders the practical value even of statistically significant prediction results questionable. Analyses based on the gray matter of functional brain networks yielded significant predictions for the fronto-parietal network and the cerebellum. However, the mean absolute errors were not reduced in contrast to the global models, suggesting that general intelligence may be related more to global than region-specific differences in gray matter volume. More generally, our study highlights the importance of predictive statistical analysis approaches for clarifying the neurobiological bases of intelligence and provides important suggestions for future research using predictive modeling.

Replication of a hippocampus specific effect of the tescalcin regulating variant rs7294919 on gray matter structure

While the hippocampus remains a region of high interest for neuropsychiatric research, the precise contributors to hippocampal morphometry are still not well understood. We and others previously reported a hippocampus specific effect of a tescalcin gene (TESC) regulating single nucleotide polymorphism (rs7294919) on gray matter volume. Here we aimed to replicate and extend these findings. Two complementary morphometric approaches (voxel based morphometry (VBM) and automated volumetric segmentation) were applied in a well-powered cohort from the Marburg-Münster Affective Disorder Cohort Study (MACS) including N=1137 participants (n=636 healthy controls, n=501 depressed patients). rs7294919 homozygous T-allele genotype was significantly associated with lower hippocampal gray matter density as well as with reduced hippocampal volume. Exploratory whole brain VBM analyses revealed no further associations with gray matter volume outside the hippocampus. No interaction effects of rs7294919 with depression nor with childhood trauma on hippocampal morphometry could be detected. Hippocampal subfield analyses revealed similar effects of rs7294919 in all hippocampal subfields. In sum, our results replicate a hippocampus specific effect of rs7294919 on brain structure. Due to the robust evidence for a pronounced association between the reported polymorphism and hippocampal morphometry, future research should consider investigating the potential clinical and functional relevance of the reported association.

Influence of electroconvulsive therapy on white matter structure in a diffusion tensor imaging study

BACKGROUND

Electroconvulsive therapy (ECT) is a fast-acting intervention for major depressive disorder. Previous studies indicated neurotrophic effects following ECT that might contribute to changes in white matter brain structure. We investigated the influence of ECT in a non-randomized prospective study focusing on white matter changes over time.

METHODS

Twenty-nine severely depressed patients receiving ECT in addition to inpatient treatment, 69 severely depressed patients with inpatient treatment (NON-ECT) and 52 healthy controls (HC) took part in a non-randomized prospective study. Participants were scanned twice, approximately 6 weeks apart, using diffusion tensor imaging, applying tract-based spatial statistics. Additional correlational analyses were conducted in the ECT subsample to investigate the effects of seizure duration and therapeutic response.

RESULTS

Mean diffusivity (MD) increased after ECT in the right hemisphere, which was an ECT-group-specific effect. Seizure duration was associated with decreased fractional anisotropy (FA) following ECT. Longitudinal changes in ECT were not associated with therapy response. However, within the ECT group only, baseline FA was positively and MD negatively associated with post-ECT symptomatology.

CONCLUSION

Our data suggest that ECT changes white matter integrity, possibly reflecting increased permeability of the blood-brain barrier, resulting in disturbed communication of fibers. Further, baseline diffusion metrics were associated with therapy response. Coherent fiber structure could be a prerequisite for a generalized seizure and inhibitory brain signaling necessary to successfully inhibit increased seizure activity.

Childhood maltreatment moderates the influence of genetic load for obesity on reward related brain structure and function in major depression

Obesity is a clinically relevant and highly prevalent somatic comorbidity of major depression (MDD). Genetic predisposition and history of childhood trauma have both independently been demonstrated to act as risk factors for obesity and to be associated with alterations in reward related brain structure and function. We therefore aimed to investigate the influence of childhood maltreatment and genetic risk for obesity on structural and functional imaging correlates associated with reward processing in MDD. 161 MDD patients underwent structural and functional MRI during a frequently used card guessing paradigm. Main and interaction effects of a polygenic risk score for obesity (PRS) and childhood maltreatment experiences as assessed using the Childhood Trauma Questionnaire (CTQ) were investigated. We found that maltreatment experiences and polygenic risk for obesity significantly interacted on a) body mass index b) gray matter volume of the orbitofrontal cortex as well as on c) BOLD response in the right insula during reward processing. While polygenic risk for obesity was associated with elevated BMI as well as with decreased OFC gray matter and increased insular BOLD response in non-maltreated patients, these associations were absent in patients with a history of childhood trauma. No significant main effect of PRS or maltreatment on gray matter or BOLD response could be detected at the applied thresholds. The present study suggests that childhood maltreatment moderates the influence of genetic load for obesity on BMI as well as on altered brain structure and function in reward related brain circuits in MDD.