Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis: A Systematic Review

Applications of MRI in Schizophrenia: Current Progress in Establishing Clinical Utility

Schizophrenia is a severe mental illness that significantly impacts the lives of affected individuals and with increasing mortality rates. Early detection and intervention are crucial for improving outcomes but the lack of validated biomarkers poses great challenges in such efforts. The use of magnetic resonance imaging (MRI) in schizophrenia enables the investigation of the disorder's etiological and neuropathological substrates in vivo. After decades of research, promising findings of MRI have been shown to aid in screening high-risk individuals and predicting illness onset, and predicting symptoms and treatment outcomes of schizophrenia. The integration of machine learning and deep learning techniques makes it possible to develop intelligent diagnostic and prognostic tools with extracted or selected imaging features. In this review, we aimed to provide an overview of current progress and prospects in establishing clinical utility of MRI in schizophrenia. We first provided an overview of MRI findings of brain abnormalities that might underpin the symptoms or treatment response process in schizophrenia patients. Then, we summarized the ongoing efforts in the computer-aided utility of MRI in schizophrenia and discussed the gap between MRI research findings and real-world applications. Finally, promising pathways to promote clinical translation were provided. EVIDENCE LEVEL: 2 TECHNICAL EFFICACY: Stage 3.

Racial and ethnic socioenvironmental inequity and neuroimaging in psychiatry: a brief review of the past and recommendations for the future

Neuroimaging is a major tool that holds immense translational potential for understanding psychiatric disorder phenomenology and treatment. However, although epidemiological and social research highlights the many ways inequity and representativeness influences mental health, there is a lack of consideration of how such issues may impact neuroimaging features in psychiatric research. More specifically, the potential extent to which racialized inequities may affect underlying neurobiology and impact the generalizability of neural models of disorders is unclear. The present review synthesizes research focused on understanding the potential consequences of racial/ethnic inequities relevant to neuroimaging in psychiatry. We first discuss historical and contemporary drivers of inequities that persist today. We then discuss the neurobiological consequences of these inequities as revealed through current research, and note emergent research demonstrating the impact such inequities have on our ability to use neuroimaging to understand psychiatric disease. We end with a set of recommendations and practices to move the field towards more equitable approaches that will advance our abilities to develop truly generalizable neurobiological models of psychiatric disorders.

A new era in cognitive neuroscience: the tidal wave of artificial intelligence (AI)

Translating artificial intelligence techniques into the realm of cognitive neuroscience holds promise for significant breakthroughs in our ability to probe the intrinsic mechanisms of the brain. The recent unprecedented development of robust AI models is changing how and what we understand about the brain. In this Editorial, we invite contributions for a BMC Neuroscience Collection on "AI and Cognitive Neuroscience".

Transcriptomic and neuroimaging data integration enhances machine learning classification of schizophrenia

Background

Schizophrenia is a polygenic disorder associated with changes in brain structure and function. Integrating macroscale brain features with microscale genetic data may provide a more complete overview of the disease etiology and may serve as potential diagnostic markers for schizophrenia.

Objective

We aim to systematically evaluate the impact of multi-scale neuroimaging and transcriptomic data fusion in schizophrenia classification models.

Methods

We collected brain imaging data and blood RNA sequencing data from 43 patients with schizophrenia and 60 age- and gender-matched healthy controls, and we extracted multi-omics features of macroscale brain morphology, brain structural and functional connectivity, and gene transcription of schizophrenia risk genes. Multi-scale data fusion was performed using a machine learning integration framework, together with several conventional machine learning methods and neural networks for patient classification.

Results

We found that multi-omics data fusion in conventional machine learning models achieved the highest accuracy (AUC ~0.76-0.92) in contrast to the single-modality models, with AUC improvements of 8.88 to 22.64%. Similar findings were observed for the neural network, showing an increase of 16.57% for the multimodal classification model (accuracy 71.43%) compared to the single-modal average. In addition, we identified several brain regions in the left posterior cingulate and right frontal pole that made a major contribution to disease classification.

Conclusion

We provide empirical evidence for the increased accuracy achieved by imaging genetic data integration in schizophrenia classification. Multi-scale data fusion holds promise for enhancing diagnostic precision, facilitating early detection and personalizing treatment regimens in schizophrenia.

Data leakage inflates prediction performance in connectome-based machine learning models

Predictive modeling is a central technique in neuroimaging to identify brain-behavior relationships and test their generalizability to unseen data. However, data leakage undermines the validity of predictive models by breaching the separation between training and test data. Leakage is always an incorrect practice but still pervasive in machine learning. Understanding its effects on neuroimaging predictive models can inform how leakage affects existing literature. Here, we investigate the effects of five forms of leakage-involving feature selection, covariate correction, and dependence between subjects-on functional and structural connectome-based machine learning models across four datasets and three phenotypes. Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects. Furthermore, small datasets exacerbate the effects of leakage. Overall, our results illustrate the variable effects of leakage and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

The effects of data leakage on connectome-based machine learning models

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Updated Perspectives on the Neurobiology of Substance Use Disorders Using Neuroimaging

Substance use problems impair social functioning, academic achievement, and employability. Psychological, biological, social, and environmental factors can contribute to substance use disorders. In recent years, neuroimaging breakthroughs have helped elucidate the mechanisms of substance misuse and its effects on the brain. Functional magnetic resonance imaging (MRI), positron emission tomography (PET), single-photon emission computed tomography (SPECT), and magnetic resonance spectroscopy (MRS) are all examples. Neuroimaging studies suggest substance misuse affects executive function, reward, memory, and stress systems. Recent neuroimaging research attempts have provided clinicians with improved tools to diagnose patients who misuse substances, comprehend the complicated neuroanatomy and neurobiology involved, and devise individually tailored and monitorable treatment regimens for individuals with substance use disorders. This review describes the most recent developments in drug misuse neuroimaging, including the neurobiology of substance use disorders, neuroimaging, and substance use disorders, established neuroimaging techniques, recent developments with established neuroimaging techniques and substance use disorders, and emerging clinical neuroimaging technology.

Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

BACKGROUND

The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation.

METHODS

Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses.

RESULTS

A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (β =  - 2.75, p < .001, R²_adj = 0.40; r =  - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF₁₀ > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance.

CONCLUSIONS

Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.