Multi-center machine learning in imaging psychiatry: A meta-model approach

Application of image recognition technology in pathological diagnosis of blood smears

Traditional manual blood smear diagnosis methods are time-consuming and prone to errors, often relying heavily on the experience of clinical laboratory analysts for accuracy. As breakthroughs in key technologies such as neural networks and deep learning continue to drive digital transformation in the medical field, image recognition technology is increasingly being leveraged to enhance existing medical processes. In recent years, advancements in computer technology have led to improved efficiency in the identification of blood cells in blood smears through the use of image recognition technology. This paper provides a comprehensive summary of the methods and steps involved in utilizing image recognition algorithms for diagnosing diseases in blood smears, with a focus on malaria and leukemia. Furthermore, it offers a forward-looking research direction for the development of a comprehensive blood cell pathological detection system.

MP-Net: A Multi-Center Privacy-Preserving Network for Medical Image Segmentation

In this paper, we present the Multi-Center Privacy-Preserving Network (MP-Net), a novel framework designed for secure medical image segmentation in multi-center collaborations. Our methodology offers a new approach to multi-center collaborative learning, capable of reducing the volume of data transmission and enhancing data privacy protection. Unlike federated learning, which requires the transmission of model data between the central server and local servers in each round, our method only necessitates a single transfer of encrypted data. The proposed MP-Net comprises a three-layer model, consisting of encryption, segmentation, and decryption networks. We encrypt the image data into ciphertext using an encryption network and introduce an improved U-Net for image ciphertext segmentation. Finally, the segmentation mask is obtained through a decryption network. This architecture enables ciphertext-based image segmentation through computable image encryption. We evaluate the effectiveness of our approach on three datasets, including two cardiac MRI datasets and a CTPA dataset. Our results demonstrate that the MP-Net can securely utilize data from multiple centers to establish a more robust and information-rich segmentation model.

Radiomics: The New Promise for Differentiating Progression, Recurrence, Pseudoprogression, and Radionecrosis in Glioma and Glioblastoma Multiforme

Glioma and glioblastoma multiform (GBM) remain among the most debilitating and life-threatening brain tumors. Despite advances in diagnosing approaches, patient follow-up after treatment (surgery and chemoradiation) is still challenging for differentiation between tumor progression/recurrence, pseudoprogression, and radionecrosis. Radiomics emerges as a promising tool in initial diagnosis, grading, and survival prediction in patients with glioma and can help differentiate these post-treatment scenarios. Preliminary published studies are promising about the role of radiomics in post-treatment glioma/GBM. However, this field faces significant challenges, including a lack of evidence-based solid data, scattering publication, heterogeneity of studies, and small sample sizes. The present review explores radiomics's capabilities in following patients with glioma/GBM status post-treatment and to differentiate tumor progression, recurrence, pseudoprogression, and radionecrosis.

Machine Learning-Based Prediction of Distant Recurrence in Invasive Breast Carcinoma Using Clinicopathological Data: A Cross-Institutional Study

Breast cancer is the most common type of cancer worldwide. Alarmingly, approximately 30% of breast cancer cases result in disease recurrence at distant organs after treatment. Distant recurrence is more common in some subtypes such as invasive breast carcinoma (IBC). While clinicians have utilized several clinicopathological measurements to predict distant recurrences in IBC, no studies have predicted distant recurrences by combining clinicopathological evaluations of IBC tumors pre- and post-therapy with machine learning (ML) models. The goal of our study was to determine whether classification-based ML techniques could predict distant recurrences in IBC patients using key clinicopathological measurements, including pathological staging of the tumor and surrounding lymph nodes assessed both pre- and post-neoadjuvant therapy, response to therapy via standard-of-care imaging, and binary status of adjuvant therapy administered to patients. We trained and tested four clinicopathological ML models using a dataset (144 and 17 patients for training and testing, respectively) from Duke University and validated the best-performing model using an external dataset (8 patients) from Dartmouth Hitchcock Medical Center. The random forest model performed better than the C-support vector classifier, multilayer perceptron, and logistic regression models, yielding AUC values of 1.0 in the testing set and 0.75 in the validation set (p < 0.002) across both institutions, thereby demonstrating the cross-institutional portability and validity of ML models in the field of clinical research in cancer. The top-ranking clinicopathological measurement impacting the prediction of distant recurrences in IBC were identified to be tumor response to neoadjuvant therapy as evaluated via SOC imaging and pathology, which included tumor as well as node staging.

Sampling inequalities affect generalization of neuroimaging-based diagnostic classifiers in psychiatry

BACKGROUND

The development of machine learning models for aiding in the diagnosis of mental disorder is recognized as a significant breakthrough in the field of psychiatry. However, clinical practice of such models remains a challenge, with poor generalizability being a major limitation.

METHODS

Here, we conducted a pre-registered meta-research assessment on neuroimaging-based models in the psychiatric literature, quantitatively examining global and regional sampling issues over recent decades, from a view that has been relatively underexplored. A total of 476 studies (n = 118,137) were included in the current assessment. Based on these findings, we built a comprehensive 5-star rating system to quantitatively evaluate the quality of existing machine learning models for psychiatric diagnoses.

RESULTS

A global sampling inequality in these models was revealed quantitatively (sampling Gini coefficient (G) = 0.81, p < .01), varying across different countries (regions) (e.g., China, G = 0.47; the USA, G = 0.58; Germany, G = 0.78; the UK, G = 0.87). Furthermore, the severity of this sampling inequality was significantly predicted by national economic levels (β =  - 2.75, p < .001, R²_adj = 0.40; r =  - .84, 95% CI: - .41 to - .97), and was plausibly predictable for model performance, with higher sampling inequality for reporting higher classification accuracy. Further analyses showed that lack of independent testing (84.24% of models, 95% CI: 81.0-87.5%), improper cross-validation (51.68% of models, 95% CI: 47.2-56.2%), and poor technical transparency (87.8% of models, 95% CI: 84.9-90.8%)/availability (80.88% of models, 95% CI: 77.3-84.4%) are prevailing in current diagnostic classifiers despite improvements over time. Relating to these observations, model performances were found decreased in studies with independent cross-country sampling validations (all p < .001, BF₁₀ > 15). In light of this, we proposed a purpose-built quantitative assessment checklist, which demonstrated that the overall ratings of these models increased by publication year but were negatively associated with model performance.

CONCLUSIONS

Together, improving sampling economic equality and hence the quality of machine learning models may be a crucial facet to plausibly translating neuroimaging-based diagnostic classifiers into clinical practice.

Artificial intelligence ecosystem for computational psychiatry: Ideas to practice

Computational psychiatry is an emerging field that not only explores the biological basis of mental illness but also considers the diagnoses and identifies the underlying mechanisms. One of the key strengths of computational psychiatry is that it may identify patterns in large datasets that are not easily identifiable. This may help researchers develop more effective treatments and interventions for mental health problems. This paper is a narrative review that reviews the literature and produces an artificial intelligence ecosystem for computational psychiatry. The artificial intelligence ecosystem for computational psychiatry includes data acquisition, preparation, modeling, application, and evaluation. This approach allows researchers to integrate data from a variety of sources, such as brain imaging, genetics, and behavioral experiments, to obtain a more complete understanding of mental health conditions. Through the process of data preprocessing, training, and testing, the data that are required for model building can be prepared. By using machine learning, neural networks, artificial intelligence, and other methods, researchers have been able to develop diagnostic tools that can accurately identify mental health conditions based on a patient’s symptoms and other factors. Despite the continuous development and breakthrough of computational psychiatry, it has not yet influenced routine clinical practice and still faces many challenges, such as data availability and quality, biological risks, equity, and data protection. As we move progress in this field, it is vital to ensure that computational psychiatry remains accessible and inclusive so that all researchers may contribute to this significant and exciting field.

Prediction of model generalizability for unseen data: Methodology and case study in brain metastases detection in T1-Weighted contrast-enhanced 3D MRI

BACKGROUND AND PURPOSE

A medical AI system's generalizability describes the continuity of its performance acquired from varying geographic, historical, and methodologic settings. Previous literature on this topic has mostly focused on "how" to achieve high generalizability (e.g., via larger datasets, transfer learning, data augmentation, model regularization schemes), with limited success. Instead, we aim to understand "when" the generalizability is achieved: Our study presents a medical AI system that could estimate its generalizability status for unseen data on-the-fly.

MATERIALS AND METHODS

We introduce a latent space mapping (LSM) approach utilizing Fréchet distance loss to force the underlying training data distribution into a multivariate normal distribution. During the deployment, a given test data's LSM distribution is processed to detect its deviation from the forced distribution; hence, the AI system could predict its generalizability status for any previously unseen data set. If low model generalizability is detected, then the user is informed by a warning message integrated into a sample deployment workflow. While the approach is applicable for most classification deep neural networks (DNNs), we demonstrate its application to a brain metastases (BM) detector for T1-weighted contrast-enhanced (T1c) 3D MRI. The BM detection model was trained using 175 T1c studies acquired internally (from the authors' institution) and tested using (1) 42 internally acquired exams and (2) 72 externally acquired exams from the publicly distributed Brain Mets dataset provided by the Stanford University School of Medicine. Generalizability scores, false positive (FP) rates, and sensitivities of the BM detector were computed for the test datasets.

RESULTS AND CONCLUSION

The model predicted its generalizability to be low for 31% of the testing data (i.e., two of the internally and 33 of the externally acquired exams), where it produced (1) ∼13.5 false positives (FPs) at 76.1% BM detection sensitivity for the low and (2) ∼10.5 FPs at 89.2% BM detection sensitivity for the high generalizability groups respectively. These results suggest that the proposed formulation enables a model to predict its generalizability for unseen data.

Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis: A Systematic Review

IMPORTANCE

Neuroimaging-based artificial intelligence (AI) diagnostic models have proliferated in psychiatry. However, their clinical applicability and reporting quality (ie, feasibility) for clinical practice have not been systematically evaluated.

OBJECTIVE

To systematically assess the risk of bias (ROB) and reporting quality of neuroimaging-based AI models for psychiatric diagnosis.

EVIDENCE REVIEW

PubMed was searched for peer-reviewed, full-length articles published between January 1, 1990, and March 16, 2022. Studies aimed at developing or validating neuroimaging-based AI models for clinical diagnosis of psychiatric disorders were included. Reference lists were further searched for suitable original studies. Data extraction followed the CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. A closed-loop cross-sequential design was used for quality control. The PROBAST (Prediction Model Risk of Bias Assessment Tool) and modified CLEAR (Checklist for Evaluation of Image-Based Artificial Intelligence Reports) benchmarks were used to systematically evaluate ROB and reporting quality.

FINDINGS

A total of 517 studies presenting 555 AI models were included and evaluated. Of these models, 461 (83.1%; 95% CI, 80.0%-86.2%) were rated as having a high overall ROB based on the PROBAST. The ROB was particular high in the analysis domain, including inadequate sample size (398 of 555 models [71.7%; 95% CI, 68.0%-75.6%]), poor model performance examination (with 100% of models lacking calibration examination), and lack of handling data complexity (550 of 555 models [99.1%; 95% CI, 98.3%-99.9%]). None of the AI models was perceived to be applicable to clinical practices. Overall reporting completeness (ie, number of reported items/number of total items) for the AI models was 61.2% (95% CI, 60.6%-61.8%), and the completeness was poorest for the technical assessment domain with 39.9% (95% CI, 38.8%-41.1%).

CONCLUSIONS AND RELEVANCE

This systematic review found that the clinical applicability and feasibility of neuroimaging-based AI models for psychiatric diagnosis were challenged by a high ROB and poor reporting quality. Particularly in the analysis domain, ROB in AI diagnostic models should be addressed before clinical application.

Morphometric Integrated Classification Index: A Multisite Model-Based, Interpretable, Shareable and Evolvable Biomarker for Schizophrenia

BACKGROUND AND HYPOTHESIS

Multisite massive schizophrenia neuroimaging data sharing is becoming critical in understanding the pathophysiological mechanism and making an objective diagnosis of schizophrenia; it remains challenging to obtain a generalizable and interpretable, shareable, and evolvable neuroimaging biomarker for schizophrenia diagnosis.

STUDY DESIGN

A Morphometric Integrated Classification Index (MICI) was proposed as a potential biomarker for schizophrenia diagnosis based on structural magnetic resonance imaging data of 1270 subjects from 10 sites (588 schizophrenia patients and 682 normal controls). An optimal XGBoost classifier plus sample-weighted SHapley Additive explanation algorithms were used to construct the MICI measure.

STUDY RESULTS

The MICI measure achieved comparable performance with the sample-weighted ensembling model and merged model based on raw data (Delong test, P > 0.82) while outperformed the single-site models (Delong test, P < 0.05) in either the independent-sample testing datasets from the 9 sites or the independent-site dataset (generalizable). Besides, when new sites were embedded in, the performance of this measure was gradually increasing (evolvable). Finally, MICI was strongly associated with the severity of schizophrenia brain structural abnormality, with the patients' positive and negative symptoms, and with the brain expression profiles of schizophrenia risk genes (interpretable).

CONCLUSIONS

In summary, the proposed MICI biomarker may provide a simple and explainable way to support clinicians for objectively diagnosing schizophrenia. Finally, we developed an online model share platform to promote biomarker generalization and provide free individual prediction services (http://micc.tmu.edu.cn/mici/index.html).

Deep-Learning Radiomics for Discrimination Conversion of Alzheimer's Disease in Patients With Mild Cognitive Impairment: A Study Based on 18F-FDG PET Imaging

Objectives: Alzheimer's disease (AD) is the most prevalent neurodegenerative disorder and the most common form of dementia in the older people. Some types of mild cognitive impairment (MCI) are the clinical precursors of AD, while other MCI forms tend to remain stable over time and do not progress to AD. To discriminate MCI patients at risk of AD from stable MCI, we propose a novel deep-learning radiomics (DLR) model based on 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) images and combine DLR features with clinical parameters (DLR+C) to improve diagnostic performance. Methods: 18F-fluorodeoxyglucose positron emission tomography (PET) data from the Alzheimer's disease Neuroimaging Initiative database (ADNI) were collected, including 168 patients with MCI who converted to AD within 3 years and 187 patients with MCI without conversion within 3 years. These subjects were randomly partitioned into 90 % for the training/validation group and 10 % for the independent test group. The proposed DLR approach consists of three steps: base DL model pre-training, network features extraction, and integration of DLR+C, where a convolution network serves as a feature encoder, and a support vector machine (SVM) operated as the classifier. In comparative experiments, we compared our DLR+C method with four other methods: the standard uptake value ratio (SUVR) method, Radiomics-ROI method, Clinical method, and SUVR + Clinical method. To guarantee the robustness, 10-fold cross-validation was processed 100 times. Results: Under the DLR model, our proposed DLR+C was advantageous and yielded the best classification performance in the diagnosis of conversion with the accuracy, sensitivity, and specificity of 90.62 ± 1.16, 87.50 ± 0.00, and 93.39 ± 2.19%, respectively. In contrast, the respective accuracy of the other four methods reached 68.38 ± 1.27, 73.31 ± 6.93, 81.09 ± 1.97, and 85.35 ± 0.72 %. These results suggested the DLR approach could be used successfully in the prediction of conversion to AD, and that our proposed DLR-combined clinical information was effective. Conclusions: This study showed DLR+C could provide a novel and valuable method for the computer-assisted diagnosis of conversion to AD from MCI. This DLR+C method provided a quantitative biomarker which could predict conversion to AD in MCI patients.

Constrained generative adversarial network ensembles for sharable synthetic medical images

Purpose: Sharing medical images between institutions, or even inside the same institution, is restricted by various laws and regulations; research projects requiring large datasets may suffer as a result. These limitations might be addressed by an abundant supply of synthetic data that (1) are representative (i.e., the synthetic data could produce comparable research results as the original data) and (2) do not closely resemble the original images (i.e., patient privacy is protected). We introduce a framework that generates data with these requirements leveraging generative adversarial network (GAN) ensembles in a controlled fashion. Approach: To this end, an adaptive ensemble scaling strategy with the objective of representativeness is defined. A sampled Fréchet distance-based constraint was then created to eliminate poorly converged candidates. Finally, a mutual information-based validation metric was embedded into the framework to confirm there are visual differences between the original and the generated synthetic images. Results: The applicability of the solution is demonstrated with a case study for generating three-dimensional brain metastasis (BM) from T1-weighted contrast-enhanced MRI studies. A previously published BM detection system was reported to produce 9.12 false-positives at 90% detection sensitivity based on the original data. By using the synthetic data generated with the proposed framework, the system produced 9.53 false-positives at the same sensitivity level. Conclusions: Achieving comparable algorithm performance relying solely on synthetic data unveils a significant potential to eliminate/reduce patient privacy concerns when sharing data in medical imaging.

Distributed learning: a reliable privacy-preserving strategy to change multicenter collaborations using AI

Purpose

The present scoping review aims to assess the non-inferiority of distributed learning over centrally and locally trained machine learning (ML) models in medical applications.

Methods

We performed a literature search using the term “distributed learning” OR “federated learning” in the PubMed/MEDLINE and EMBASE databases. No start date limit was used, and the search was extended until July 21, 2020. We excluded articles outside the field of interest; guidelines or expert opinion, review articles and meta-analyses, editorials, letters or commentaries, and conference abstracts; articles not in the English language; and studies not using medical data. Selected studies were classified and analysed according to their aim(s).

Results

We included 26 papers aimed at predicting one or more outcomes: namely risk, diagnosis, prognosis, and treatment side effect/adverse drug reaction. Distributed learning was compared to centralized or localized training in 21/26 and 14/26 selected papers, respectively. Regardless of the aim, the type of input, the method, and the classifier, distributed learning performed close to centralized training, but two experiments focused on diagnosis. In all but 2 cases, distributed learning outperformed locally trained models.

Conclusion

Distributed learning resulted in a reliable strategy for model development; indeed, it performed equally to models trained on centralized datasets. Sensitive data can get preserved since they are not shared for model development. Distributed learning constitutes a promising solution for ML-based research and practice since large, diverse datasets are crucial for success.

NEURO-LEARN: a Solution for Collaborative Pattern Analysis of Neuroimaging Data

The development of neuroimaging instrumentation has boosted neuroscience researches. Consequently, both the fineness and the cost of data acquisition have profoundly increased, leading to the main bottleneck of this field: limited sample size and high dimensionality of neuroimaging data. Therefore, the emphasis of ideas of data pooling and research collaboration has increased over the past decade. Collaborative analysis techniques emerge as the idea developed. In this paper, we present NEURO-LEARN, a solution for collaborative pattern analysis of neuroimaging data. Its collaboration scheme consists of four parts: projects, data, analysis, and reports. While data preparation workflows defined in projects reduce the high dimensionality of neuroimaging data by collaborative computation, pooling of derived data and sharing of pattern analysis workflows along with generated reports on the Web enlarge the sample size and ensure the reliability and reproducibility of pattern analysis. Incorporating this scheme, NEURO-LEARN provides an easy-to-use Web application that allows users from different sites to share projects and processed data, perform pattern analysis, and obtain result reports. We anticipate that this solution will help neuroscientists to enlarge sample size, conquer the curse of dimensionality and conduct reproducible studies on neuroimaging data with efficiency and validity.

Magnetic resonance imaging for individual prediction of treatment response in major depressive disorder: a systematic review and meta-analysis

No tools are currently available to predict whether a patient suffering from major depressive disorder (MDD) will respond to a certain treatment. Machine learning analysis of magnetic resonance imaging (MRI) data has shown potential in predicting response for individual patients, which may enable personalized treatment decisions and increase treatment efficacy. Here, we evaluated the accuracy of MRI-guided response prediction in MDD. We conducted a systematic review and meta-analysis of all studies using MRI to predict single-subject response to antidepressant treatment in patients with MDD. Classification performance was calculated using a bivariate model and expressed as area under the curve, sensitivity, and specificity. In addition, we analyzed differences in classification performance between different interventions and MRI modalities. Meta-analysis of 22 samples including 957 patients showed an overall area under the bivariate summary receiver operating curve of 0.84 (95% CI 0.81-0.87), sensitivity of 77% (95% CI 71-82), and specificity of 79% (95% CI 73-84). Although classification performance was higher for electroconvulsive therapy outcome prediction (n = 285, 80% sensitivity, 83% specificity) than medication outcome prediction (n = 283, 75% sensitivity, 72% specificity), there was no significant difference in classification performance between treatments or MRI modalities. Prediction of treatment response using machine learning analysis of MRI data is promising but should not yet be implemented into clinical practice. Future studies with more generalizable samples and external validation are needed to establish the potential of MRI to realize individualized patient care in MDD.

Functional Magnetic Resonance Imaging Connectivity Accurately Distinguishes Cases With Psychotic Disorders From Healthy Controls, Based on Cortical Features Associated With Brain Network Development

BACKGROUND

Machine learning (ML) can distinguish cases with psychotic disorder from healthy controls based on magnetic resonance imaging (MRI) data, but it is not yet clear which MRI metrics are the most informative for case-control ML, or how ML algorithms relate to the underlying biology.

METHODS

We analyzed multimodal MRI data from 2 independent case-control studies of psychotic disorders (cases, n = 65, 28; controls, n = 59, 80) and compared ML accuracy across 5 selected MRI metrics from 3 modalities. Cortical thickness, mean diffusivity, and fractional anisotropy were estimated at each of 308 cortical regions, as well as functional and structural connectivity between each pair of regions. Functional connectivity data were also used to classify nonpsychotic siblings of cases (n = 64) and to distinguish cases from controls in a third independent study (cases, n = 67; controls, n = 81).

RESULTS

In both principal studies, the most informative metric was functional MRI connectivity: The areas under the receiver operating characteristic curve were 88% and 76%, respectively. The cortical map of diagnostic connectivity features (ML weights) was replicable between studies (r = .27, p < .001); correlated with replicable case-control differences in functional MRI degree centrality and with a prior cortical map of adolescent development of functional connectivity; predicted intermediate probabilities of psychosis in siblings; and was replicated in the third case-control study.

CONCLUSIONS

ML most accurately distinguished cases from controls by a replicable pattern of functional MRI connectivity features, highlighting abnormal hubness of cortical nodes in an anatomical pattern consistent with the concept of psychosis as a disorder of network development.

Systematic Review of Privacy-Preserving Distributed Machine Learning From Federated Databases in Health Care

Big data for health care is one of the potential solutions to deal with the numerous challenges of health care, such as rising cost, aging population, precision medicine, universal health coverage, and the increase of noncommunicable diseases. However, data centralization for big data raises privacy and regulatory concerns.Covered topics include (1) an introduction to privacy of patient data and distributed learning as a potential solution to preserving these data, a description of the legal context for patient data research, and a definition of machine/deep learning concepts; (2) a presentation of the adopted review protocol; (3) a presentation of the search results; and (4) a discussion of the findings, limitations of the review, and future perspectives.Distributed learning from federated databases makes data centralization unnecessary. Distributed algorithms iteratively analyze separate databases, essentially sharing research questions and answers between databases instead of sharing the data. In other words, one can learn from separate and isolated datasets without patient data ever leaving the individual clinical institutes.Distributed learning promises great potential to facilitate big data for medical application, in particular for international consortiums. Our purpose is to review the major implementations of distributed learning in health care.

Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence

Despite the high level of interest in the use of machine learning (ML) and neuroimaging to detect psychosis at the individual level, the reliability of the findings is unclear due to potential methodological issues that may have inflated the existing literature. This study aimed to elucidate the extent to which the application of ML to neuroanatomical data allows detection of first episode psychosis (FEP), while putting in place methodological precautions to avoid overoptimistic results. We tested both traditional ML and an emerging approach known as deep learning (DL) using 3 feature sets of interest: (1) surface-based regional volumes and cortical thickness, (2) voxel-based gray matter volume (GMV) and (3) voxel-based cortical thickness (VBCT). To assess the reliability of the findings, we repeated all analyses in 5 independent datasets, totaling 956 participants (514 FEP and 444 within-site matched controls). The performance was assessed via nested cross-validation (CV) and cross-site CV. Accuracies ranged from 50% to 70% for surfaced-based features; from 50% to 63% for GMV; and from 51% to 68% for VBCT. The best accuracies (70%) were achieved when DL was applied to surface-based features; however, these models generalized poorly to other sites. Findings from this study suggest that, when methodological precautions are adopted to avoid overoptimistic results, detection of individuals in the early stages of psychosis is more challenging than originally thought. In light of this, we argue that the current evidence for the diagnostic value of ML and structural neuroimaging should be reconsidered toward a more cautious interpretation.

Using structural MRI to identify bipolar disorders - 13 site machine learning study in 3020 individuals from the ENIGMA Bipolar Disorders Working Group

Bipolar disorders (BDs) are among the leading causes of morbidity and disability. Objective biological markers, such as those based on brain imaging, could aid in clinical management of BD. Machine learning (ML) brings neuroimaging analyses to individual subject level and may potentially allow for their diagnostic use. However, fair and optimal application of ML requires large, multi-site datasets. We applied ML (support vector machines) to MRI data (regional cortical thickness, surface area, subcortical volumes) from 853 BD and 2167 control participants from 13 cohorts in the ENIGMA consortium. We attempted to differentiate BD from control participants, investigated different data handling strategies and studied the neuroimaging/clinical features most important for classification. Individual site accuracies ranged from 45.23% to 81.07%. Aggregate subject-level analyses yielded the highest accuracy (65.23%, 95% CI = 63.47-67.00, ROC-AUC = 71.49%, 95% CI = 69.39-73.59), followed by leave-one-site-out cross-validation (accuracy = 58.67%, 95% CI = 56.70-60.63). Meta-analysis of individual site accuracies did not provide above chance results. There was substantial agreement between the regions that contributed to identification of BD participants in the best performing site and in the aggregate dataset (Cohen's Kappa = 0.83, 95% CI = 0.829-0.831). Treatment with anticonvulsants and age were associated with greater odds of correct classification. Although short of the 80% clinically relevant accuracy threshold, the results are promising and provide a fair and realistic estimate of classification performance, which can be achieved in a large, ecologically valid, multi-site sample of BD participants based on regional neurostructural measures. Furthermore, the significant classification in different samples was based on plausible and similar neuroanatomical features. Future multi-site studies should move towards sharing of raw/voxelwise neuroimaging data.

Staging and quantification of florbetaben PET images using machine learning: impact of predicted regional cortical tracer uptake and amyloid stage on clinical outcomes

Purpose

We developed a machine learning–based classifier for in vivo amyloid positron emission tomography (PET) staging, quantified cortical uptake of the PET tracer by using a machine learning method, and investigated the impact of these amyloid PET parameters on clinical and structural outcomes.

Methods

A total of 337 ¹⁸F-florbetaben PET scans obtained at Samsung Medical Center were assessed. We defined a feature vector representing the change in PET tracer uptake from grey to white matter. Using support vector machine (SVM) regression and SVM classification, we quantified the cortical uptake as predicted regional cortical tracer uptake (pRCTU) and categorised the scans as positive and negative. Positive scans were further classified into two stages according to the striatal uptake. We compared outcome parameters among stages and further assessed the association between the pRCTU and outcome variables. Finally, we performed path analysis to determine mediation effects between PET variables.

Results

The classification accuracy was 97.3% for cortical amyloid positivity and 91.1% for striatal positivity. The left frontal and precuneus/posterior cingulate regions, as well as the anterior portion of the striatum, were important in determination of stages. The clinical scores and magnetic resonance imaging parameters showed negative associations with PET stage. However, except for the hippocampal volume, most outcomes were associated with the stage through the complete mediation effect of pRCTU.

Conclusion

Using a machine learning algorithm, we achieved high accuracy for in vivo amyloid PET staging. The in vivo amyloid stage was associated with cognitive function and cerebral atrophy mostly through the mediation effect of cortical amyloid.

Electronic supplementary material

The online version of this article (10.1007/s00259-019-04663-3) contains supplementary material, which is available to authorized users.

Improving individual predictions: Machine learning approaches for detecting and attacking heterogeneity in schizophrenia (and other psychiatric diseases)

Psychiatric diseases are very heterogeneous both in clinical manifestation and etiology. With the recent rise of using machine learning techniques to attempt to diagnose and prognose these disorders, the issue of heterogeneity becomes increasingly important. With the growing interest in personalized medicine, it becomes even more important to not only classify someone as a patient with a certain disorder, its treatment needs a more precise definition of the underlying neurobiology, since different biological origins of the same disease may require (very) different treatments. We review the possible contributions that machine learning techniques could make to explore the heterogeneous nature of psychiatric disorders with a focus on schizophrenia. First we will review how heterogeneity shows up and how machine learning, or multivariate pattern recognition methods in general, can be used to discover it. Secondly, we will discuss the possible uses of these techniques to attack heterogeneity, leading to improved predictions and understanding of the neurobiological background of the disorder.

Distributed deep learning networks among institutions for medical imaging

Objective

Deep learning has become a promising approach for automated support for clinical diagnosis. When medical data samples are limited, collaboration among multiple institutions is necessary to achieve high algorithm performance. However, sharing patient data often has limitations due to technical, legal, or ethical concerns. In this study, we propose methods of distributing deep learning models as an attractive alternative to sharing patient data.

Methods

We simulate the distribution of deep learning models across 4 institutions using various training heuristics and compare the results with a deep learning model trained on centrally hosted patient data. The training heuristics investigated include ensembling single institution models, single weight transfer, and cyclical weight transfer. We evaluated these approaches for image classification in 3 independent image collections (retinal fundus photos, mammography, and ImageNet).

Results

We find that cyclical weight transfer resulted in a performance that was comparable to that of centrally hosted patient data. We also found that there is an improvement in the performance of cyclical weight transfer heuristic with a high frequency of weight transfer.

Conclusions

We show that distributing deep learning models is an effective alternative to sharing patient data. This finding has implications for any collaborative deep learning study.

Machine learning in mental health: a scoping review of methods and applications

BACKGROUND

This paper aims to synthesise the literature on machine learning (ML) and big data applications for mental health, highlighting current research and applications in practice.

METHODS

We employed a scoping review methodology to rapidly map the field of ML in mental health. Eight health and information technology research databases were searched for papers covering this domain. Articles were assessed by two reviewers, and data were extracted on the article's mental health application, ML technique, data type, and study results. Articles were then synthesised via narrative review.

RESULTS

Three hundred papers focusing on the application of ML to mental health were identified. Four main application domains emerged in the literature, including: (i) detection and diagnosis; (ii) prognosis, treatment and support; (iii) public health, and; (iv) research and clinical administration. The most common mental health conditions addressed included depression, schizophrenia, and Alzheimer's disease. ML techniques used included support vector machines, decision trees, neural networks, latent Dirichlet allocation, and clustering.

CONCLUSIONS

Overall, the application of ML to mental health has demonstrated a range of benefits across the areas of diagnosis, treatment and support, research, and clinical administration. With the majority of studies identified focusing on the detection and diagnosis of mental health conditions, it is evident that there is significant room for the application of ML to other areas of psychology and mental health. The challenges of using ML techniques are discussed, as well as opportunities to improve and advance the field.

Brain Morphometry Methods for Feature Extraction in Random Subspace Ensemble Neural Network Classification of First-Episode Schizophrenia

Machine learning (ML) is a growing field that provides tools for automatic pattern recognition. The neuroimaging community currently tries to take advantage of ML in order to develop an auxiliary diagnostic tool for schizophrenia diagnostics. In this letter, we present a classification framework based on features extracted from magnetic resonance imaging (MRI) data using two automatic whole-brain morphometry methods: voxel-based (VBM) and deformation-based morphometry (DBM). The framework employs a random subspace ensemble-based artificial neural network classifier—in particular, a multilayer perceptron (MLP). The framework was tested on data from first-episode schizophrenia patients and healthy controls. The experiments differed in terms of feature extraction methods, using VBM, DBM, and a combination of both morphometry methods. Thus, features of different types were available for model adaptation. As we expected, the combination of features increased the MLP classification accuracy up to 73.12%—an improvement of 5% versus MLP-based only on VBM or DBM features. To further verify the findings, other comparisons using support vector machines in place of MLPs were made within the framework. However, it cannot be concluded that any classifier was better than another.

Translational machine learning for psychiatric neuroimaging

Despite its initial promise, neuroimaging has not been widely translated into clinical psychiatry to assist in the prediction of diagnoses, prognoses, and optimal therapeutic strategies. Machine learning approaches may enhance the translational potential of neuroimaging because they specifically focus on overcoming biases by optimizing the generalizability of pipelines that measure complex brain patterns to predict targets at a single-subject level. This article introduces some fundamentals of a translational machine learning approach before selectively reviewing literature to-date. Promising initial results are then balanced by the description of limitations that should be considered in order to interpret existing research and maximize the possibility of future translation. Future directions are then presented in order to inspire further research and progress the field towards clinical translation.

Predicting major mental illness: ethical and practical considerations

SummaryAn increasing body of genetic and imaging research shows that it is becoming possible to forecast the onset of major psychiatric disorders such as depression and schizophrenia before people become ill with ever improving accuracy. Practical issues such as the optimal combination of clinical and biological variables are being addressed, but the application of predictive algorithms to individuals or in routine clinical settings have yet to be tested. The development of predictive methods in mental health comes with substantial ethical questions, including whether people wish to know their level of risk, as well as individual and societal attitudes to the potential adverse effects of data sharing, early diagnosis and treatment, which so far have been largely ignored. Preliminary data suggests that at least some people think predictive research is valuable and would take part in such studies, and some would welcome knowing the results. Future initiatives should systematically assess opinions and attitudes in conjunction with scientific and technical advances.Declaration of interestIn the past 3 years, S.M.L. has received personal fees from Otsuaka, Sunovion and Janssen, and research grant support from Janssen and Lundbeck. A.M.M. has received research support from the Sackler Trust, Eli Lilly and Janssen. S.M.L. is part of the PSYSCAN consortium.

Brain morphologic changes in early stages of psychosis: Implications for clinical application and early intervention

To date, a large number of magnetic resonance imaging (MRI) studies have been conducted in schizophrenia, which generally demonstrate gray matter reduction, predominantly in the frontal and temporo-limbic regions, as well as gross brain abnormalities (e.g., a deviated sulcogyral pattern). Although the causes as well as timing and course of these findings remain elusive, these morphologic changes (especially gross brain abnormalities and medial temporal lobe atrophy) are likely present at illness onset, possibly reflecting early neurodevelopmental abnormalities. In addition, longitudinal MRI studies suggest that patients with schizophrenia and related psychoses also have progressive gray matter reduction during the transition period from prodrome to overt psychosis, as well as initial periods after psychosis onset, while such changes may become almost stable in the chronic stage. These active brain changes during the early phases seem to be relevant to the development of clinical symptoms in a region-specific manner (e.g., superior temporal gyrus atrophy and positive psychotic symptoms), but may be at least partly ameliorated by antipsychotic medication. Recently, increasing evidence from MRI findings in individuals at risk for developing psychosis has suggested that those who subsequently develop psychosis have baseline brain changes, which could be at least partly predictive of later transition into psychosis. In this article, we selectively review previous MRI findings during the course of psychosis and also refer to the possible clinical applicability of these neuroimaging research findings, especially in the diagnosis of schizophrenia and early intervention for psychosis.

Detecting Neuroimaging Biomarkers for Psychiatric Disorders: Sample Size Matters

In a recent review, it was suggested that much larger cohorts are needed to prove the diagnostic value of neuroimaging biomarkers in psychiatry. While within a sample, an increase of diagnostic accuracy of schizophrenia (SZ) with number of subjects (N) has been shown, the relationship between N and accuracy is completely different between studies. Using data from a recent meta-analysis of machine learning (ML) in imaging SZ, we found that while low-N studies can reach 90% and higher accuracy, above N/2 = 50 the maximum accuracy achieved steadily drops to below 70% for N/2 > 150. We investigate the role N plays in the wide variability in accuracy results in SZ studies (63-97%). We hypothesize that the underlying cause of the decrease in accuracy with increasing N is sample heterogeneity. While smaller studies more easily include a homogeneous group of subjects (strict inclusion criteria are easily met; subjects live close to study site), larger studies inevitably need to relax the criteria/recruit from large geographic areas. A SZ prediction model based on a heterogeneous group of patients with presumably a heterogeneous pattern of structural or functional brain changes will not be able to capture the whole variety of changes, thus being limited to patterns shared by most patients. In addition to heterogeneity (sample size), we investigate other factors influencing accuracy and introduce a ML effect size. We derive a simple model of how the different factors, such as sample heterogeneity and study setup determine this ML effect size, and explain the variation in prediction accuracies found from the literature, both in cross-validation and independent sample testing. From this, we argue that smaller-N studies may reach high prediction accuracy at the cost of lower generalizability to other samples. Higher-N studies, on the other hand, will have more generalization power, but at the cost of lower accuracy. In conclusion, when comparing results from different ML studies, the sample sizes should be taken into account. To assess the generalizability of the models, validation (by direct application) of the prediction models should be tested in independent samples. The prediction of more complex measures such as outcome, which are expected to have an underlying pattern of more subtle brain abnormalities (lower effect size), will require large samples.