1
|
Fisch L, Zumdick S, Barkhau C, Emden D, Ernsting J, Leenings R, Sarink K, Winter NR, Risse B, Dannlowski U, Hahn T. deepbet: Fast brain extraction of T1-weighted MRI using Convolutional Neural Networks. Comput Biol Med 2024; 179:108845. [PMID: 39002314 DOI: 10.1016/j.compbiomed.2024.108845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Revised: 07/01/2024] [Accepted: 07/02/2024] [Indexed: 07/15/2024]
Abstract
BACKGROUND Brain extraction in magnetic resonance imaging (MRI) data is an important segmentation step in many neuroimaging preprocessing pipelines. Image segmentation is one of the research fields in which deep learning had the biggest impact in recent years. Consequently, traditional brain extraction methods are now being replaced by deep learning-based methods. METHOD Here, we used a unique dataset compilation comprising 7837 T1-weighted (T1w) MR images from 191 different OpenNeuro datasets in combination with advanced deep learning methods to build a fast, high-precision brain extraction tool called deepbet. RESULTS deepbet sets a novel state-of-the-art performance during cross-dataset validation with a median Dice score (DSC) of 99.0 on unseen datasets, outperforming the current best performing deep learning (DSC=97.9) and classic (DSC=96.5) methods. While current methods are more sensitive to outliers, deepbet achieves a Dice score of >97.4 across all 7837 images from 191 different datasets. This robustness was additionally tested in 5 external datasets, which included challenging clinical MR images. During visual exploration of each method's output which resulted in the lowest Dice score, major errors could be found for all of the tested tools except deepbet. Finally, deepbet uses a compute efficient variant of the UNet architecture, which accelerates brain extraction by a factor of ≈10 compared to current methods, enabling the processing of one image in ≈2 s on low level hardware. CONCLUSIONS In conclusion, deepbet demonstrates superior performance and reliability in brain extraction across a wide range of T1w MR images of adults, outperforming existing top tools. Its high minimal Dice score and minimal objective errors, even in challenging conditions, validate deepbet as a highly dependable tool for accurate brain extraction. deepbet can be conveniently installed via "pip install deepbet" and is publicly accessible at https://github.com/wwu-mmll/deepbet.
Collapse
Affiliation(s)
- Lukas Fisch
- University of Münster, Institute for Translational Psychiatry, Münster, Germany.
| | - Stefan Zumdick
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Carlotta Barkhau
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Daniel Emden
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Jan Ernsting
- University of Münster, Institute for Translational Psychiatry, Münster, Germany; Department of Mathematics and Computer Science, University of Münster, Münster, Germany
| | - Ramona Leenings
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Kelvin Sarink
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Nils R Winter
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Benjamin Risse
- Department of Mathematics and Computer Science, University of Münster, Münster, Germany
| | - Udo Dannlowski
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| | - Tim Hahn
- University of Münster, Institute for Translational Psychiatry, Münster, Germany
| |
Collapse
|
2
|
Manthe M, Duffner S, Lartizien C. Federated brain tumor segmentation: An extensive benchmark. Med Image Anal 2024; 97:103270. [PMID: 39059241 DOI: 10.1016/j.media.2024.103270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 06/14/2024] [Accepted: 07/08/2024] [Indexed: 07/28/2024]
Abstract
Recently, federated learning has raised increasing interest in the medical image analysis field due to its ability to aggregate multi-center data with privacy-preserving properties. A large amount of federated training schemes have been published, which we categorize into global (one final model), personalized (one model per institution) or hybrid (one model per cluster of institutions) methods. However, their applicability on the recently published Federated Brain Tumor Segmentation 2022 dataset has not been explored yet. We propose an extensive benchmark of federated learning algorithms from all three classes on this task. While standard FedAvg already performs very well, we show that some methods from each category can bring a slight performance improvement and potentially limit the final model(s) bias toward the predominant data distribution of the federation. Moreover, we provide a deeper understanding of the behavior of federated learning on this task through alternative ways of distributing the pooled dataset among institutions, namely an Independent and Identical Distributed (IID) setup, and a limited data setup. Our code is available at (https://github.com/MatthisManthe/Benchmark_FeTS2022).
Collapse
Affiliation(s)
- Matthis Manthe
- INSA Lyon, Universite Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621 Lyon, France; INSA Lyon, CNRS, Universite Claude Bernard Lyon 1, Centrale Lyon, Université Lumière Lyon 2, LIRIS, UMR5205, F-69621 Villeurbanne, France.
| | - Stefan Duffner
- INSA Lyon, CNRS, Universite Claude Bernard Lyon 1, Centrale Lyon, Université Lumière Lyon 2, LIRIS, UMR5205, F-69621 Villeurbanne, France
| | - Carole Lartizien
- INSA Lyon, Universite Claude Bernard Lyon 1, CNRS, Inserm, CREATIS UMR 5220, U1294, F-69621 Lyon, France
| |
Collapse
|
3
|
Soldatelli MD, Namdar K, Tabori U, Hawkins C, Yeom K, Khalvati F, Ertl-Wagner BB, Wagner MW. Identification of Multiclass Pediatric Low-Grade Neuroepithelial Tumor Molecular Subtype with ADC MR Imaging and Machine Learning. AJNR Am J Neuroradiol 2024; 45:753-760. [PMID: 38604736 PMCID: PMC11288584 DOI: 10.3174/ajnr.a8199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Accepted: 01/16/2024] [Indexed: 04/13/2024]
Abstract
BACKGROUND AND PURPOSE Molecular biomarker identification increasingly influences the treatment planning of pediatric low-grade neuroepithelial tumors (PLGNTs). We aimed to develop and validate a radiomics-based ADC signature predictive of the molecular status of PLGNTs. MATERIALS AND METHODS In this retrospective bi-institutional study, we searched the PACS for baseline brain MRIs from children with PLGNTs. Semiautomated tumor segmentation on ADC maps was performed using the semiautomated level tracing effect tool with 3D Slicer. Clinical variables, including age, sex, and tumor location, were collected from chart review. The molecular status of tumors was derived from biopsy. Multiclass random forests were used to predict the molecular status and fine-tuned using a grid search on the validation sets. Models were evaluated using independent and unseen test sets based on the combined data, and the area under the receiver operating characteristic curve (AUC) was calculated for the prediction of 3 classes: KIAA1549-BRAF fusion, BRAF V600E mutation, and non-BRAF cohorts. Experiments were repeated 100 times using different random data splits and model initializations to ensure reproducible results. RESULTS Two hundred ninety-nine children from the first institution and 23 children from the second institution were included (53.6% male; mean, age 8.01 years; 51.8% supratentorial; 52.2% with KIAA1549-BRAF fusion). For the 3-class prediction using radiomics features only, the average test AUC was 0.74 (95% CI, 0.73-0.75), and using clinical features only, the average test AUC was 0.67 (95% CI, 0.66-0.68). The combination of both radiomics and clinical features improved the AUC to 0.77 (95% CI, 0.75-0.77). The diagnostic performance of the per-class test AUC was higher in identifying KIAA1549-BRAF fusion tumors among the other subgroups (AUC = 0.81 for the combined radiomics and clinical features versus 0.75 and 0.74 for BRAF V600E mutation and non-BRAF, respectively). CONCLUSIONS ADC values of tumor segmentations have differentiative signals that can be used for training machine learning classifiers for molecular biomarker identification of PLGNTs. ADC-based pretherapeutic differentiation of the BRAF status of PLGNTs has the potential to avoid invasive tumor biopsy and enable earlier initiation of targeted therapy.
Collapse
Affiliation(s)
- Matheus D Soldatelli
- From the Department Diagnostic Imaging (M.D.S., B.B.E.-W., M.W.W.), Division of Neuroradiology, The Hospital for Sick Children, Toronto, Ontario, Canada
- Department of Medical Imaging (M.D.S., K.N., F.K., B.B.E.-W., M.W.W.), University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science (M.D.S., K.N., U.T., F.K., B.B.E.-W.), University of Toronto, Toronto, Ontario, Canada
| | - Khashayar Namdar
- Department of Medical Imaging (M.D.S., K.N., F.K., B.B.E.-W., M.W.W.), University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science (M.D.S., K.N., U.T., F.K., B.B.E.-W.), University of Toronto, Toronto, Ontario, Canada
- Vector Institute (K.N., F.K.), Toronto, Ontario, Canada
| | - Uri Tabori
- Institute of Medical Science (M.D.S., K.N., U.T., F.K., B.B.E.-W.), University of Toronto, Toronto, Ontario, Canada
- The Arthur and Sonia Labatt Brain Tumour Research Centre (U.T., C.H.), The Hospital for Sick Children, Toronto, Ontario, Canada
- Program in Genetics and Genome Biology (U.T.) The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Cynthia Hawkins
- The Arthur and Sonia Labatt Brain Tumour Research Centre (U.T., C.H.), The Hospital for Sick Children, Toronto, Ontario, Canada
- Department of Laboratory Medicine and Pathobiology (C.H.), University of Toronto, Toronto, Ontario, Canada
- Division of Pathology (C.H.), The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Kristen Yeom
- Department of Radiology (K.Y.), Lucile Packard Children's Hospital, Stanford University School of Medicine, Stanford, California
| | - Farzad Khalvati
- Department of Medical Imaging (M.D.S., K.N., F.K., B.B.E.-W., M.W.W.), University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science (M.D.S., K.N., U.T., F.K., B.B.E.-W.), University of Toronto, Toronto, Ontario, Canada
- Vector Institute (K.N., F.K.), Toronto, Ontario, Canada
- Department of Computer Science (F.K.), University of Toronto, Toronto, Ontario, Canada
| | - Birgit B Ertl-Wagner
- From the Department Diagnostic Imaging (M.D.S., B.B.E.-W., M.W.W.), Division of Neuroradiology, The Hospital for Sick Children, Toronto, Ontario, Canada
- Department of Medical Imaging (M.D.S., K.N., F.K., B.B.E.-W., M.W.W.), University of Toronto, Toronto, Ontario, Canada
- Institute of Medical Science (M.D.S., K.N., U.T., F.K., B.B.E.-W.), University of Toronto, Toronto, Ontario, Canada
| | - Matthias W Wagner
- From the Department Diagnostic Imaging (M.D.S., B.B.E.-W., M.W.W.), Division of Neuroradiology, The Hospital for Sick Children, Toronto, Ontario, Canada
- Department of Medical Imaging (M.D.S., K.N., F.K., B.B.E.-W., M.W.W.), University of Toronto, Toronto, Ontario, Canada
- Department of Diagnostic and Interventional Neuroradiology (M.W.W.), University Hospital Augsburg, Augsburg, Germany
| |
Collapse
|
4
|
Liu K, Lin Z, Zheng T, Ba R, Zhang Z, Li H, Zhang H, Tal A, Wu D. Improving Microstructural Estimation in Time-Dependent Diffusion MRI With a Bayesian Method. J Magn Reson Imaging 2024. [PMID: 38769739 DOI: 10.1002/jmri.29434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 04/20/2024] [Accepted: 04/22/2024] [Indexed: 05/22/2024] Open
Abstract
BACKGROUND Accurately fitting diffusion-time-dependent diffusion MRI (td-dMRI) models poses challenges due to complex and nonlinear formulas, signal noise, and limited clinical data acquisition. PURPOSE Introduce a Bayesian methodology to refine microstructural fitting within the IMPULSED (Imaging Microstructural Parameters Using Limited Spectrally Edited Diffusion) model and optimize the prior distribution within the Bayesian framework. STUDY TYPE Retrospective. POPULATION Involving 69 pediatric patients (median age 6 years, interquartile range [IQR] 3-9 years, 61% male) with 41 low-grade and 28 high-grade gliomas, of which 76.8% were identified within the brainstem or cerebellum. FIELD STRENGTH/SEQUENCE 3 T, oscillating gradient spin-echo (OGSE) and pulsed gradient spin-echo (PGSE). ASSESSMENT The Bayesian method's performance in fitting cell diameter (d $$ d $$ ), intracellular volume fraction (f in $$ {f}_{in} $$ ), and extracellular diffusion coefficient (D ex $$ {D}_{ex} $$ ) was compared against the NLLS method, considering simulated and experimental data. The tumor region-of-interest (ROI) were manually delineated on the b0 images. The diagnostic performance in distinguishing high- and low-grade gliomas was assessed, and fitting accuracy was validated against H&E-stained pathology. STATISTICAL TESTS T-test, receiver operating curve (ROC), area under the curve (AUC) and DeLong's test were conducted. Significance considered at P < 0.05. RESULTS Bayesian methodology manifested increased accuracy with robust estimates in simulation (RMSE decreased by 29.6%, 40.9%, 13.6%, and STD decreased by 29.2%, 43.5%, and 24.0%, respectively ford $$ d $$ ,f in $$ {f}_{in} $$ , andD ex $$ {D}_{ex} $$ compared to NLLS), indicating fewer outliers and reduced error. Diagnostic performance for tumor grade was similar in both methods, however, Bayesian method generated smoother microstructural maps (outliers ratio decreased by 45.3% ± 19.4%) and a marginal enhancement in correlation with H&E staining result (r = 0.721 forf in $$ {f}_{in} $$ compared to r = 0.698 using NLLS, P = 0.5764). DATA CONCLUSION The proposed Bayesian method substantially enhances the accuracy and robustness of IMPULSED model estimation, suggesting its potential clinical utility in characterizing cellular microstructure. EVIDENCE LEVEL 3 TECHNICAL EFFICACY: Stage 1.
Collapse
Affiliation(s)
- Kuiyuan Liu
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
| | - Zixuan Lin
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
| | - Tianshu Zheng
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
| | - Ruicheng Ba
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
| | - Zelin Zhang
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
| | - Haotian Li
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
| | - Hongxi Zhang
- Department of Radiology, Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Assaf Tal
- Department of Chemical and Biological Physics, Weizmann Institute of Science, Rehovot, Israel
| | - Dan Wu
- Department of Biomedical Engineering, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
- Department of Radiology, Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China
| |
Collapse
|
5
|
Leonardsen EH, Persson K, Grødem E, Dinsdale N, Schellhorn T, Roe JM, Vidal-Piñeiro D, Sørensen Ø, Kaufmann T, Westman E, Marquand A, Selbæk G, Andreassen OA, Wolfers T, Westlye LT, Wang Y. Constructing personalized characterizations of structural brain aberrations in patients with dementia using explainable artificial intelligence. NPJ Digit Med 2024; 7:110. [PMID: 38698139 PMCID: PMC11066104 DOI: 10.1038/s41746-024-01123-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 04/23/2024] [Indexed: 05/05/2024] Open
Abstract
Deep learning approaches for clinical predictions based on magnetic resonance imaging data have shown great promise as a translational technology for diagnosis and prognosis in neurological disorders, but its clinical impact has been limited. This is partially attributed to the opaqueness of deep learning models, causing insufficient understanding of what underlies their decisions. To overcome this, we trained convolutional neural networks on structural brain scans to differentiate dementia patients from healthy controls, and applied layerwise relevance propagation to procure individual-level explanations of the model predictions. Through extensive validations we demonstrate that deviations recognized by the model corroborate existing knowledge of structural brain aberrations in dementia. By employing the explainable dementia classifier in a longitudinal dataset of patients with mild cognitive impairment, we show that the spatially rich explanations complement the model prediction when forecasting transition to dementia and help characterize the biological manifestation of disease in the individual brain. Overall, our work exemplifies the clinical potential of explainable artificial intelligence in precision medicine.
Collapse
Affiliation(s)
- Esten H Leonardsen
- Department of Psychology, University of Oslo, Oslo, Norway.
- Norwegian Centre for Mental Disorders Research (NORMENT), Oslo University Hospital & Institute of Clinical Medicine, University of Oslo, Oslo, Norway.
| | - Karin Persson
- The Norwegian National Centre for Ageing and Health, Vestfold Hospital Trust, Tønsberg, Norway
- Department of Geriatric Medicine, Oslo University Hospital, Oslo, Norway
| | - Edvard Grødem
- Department of Psychology, University of Oslo, Oslo, Norway
- Computational Radiology & Artificial Intelligence (CRAI) Unit, Division of Radiology and Nuclear Medicine, Oslo University Hospital, Oslo, Norway
| | - Nicola Dinsdale
- Oxford Machine Learning in NeuroImaging (OMNI) Lab, University of Oxford, Oxford, UK
| | - Till Schellhorn
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- Division of Radiology and Nuclear Medicine, Oslo University Hospital, Oslo, Norway
| | - James M Roe
- Department of Psychology, University of Oslo, Oslo, Norway
| | | | | | - Tobias Kaufmann
- Norwegian Centre for Mental Disorders Research (NORMENT), Oslo University Hospital & Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health, University of Tübingen, Tübingen, Germany
- German Center for Mental Health (DZPG), Munich, Germany
| | - Eric Westman
- Division of Clinical Geriatrics, Department of Neurobiology, Care Sciences, and Society, Karolinska Institutet, Stockholm, Sweden
| | - Andre Marquand
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Medical Centre, Nijmegen, the Netherlands
| | - Geir Selbæk
- The Norwegian National Centre for Ageing and Health, Vestfold Hospital Trust, Tønsberg, Norway
- Department of Geriatric Medicine, Oslo University Hospital, Oslo, Norway
| | - Ole A Andreassen
- Norwegian Centre for Mental Disorders Research (NORMENT), Oslo University Hospital & Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- KG Jebsen Center for Neurodevelopmental Disorders, University of Oslo, Oslo, Norway
| | - Thomas Wolfers
- Department of Psychology, University of Oslo, Oslo, Norway
- Norwegian Centre for Mental Disorders Research (NORMENT), Oslo University Hospital & Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health, University of Tübingen, Tübingen, Germany
- German Center for Mental Health (DZPG), Munich, Germany
| | - Lars T Westlye
- Department of Psychology, University of Oslo, Oslo, Norway
- Norwegian Centre for Mental Disorders Research (NORMENT), Oslo University Hospital & Institute of Clinical Medicine, University of Oslo, Oslo, Norway
- KG Jebsen Center for Neurodevelopmental Disorders, University of Oslo, Oslo, Norway
| | - Yunpeng Wang
- Department of Psychology, University of Oslo, Oslo, Norway
| |
Collapse
|
6
|
Han G, Guo W, Zhang H, Jin J, Gan X, Zhao X. Sample self-selection using dual teacher networks for pathological image classification with noisy labels. Comput Biol Med 2024; 174:108489. [PMID: 38640633 DOI: 10.1016/j.compbiomed.2024.108489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 04/02/2024] [Accepted: 04/15/2024] [Indexed: 04/21/2024]
Abstract
Deep neural networks (DNNs) involve advanced image processing but depend on large quantities of high-quality labeled data. The presence of noisy data significantly degrades the DNN model performance. In the medical field, where model accuracy is crucial and labels for pathological images are scarce and expensive to obtain, the need to handle noisy data is even more urgent. Deep networks exhibit a memorization effect, they tend to prioritize remembering clean labels initially. Therefore, early stopping is highly effective in managing learning with noisy labels. Previous research has often concentrated on developing robust loss functions or implementing training constraints to mitigate the impact of noisy labels; however, such approaches have frequently resulted in underfitting. We propose using knowledge distillation to slow the learning process of the target network rather than preventing late-stage training from being affected by noisy labels. In this paper, we introduce a data sample self-selection strategy based on early stopping to filter out most of the noisy data. Additionally, we employ the distillation training method with dual teacher networks to ensure the steady learning of the student network. The experimental results show that our method outperforms current state-of-the-art methods for handling noisy labels on both synthetic and real-world noisy datasets. In particular, on the real-world pathological image dataset Chaoyang, the highest classification accuracy increased by 2.39 %. Our method leverages the model's predictions based on training history to select cleaner datasets and retrains them using these cleaner datasets, significantly mitigating the impact of noisy labels on model performance.
Collapse
Affiliation(s)
- Gang Han
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, China; School of Electronic and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Wenping Guo
- School of Electronic and Information Engineering, Taizhou University, Taizhou 318000, China.
| | - Haibo Zhang
- School of Electronic and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Jie Jin
- School of Electronic and Information Engineering, Taizhou University, Taizhou 318000, China
| | - Xingli Gan
- School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou 310023, China
| | - Xiaoming Zhao
- School of Electronic and Information Engineering, Taizhou University, Taizhou 318000, China
| |
Collapse
|
7
|
Fidon L, Aertsen M, Kofler F, Bink A, David AL, Deprest T, Emam D, Guffens F, Jakab A, Kasprian G, Kienast P, Melbourne A, Menze B, Mufti N, Pogledic I, Prayer D, Stuempflen M, Van Elslander E, Ourselin S, Deprest J, Vercauteren T. A Dempster-Shafer Approach to Trustworthy AI With Application to Fetal Brain MRI Segmentation. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024; 46:3784-3795. [PMID: 38198270 DOI: 10.1109/tpami.2023.3346330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of four backbone AI models for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.
Collapse
|
8
|
Jin X, Hao Y, Hilliard J, Zhang Z, Thomas MA, Li H, Jha AK, Hugo GD. A quality assurance framework for routine monitoring of deep learning cardiac substructure computed tomography segmentation models in radiotherapy. Med Phys 2024; 51:2741-2758. [PMID: 38015793 DOI: 10.1002/mp.16846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 10/24/2023] [Accepted: 10/30/2023] [Indexed: 11/30/2023] Open
Abstract
BACKGROUND For autosegmentation models, the data used to train the model (e.g., public datasets and/or vendor-collected data) and the data on which the model is deployed in the clinic are typically not the same, potentially impacting the performance of these models by a process called domain shift. Tools to routinely monitor and predict segmentation performance are needed for quality assurance. Here, we develop an approach to perform such monitoring and performance prediction for cardiac substructure segmentation. PURPOSE To develop a quality assurance (QA) framework for routine or continuous monitoring of domain shift and the performance of cardiac substructure autosegmentation algorithms. METHODS A benchmark dataset consisting of computed tomography (CT) images along with manual cardiac substructure delineations of 241 breast cancer radiotherapy patients were collected, including one "normal" image domain of clean images and five "abnormal" domains containing images with artifact (metal, contrast), pathology, or quality variations due to scanner protocol differences (field of view, noise, reconstruction kernel, and slice thickness). The QA framework consisted of an image domain shift detector which operated on the input CT images and a shape quality detector on the output of an autosegmentation model, and a regression model for predicting autosegmentation model performance. The image domain shift detector was composed of a trained denoising autoencoder (DAE) and two hand-engineered image quality features to detect normal versus abnormal domains in the input CT images. The shape quality detector was a variational autoencoder (VAE) trained to estimate the shape quality of the auto-segmentation results. The output from the image domain shift and shape quality detectors was used to train a regression model to predict the per-patient segmentation accuracy, measured by Dice coefficient similarity (DSC) to physician contours. Different regression techniques were investigated including linear regression, Bagging, Gaussian process regression, random forest, and gradient boost regression. Of the 241 patients, 60 were used to train the autosegmentation models, 120 for training the QA framework, and the remaining 61 for testing the QA framework. A total of 19 autosegmentation models were used to evaluate QA framework performance, including 18 convolutional neural network (CNN)-based and one transformer-based model. RESULTS When tested on the benchmark dataset, all abnormal domains resulted in a significant DSC decrease relative to the normal domain for CNN models (p < 0.001 $p < 0.001$ ), but only for some domains for the transformer model. No significant relationship was found between the performance of an autosegmentation model and scanner protocol parameters (p = 0.42 $p = 0.42$ ) except noise (p = 0.01 $p = 0.01$ ). CNN-based autosegmentation models demonstrated a decreased DSC ranging from 0.07 to 0.41 with added noise, while the transformer-based model was not significantly affected (ANOVA,p = 0.99 $p=0.99$ ). For the QA framework, linear regression models with bootstrap aggregation resulted in the highest mean absolute error (MAE) of0.041 ± 0.002 $0.041 \pm 0.002$ , in predicted DSC (relative to true DSC between autosegmentation and physician). MAE was lowest when combining both input (image) detectors and output (shape) detectors compared to output detectors alone. CONCLUSIONS A QA framework was able to predict cardiac substructure autosegmentation model performance for clinically anticipated "abnormal" domain shifts.
Collapse
Affiliation(s)
- Xiyao Jin
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| | - Yao Hao
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| | - Jessica Hilliard
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| | - Zhehao Zhang
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| | - Maria A Thomas
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| | - Hua Li
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| | - Abhinav K Jha
- Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, Missouri, USA
- Mallinckrodt Institute of Radiology, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Geoffrey D Hugo
- Department of Radiation Oncology, Washington University in St. Louis School of Medicine, St. Louis, Missouri, USA
| |
Collapse
|
9
|
Nakamura H, Fukuda M, Matsuda A, Makino N, Kimura H, Ohtaki Y, Nawa Y, Oyama S, Suzuki Y, Kobayashi T, Ishizawa T, Kakizaki Y, Ueno Y. Differentiating localized autoimmune pancreatitis and pancreatic ductal adenocarcinoma using endoscopic ultrasound images with deep learning. DEN OPEN 2024; 4:e344. [PMID: 38434146 PMCID: PMC10908399 DOI: 10.1002/deo2.344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Revised: 01/29/2024] [Accepted: 02/10/2024] [Indexed: 03/05/2024]
Abstract
Objectives Localized autoimmune pancreatitis is difficult to differentiate from pancreatic ductal adenocarcinoma on endoscopic ultrasound images. In recent years, deep learning methods have improved the diagnosis of diseases. Hence, we developed a special cross-validation framework to search for effective methodologies of deep learning in distinguishing autoimmune pancreatitis from pancreatic ductal adenocarcinoma on endoscopic ultrasound images. Methods Data from 24 patients diagnosed with localized autoimmune pancreatitis (8751 images) and 61 patients diagnosed with pancreatic ductal adenocarcinoma (20,584 images) were collected from 2016 to 2022. We applied transfer learning to a convolutional neural network called ResNet152, together with our innovative imaging method contributing to data augmentation and temporal data process. We divided patients into five groups according to different factors for 5-fold cross-validation, where the ordered and balanced datasets were created for the performance evaluations. Results ResNet152 surpassed the endoscopists in all evaluation metrics with almost all datasets. Interestingly, when the dataset is balanced according to the factor of the endoscopists' diagnostic accuracy, the area under the receiver operating characteristic curve and accuracy were highest at 0.85 and 0.80, respectively. Conclusions It is deduced that image features useful for ResNet152 correlate with those used by endoscopists for their diagnoses. This finding may contribute to sample-efficient dataset preparation to train convolutional neural networks for endoscopic ultrasonography-imaging diagnosis.
Collapse
Affiliation(s)
- Hitomi Nakamura
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Motohisa Fukuda
- Department of ScienceFaculty of ScienceYamagata UniversityYamagataJapan
| | - Akiko Matsuda
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Naohiko Makino
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | | | - Yu Ohtaki
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Yoshihito Nawa
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Soushi Oyama
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Yuya Suzuki
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Toshikazu Kobayashi
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Tetsuya Ishizawa
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Yasuharu Kakizaki
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| | - Yoshiyuki Ueno
- Department of GastroenterologyFaculty of MedicineYamagata UniversityYamagataJapan
| |
Collapse
|
10
|
Morita K, Karashima S, Terao T, Yoshida K, Yamashita T, Yoroidaka T, Tanabe M, Imi T, Zaimoku Y, Yoshida A, Maruyama H, Iwaki N, Aoki G, Kotani T, Murata R, Miyamoto T, Machida Y, Matsue K, Nambo H, Takamatsu H. 3D CNN-based Deep Learning Model-based Explanatory Prognostication in Patients with Multiple Myeloma using Whole-body MRI. J Med Syst 2024; 48:30. [PMID: 38456950 DOI: 10.1007/s10916-024-02040-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 02/05/2024] [Indexed: 03/09/2024]
Abstract
Although magnetic resonance imaging (MRI) data of patients with multiple myeloma (MM) are used to predict prognosis, few reports have applied artificial intelligence (AI) techniques for this purpose. We aimed to analyze whole-body diffusion-weighted MRI data using three-dimensional (3D) convolutional neural networks (CNNs) and Gradient-weighted Class Activation Mapping (Grad-CAM), an explainable AI, to predict prognosis and explore the factors involved in prediction. We retrospectively analyzed the MRI data of a total of 142 patients with MM obtained from two medical centers. We defined the occurrence of progressive disease after MRI evaluation within 12 months as a poor prognosis and constructed a 3D CNN-based deep learning model to predict prognosis. Images from 111 cases were used as the training and internal validation data; images from 31 cases were used as the external validation data. Internal validation of the AI model with stratified 5-fold cross-validation resulted in a significant difference in progression-free survival (PFS) between good and poor prognostic cases (2-year PFS, 91.2% versus [vs.] 61.1%, P = 0.0002). The AI model clearly stratified good and poor prognostic cases in the external validation cohort (2-year PFS, 92.9% vs. 55.6%, P = 0.004), with an area under the receiver operating characteristic curve of 0.804. According to Grad-CAM, the MRI signals of the spleen and bones of the vertebrae and pelvis contributed to prognosis prediction. This study is the first to show that image analysis of whole-body MRI using a 3D CNN without any other clinical data is effective in predicting the prognosis of patients with MM.
Collapse
Affiliation(s)
- Kento Morita
- School of Electrical, Information and Communication Engineering, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan
| | | | - Toshiki Terao
- Department of Hematology/Oncology, Kameda Medical Center, Kamogawa, Japan
- Department of Hematology and Oncology, Okayama University Hospital, Okayama, Japan
| | - Kotaro Yoshida
- Department of Radiology, Kanazawa University, Kanazawa, Japan
| | - Takeshi Yamashita
- Division of Internal Medicine, Keiju Kanazawa Hospital, Kanazawa, Japan
| | - Takeshi Yoroidaka
- Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Mikoto Tanabe
- Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan
| | - Tatsuya Imi
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Yoshitaka Zaimoku
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Akiyo Yoshida
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Hiroyuki Maruyama
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Noriko Iwaki
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Go Aoki
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Takeharu Kotani
- Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan
| | - Ryoichi Murata
- Division of Internal Medicine, Keiju Kanazawa Hospital, Kanazawa, Japan
| | - Toshihiro Miyamoto
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Youichi Machida
- Department of Radiology, Kameda Medical Center, Kamogawa, Japan
| | - Kosei Matsue
- Department of Hematology/Oncology, Kameda Medical Center, Kamogawa, Japan
| | - Hidetaka Nambo
- Faculty of Transdisciplinary Sciences for Innovation, Institute of Transdisciplinary Sciences for Innovation, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan.
| | - Hiroyuki Takamatsu
- Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan.
- Faculty of Transdisciplinary Sciences for Innovation, Institute of Transdisciplinary Sciences for Innovation, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan.
| |
Collapse
|
11
|
de Boer M, Kos TM, Fick T, van Doormaal JAM, Colombo E, Kuijf HJ, Robe PAJT, Regli LP, Bartels LW, van Doormaal TPC. NnU-Net versus mesh growing algorithm as a tool for the robust and timely segmentation of neurosurgical 3D images in contrast-enhanced T1 MRI scans. Acta Neurochir (Wien) 2024; 166:92. [PMID: 38376564 PMCID: PMC10879314 DOI: 10.1007/s00701-024-05973-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 01/22/2024] [Indexed: 02/21/2024]
Abstract
PURPOSE This study evaluates the nnU-Net for segmenting brain, skin, tumors, and ventricles in contrast-enhanced T1 (T1CE) images, benchmarking it against an established mesh growing algorithm (MGA). METHODS We used 67 retrospectively collected annotated single-center T1CE brain scans for training models for brain, skin, tumor, and ventricle segmentation. An additional 32 scans from two centers were used test performance compared to that of the MGA. The performance was measured using the Dice-Sørensen coefficient (DSC), intersection over union (IoU), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASSD) metrics, with time to segment also compared. RESULTS The nnU-Net models significantly outperformed the MGA (p < 0.0125) with a median brain segmentation DSC of 0.971 [95CI: 0.945-0.979], skin: 0.997 [95CI: 0.984-0.999], tumor: 0.926 [95CI: 0.508-0.968], and ventricles: 0.910 [95CI: 0.812-0.968]. Compared to the MGA's median DSC for brain: 0.936 [95CI: 0.890, 0.958], skin: 0.991 [95CI: 0.964, 0.996], tumor: 0.723 [95CI: 0.000-0.926], and ventricles: 0.856 [95CI: 0.216-0.916]. NnU-Net performance between centers did not significantly differ except for the skin segmentations Additionally, the nnU-Net models were faster (mean: 1139 s [95CI: 685.0-1616]) than the MGA (mean: 2851 s [95CI: 1482-6246]). CONCLUSIONS The nnU-Net is a fast, reliable tool for creating automatic deep learning-based segmentation pipelines, reducing the need for extensive manual tuning and iteration. The models are able to achieve this performance despite a modestly sized training set. The ability to create high-quality segmentations in a short timespan can prove invaluable in neurosurgical settings.
Collapse
Affiliation(s)
- Mathijs de Boer
- Image Sciences Institute, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX, Utrecht, The Netherlands.
| | - Tessa M Kos
- Image Sciences Institute, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX, Utrecht, The Netherlands
| | - Tim Fick
- Department of Neuro-Oncology, Princess Máxima Center for Pediatric Oncology, Utrecht, The Netherlands
| | | | - Elisa Colombo
- Department of Neurosurgery, University Hospital of Zürich, Zurich, Switzerland
| | - Hugo J Kuijf
- Image Sciences Institute, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX, Utrecht, The Netherlands
| | - Pierre A J T Robe
- Department of Neurosurgery, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Luca P Regli
- Department of Neurosurgery, University Hospital of Zürich, Zurich, Switzerland
| | - Lambertus W Bartels
- Image Sciences Institute, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX, Utrecht, The Netherlands
| | - Tristan P C van Doormaal
- Department of Neurosurgery, University Medical Center Utrecht, Utrecht, The Netherlands
- Department of Neurosurgery, University Hospital of Zürich, Zurich, Switzerland
| |
Collapse
|
12
|
Belov V, Erwin-Grabner T, Aghajani M, Aleman A, Amod AR, Basgoze Z, Benedetti F, Besteher B, Bülow R, Ching CRK, Connolly CG, Cullen K, Davey CG, Dima D, Dols A, Evans JW, Fu CHY, Gonul AS, Gotlib IH, Grabe HJ, Groenewold N, Hamilton JP, Harrison BJ, Ho TC, Mwangi B, Jaworska N, Jahanshad N, Klimes-Dougan B, Koopowitz SM, Lancaster T, Li M, Linden DEJ, MacMaster FP, Mehler DMA, Melloni E, Mueller BA, Ojha A, Oudega ML, Penninx BWJH, Poletti S, Pomarol-Clotet E, Portella MJ, Pozzi E, Reneman L, Sacchet MD, Sämann PG, Schrantee A, Sim K, Soares JC, Stein DJ, Thomopoulos SI, Uyar-Demir A, van der Wee NJA, van der Werff SJA, Völzke H, Whittle S, Wittfeld K, Wright MJ, Wu MJ, Yang TT, Zarate C, Veltman DJ, Schmaal L, Thompson PM, Goya-Maldonado R. Multi-site benchmark classification of major depressive disorder using machine learning on cortical and subcortical measures. Sci Rep 2024; 14:1084. [PMID: 38212349 PMCID: PMC10784593 DOI: 10.1038/s41598-023-47934-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 11/19/2023] [Indexed: 01/13/2024] Open
Abstract
Machine learning (ML) techniques have gained popularity in the neuroimaging field due to their potential for classifying neuropsychiatric disorders. However, the diagnostic predictive power of the existing algorithms has been limited by small sample sizes, lack of representativeness, data leakage, and/or overfitting. Here, we overcome these limitations with the largest multi-site sample size to date (N = 5365) to provide a generalizable ML classification benchmark of major depressive disorder (MDD) using shallow linear and non-linear models. Leveraging brain measures from standardized ENIGMA analysis pipelines in FreeSurfer, we were able to classify MDD versus healthy controls (HC) with a balanced accuracy of around 62%. But after harmonizing the data, e.g., using ComBat, the balanced accuracy dropped to approximately 52%. Accuracy results close to random chance levels were also observed in stratified groups according to age of onset, antidepressant use, number of episodes and sex. Future studies incorporating higher dimensional brain imaging/phenotype features, and/or using more advanced machine and deep learning methods may yield more encouraging prospects.
Collapse
Affiliation(s)
- Vladimir Belov
- Laboratory of Systems Neuroscience and Imaging in Psychiatry (SNIP-Lab), Department of Psychiatry and Psychotherapy, University Medical Center Göttingen (UMG), Georg-August University, Von-Siebold-Str. 5, 37075, Göttingen, Germany
| | - Tracy Erwin-Grabner
- Laboratory of Systems Neuroscience and Imaging in Psychiatry (SNIP-Lab), Department of Psychiatry and Psychotherapy, University Medical Center Göttingen (UMG), Georg-August University, Von-Siebold-Str. 5, 37075, Göttingen, Germany
| | - Moji Aghajani
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Institute of Education and Child Studies, Section Forensic Family and Youth Care, Leiden University, Leiden, The Netherlands
| | - Andre Aleman
- Department of Biomedical Sciences of Cells and Systems, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
| | - Alyssa R Amod
- Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
| | - Zeynep Basgoze
- Department of Psychiatry and Behavioral Science, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Francesco Benedetti
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Bianca Besteher
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
| | - Robin Bülow
- Institute for Radiology and Neuroradiology, University Medicine Greifswald, Greifswald, Germany
| | - Christopher R K Ching
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | - Colm G Connolly
- Department of Biomedical Sciences, Florida State University, Tallahassee, FL, USA
| | - Kathryn Cullen
- Department of Psychiatry and Behavioral Science, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Christopher G Davey
- Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, VIC, Australia
| | - Danai Dima
- Department of Psychology, School of Arts and Social Sciences, City, University of London, London, UK
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Annemiek Dols
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Jennifer W Evans
- Experimental Therapeutics and Pathophysiology Branch, National Institute for Mental Health, National Institutes of Health, Bethesda, MD, USA
| | - Cynthia H Y Fu
- School of Psychology, University of East London, London, UK
- Centre for Affective Disorders, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Ali Saffet Gonul
- SoCAT Lab, Department of Psychiatry, School of Medicine, Ege University, Izmir, Turkey
| | - Ian H Gotlib
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Hans J Grabe
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, Greifswald, Germany
| | - Nynke Groenewold
- Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa
| | - J Paul Hamilton
- Center for Social and Affective Neuroscience, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
- Center for Medical Imaging and Visualization, Linköping University, Linköping, Sweden
| | - Ben J Harrison
- Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, VIC, Australia
| | - Tiffany C Ho
- Department of Psychiatry and Behavioral Sciences, Division of Child and Adolescent Psychiatry, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, USA
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Benson Mwangi
- Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences, The University of Texas Health Science Center at Houston, Houston, TX, USA
- Center Of Excellence On Mood Disorders, Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences at McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Natalia Jaworska
- Department of Psychiatry, McGill University, Montreal, QC, Canada
| | - Neda Jahanshad
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | | | | | - Thomas Lancaster
- Cardiff University Brain Research Imaging Center, Cardiff University, Cardiff, UK
- MRC Center for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, UK
| | - Meng Li
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
| | - David E J Linden
- Cardiff University Brain Research Imaging Center, Cardiff University, Cardiff, UK
- MRC Center for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, UK
- Division of Psychological Medicine and Clinical Neurosciences, Cardiff University, Cardiff, UK
- School of Mental Health and Neuroscience, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Frank P MacMaster
- Departments of Psychiatry and Pediatrics, University of Calgary, Calgary, AB, Canada
| | - David M A Mehler
- Cardiff University Brain Research Imaging Center, Cardiff University, Cardiff, UK
- MRC Center for Neuropsychiatric Genetics and Genomics, Cardiff University, Cardiff, UK
- Department of Psychiatry, Psychotherapy and Psychosomatics, Medical School, RWTH Aachen University, Aachen, Germany
| | - Elisa Melloni
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Bryon A Mueller
- Department of Psychiatry and Behavioral Science, University of Minnesota Medical School, Minneapolis, MN, USA
| | - Amar Ojha
- Center for Neuroscience, University of Pittsburgh, Pittsburgh, PA, USA
- Center for Neural Basis of Cognition, University of Pittsburgh, Pittsburgh, PA, USA
| | - Mardien L Oudega
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Brenda W J H Penninx
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Sara Poletti
- Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Edith Pomarol-Clotet
- FIDMAG Germanes Hospitalàries Research Foundation, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Barcelona, Catalonia, Spain
| | - Maria J Portella
- Sant Pau Mental Health Research Group, Institut de Recerca de L'Hospital de La Santa Creu I Sant Pau, Barcelona, Catalonia, Spain
| | - Elena Pozzi
- Centre for Youth Mental Health, The University of Melbourne, Parkville, VIC, Australia
- Orygen, Parkville, VIC, Australia
| | - Liesbeth Reneman
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Matthew D Sacchet
- Meditation Research Program, Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | | | - Anouk Schrantee
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Kang Sim
- West Region, Institute of Mental Health, Singapore, Singapore
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Jair C Soares
- Center Of Excellence On Mood Disorders, Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences at McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Dan J Stein
- SA MRC Research Unit on Risk and Resilience in Mental Disorders, Department of Psychiatry and Neuroscience Institute, University of Cape Town, Cape Town, South Africa
| | - Sophia I Thomopoulos
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | - Aslihan Uyar-Demir
- SoCAT Lab, Department of Psychiatry, School of Medicine, Ege University, Izmir, Turkey
| | - Nic J A van der Wee
- Leiden Institute for Brain and Cognition, Leiden University Medical Center, Leiden, The Netherlands
| | - Steven J A van der Werff
- Leiden Institute for Brain and Cognition, Leiden University Medical Center, Leiden, The Netherlands
- Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands
| | - Henry Völzke
- Institute for Community Medicine, University Medicine Greifswald, Greifswald, Germany
| | - Sarah Whittle
- Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne and Melbourne Health, Melbourne, VIC, Australia
| | - Katharina Wittfeld
- Department of Psychiatry and Psychotherapy, University Medicine Greifswald, Greifswald, Germany
- German Center for Neurodegenerative Diseases (DZNE), Site Rostock/ Greifswald, Greifswald, Germany
| | - Margaret J Wright
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
- Centre for Advanced Imaging, The University of Queensland, Brisbane, QLD, Australia
| | - Mon-Ju Wu
- Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences, The University of Texas Health Science Center at Houston, Houston, TX, USA
- Center Of Excellence On Mood Disorders, Louis A. Faillace, MD, Department of Psychiatry and Behavioral Sciences at McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Tony T Yang
- Department of Psychiatry and Behavioral Sciences, Division of Child and Adolescent Psychiatry, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, USA
| | - Carlos Zarate
- Section on the Neurobiology and Treatment of Mood Disorders, National Institute of Mental Health, Bethesda, MD, USA
| | - Dick J Veltman
- Department of Psychiatry, Amsterdam UMC, Amsterdam Neuroscience, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Lianne Schmaal
- Centre for Youth Mental Health, The University of Melbourne, Parkville, VIC, Australia
- Orygen, Parkville, VIC, Australia
| | - Paul M Thompson
- Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA, USA
| | - Roberto Goya-Maldonado
- Laboratory of Systems Neuroscience and Imaging in Psychiatry (SNIP-Lab), Department of Psychiatry and Psychotherapy, University Medical Center Göttingen (UMG), Georg-August University, Von-Siebold-Str. 5, 37075, Göttingen, Germany.
| |
Collapse
|
13
|
Dartora C, Marseglia A, Mårtensson G, Rukh G, Dang J, Muehlboeck JS, Wahlund LO, Moreno R, Barroso J, Ferreira D, Schiöth HB, Westman E. A deep learning model for brain age prediction using minimally preprocessed T1w images as input. Front Aging Neurosci 2024; 15:1303036. [PMID: 38259636 PMCID: PMC10800627 DOI: 10.3389/fnagi.2023.1303036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 12/04/2023] [Indexed: 01/24/2024] Open
Abstract
Introduction In the last few years, several models trying to calculate the biological brain age have been proposed based on structural magnetic resonance imaging scans (T1-weighted MRIs, T1w) using multivariate methods and machine learning. We developed and validated a convolutional neural network (CNN)-based biological brain age prediction model that uses one T1w MRI preprocessing step when applying the model to external datasets to simplify implementation and increase accessibility in research settings. Our model only requires rigid image registration to the MNI space, which is an advantage compared to previous methods that require more preprocessing steps, such as feature extraction. Methods We used a multicohort dataset of cognitively healthy individuals (age range = 32.0-95.7 years) comprising 17,296 MRIs for training and evaluation. We compared our model using hold-out (CNN1) and cross-validation (CNN2-4) approaches. To verify generalisability, we used two external datasets with different populations and MRI scan characteristics to evaluate the model. To demonstrate its usability, we included the external dataset's images in the cross-validation training (CNN3). To ensure that our model used only the brain signal on the image, we also predicted brain age using skull-stripped images (CNN4). Results The trained models achieved a mean absolute error of 2.99, 2.67, 2.67, and 3.08 years for CNN1-4, respectively. The model's performance in the external dataset was in the typical range of mean absolute error (MAE) found in the literature for testing sets. Adding the external dataset to the training set (CNN3), overall, MAE is unaffected, but individual cohort MAE improves (5.63-2.25 years). Salience maps of predictions reveal that periventricular, temporal, and insular regions are the most important for age prediction. Discussion We provide indicators for using biological (predicted) brain age as a metric for age correction in neuroimaging studies as an alternative to the traditional chronological age. In conclusion, using different approaches, our CNN-based model showed good performance using one T1w brain MRI preprocessing step. The proposed CNN model is made publicly available for the research community to be easily implemented and used to study ageing and age-related disorders.
Collapse
Affiliation(s)
- Caroline Dartora
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| | - Anna Marseglia
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| | - Gustav Mårtensson
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| | - Gull Rukh
- Department of Surgical Sciences, Functional Pharmacology and Neuroscience, Uppsala University, Uppsala, Sweden
| | - Junhua Dang
- Department of Surgical Sciences, Functional Pharmacology and Neuroscience, Uppsala University, Uppsala, Sweden
| | - J-Sebastian Muehlboeck
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| | - Lars-Olof Wahlund
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| | - Rodrigo Moreno
- Department of Biomedical Engineering and Health Systems, KTH Royal Institute of Technology, Stockholm, Sweden
| | - José Barroso
- Facultad de Ciencias de la Salud, Universidad Fernando Pessoa Canarias, Las Palmas, España
| | - Daniel Ferreira
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
- Facultad de Ciencias de la Salud, Universidad Fernando Pessoa Canarias, Las Palmas, España
| | - Helgi B. Schiöth
- Department of Surgical Sciences, Functional Pharmacology and Neuroscience, Uppsala University, Uppsala, Sweden
| | - Eric Westman
- Division of Clinical Geriatrics, Center for Alzheimer Research, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
- Department of Neuroimaging, Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
| | | | | | | | | |
Collapse
|
14
|
Shah J, Siddiquee MMR, Su Y, Wu T, Li B. Ordinal Classification with Distance Regularization for Robust Brain Age Prediction. IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION. IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION 2024; 2024:7867-7876. [PMID: 38606366 PMCID: PMC11008505 DOI: 10.1109/wacv57701.2024.00770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/13/2024]
Abstract
Age is one of the major known risk factors for Alzheimer's Disease (AD). Detecting AD early is crucial for effective treatment and preventing irreversible brain damage. Brain age, a measure derived from brain imaging reflecting structural changes due to aging, may have the potential to identify AD onset, assess disease risk, and plan targeted interventions. Deep learning-based regression techniques to predict brain age from magnetic resonance imaging (MRI) scans have shown great accuracy recently. However, these methods are subject to an inherent regression to the mean effect, which causes a systematic bias resulting in an overestimation of brain age in young subjects and underestimation in old subjects. This weakens the reliability of predicted brain age as a valid biomarker for downstream clinical applications. Here, we reformulate the brain age prediction task from regression to classification to address the issue of systematic bias. Recognizing the importance of preserving ordinal information from ages to understand aging trajectory and monitor aging longitudinally, we propose a novel ORdinal Distance Encoded Regularization (ORDER) loss that incorporates the order of age labels, enhancing the model's ability to capture age-related patterns. Extensive experiments and ablation studies demonstrate that this framework reduces systematic bias, outperforms state-of-art methods by statistically significant margins, and can better capture subtle differences between clinical groups in an independent AD dataset. Our implementation is publicly available at https://github.com/jaygshah/Robust-Brain-Age-Prediction.
Collapse
Affiliation(s)
- Jay Shah
- Arizona State University
- ASU-Mayo Center for Innovative Imaging
| | | | - Yi Su
- ASU-Mayo Center for Innovative Imaging
- Banner Alzheimer's Institute
| | - Teresa Wu
- Arizona State University
- ASU-Mayo Center for Innovative Imaging
| | - Baoxin Li
- Arizona State University
- ASU-Mayo Center for Innovative Imaging
| |
Collapse
|
15
|
Mehrtash A, Ziegler E, Idris T, Somarouthu B, Urban T, LaCasce AS, Jacene H, Van Den Abbeele AD, Pieper S, Harris G, Kikinis R, Kapur T. Evaluation of mediastinal lymph node segmentation of heterogeneous CT data with full and weak supervision. Comput Med Imaging Graph 2024; 111:102312. [PMID: 38141568 PMCID: PMC10807363 DOI: 10.1016/j.compmedimag.2023.102312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 10/31/2023] [Accepted: 11/11/2023] [Indexed: 12/25/2023]
Abstract
Accurate lymph node size estimation is critical for staging cancer patients, initial therapeutic management, and assessing response to therapy. Current standard practice for quantifying lymph node size is based on a variety of criteria that use uni-directional or bi-directional measurements. Segmentation in 3D can provide more accurate evaluations of the lymph node size. Fully convolutional neural networks (FCNs) have achieved state-of-the-art results in segmentation for numerous medical imaging applications, including lymph node segmentation. Adoption of deep learning segmentation models in clinical trials often faces numerous challenges. These include lack of pixel-level ground truth annotations for training, generalizability of the models on unseen test domains due to the heterogeneity of test cases and variation of imaging parameters. In this paper, we studied and evaluated the performance of lymph node segmentation models on a dataset that was completely independent of the one used to create the models. We analyzed the generalizability of the models in the face of a heterogeneous dataset and assessed the potential effects of different disease conditions and imaging parameters. Furthermore, we systematically compared fully-supervised and weakly-supervised methods in this context. We evaluated the proposed methods using an independent dataset comprising 806 mediastinal lymph nodes from 540 unique patients. The results show that performance achieved on the independent test set is comparable to that on the training set. Furthermore, neither the underlying disease nor the heterogeneous imaging parameters impacted the performance of the models. Finally, the results indicate that our weakly-supervised method attains 90%- 91% of the performance achieved by the fully supervised training.
Collapse
Affiliation(s)
- Alireza Mehrtash
- Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA.
| | | | - Tagwa Idris
- Department of Radiology, Mass General Hospital, Boston, MA, USA
| | | | | | - Ann S LaCasce
- Department of Medical Oncology, Dana Farber Cancer Institute, Boston, MA, USA
| | - Heather Jacene
- Department of Radiology, Brigham and Women's Hospital; Boston, MA, USA; Department of Imaging, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Annick D Van Den Abbeele
- Department of Imaging, Dana-Farber Cancer Institute, USA; Department of Radiology, Mass General Brigham, USA; Harvard Medical School, USA
| | | | - Gordon Harris
- Yunu, Inc., Cary, NC, USA; Department of Radiology, Mass General Hospital, Boston, MA, USA
| | - Ron Kikinis
- Harvard Medical School, Department of Radiology, Brigham and Women's Hospital, USA
| | - Tina Kapur
- Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA
| |
Collapse
|
16
|
Aggarwal K, Manso Jimeno M, Ravi KS, Gonzalez G, Geethanath S. Developing and deploying deep learning models in brain magnetic resonance imaging: A review. NMR IN BIOMEDICINE 2023; 36:e5014. [PMID: 37539775 DOI: 10.1002/nbm.5014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/05/2023]
Abstract
Magnetic resonance imaging (MRI) of the brain has benefited from deep learning (DL) to alleviate the burden on radiologists and MR technologists, and improve throughput. The easy accessibility of DL tools has resulted in a rapid increase of DL models and subsequent peer-reviewed publications. However, the rate of deployment in clinical settings is low. Therefore, this review attempts to bring together the ideas from data collection to deployment in the clinic, building on the guidelines and principles that accreditation agencies have espoused. We introduce the need for and the role of DL to deliver accessible MRI. This is followed by a brief review of DL examples in the context of neuropathologies. Based on these studies and others, we collate the prerequisites to develop and deploy DL models for brain MRI. We then delve into the guiding principles to develop good machine learning practices in the context of neuroimaging, with a focus on explainability. A checklist based on the United States Food and Drug Administration's good machine learning practices is provided as a summary of these guidelines. Finally, we review the current challenges and future opportunities in DL for brain MRI.
Collapse
Affiliation(s)
- Kunal Aggarwal
- Accessible MR Laboratory, Biomedical Engineering and Imaging Institute, Department of Diagnostic, Molecular and Interventional Radiology, Mount Sinai Hospital, New York, USA
- Department of Electrical and Computer Engineering, Technical University Munich, Munich, Germany
| | - Marina Manso Jimeno
- Department of Biomedical Engineering, Columbia University in the City of New York, New York, New York, USA
- Columbia Magnetic Resonance Research Center, Columbia University in the City of New York, New York, New York, USA
| | - Keerthi Sravan Ravi
- Department of Biomedical Engineering, Columbia University in the City of New York, New York, New York, USA
- Columbia Magnetic Resonance Research Center, Columbia University in the City of New York, New York, New York, USA
| | - Gilberto Gonzalez
- Division of Neuroradiology, Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Sairam Geethanath
- Accessible MR Laboratory, Biomedical Engineering and Imaging Institute, Department of Diagnostic, Molecular and Interventional Radiology, Mount Sinai Hospital, New York, USA
| |
Collapse
|
17
|
Bucholc M, James C, Khleifat AA, Badhwar A, Clarke N, Dehsarvi A, Madan CR, Marzi SJ, Shand C, Schilder BM, Tamburin S, Tantiangco HM, Lourida I, Llewellyn DJ, Ranson JM. Artificial intelligence for dementia research methods optimization. Alzheimers Dement 2023; 19:5934-5951. [PMID: 37639369 DOI: 10.1002/alz.13441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 07/19/2023] [Accepted: 07/23/2023] [Indexed: 08/31/2023]
Abstract
Artificial intelligence (AI) and machine learning (ML) approaches are increasingly being used in dementia research. However, several methodological challenges exist that may limit the insights we can obtain from high-dimensional data and our ability to translate these findings into improved patient outcomes. To improve reproducibility and replicability, researchers should make their well-documented code and modeling pipelines openly available. Data should also be shared where appropriate. To enhance the acceptability of models and AI-enabled systems to users, researchers should prioritize interpretable methods that provide insights into how decisions are generated. Models should be developed using multiple, diverse datasets to improve robustness, generalizability, and reduce potentially harmful bias. To improve clarity and reproducibility, researchers should adhere to reporting guidelines that are co-produced with multiple stakeholders. If these methodological challenges are overcome, AI and ML hold enormous promise for changing the landscape of dementia research and care. HIGHLIGHTS: Machine learning (ML) can improve diagnosis, prevention, and management of dementia. Inadequate reporting of ML procedures affects reproduction/replication of results. ML models built on unrepresentative datasets do not generalize to new datasets. Obligatory metrics for certain model structures and use cases have not been defined. Interpretability and trust in ML predictions are barriers to clinical translation.
Collapse
Affiliation(s)
- Magda Bucholc
- Cognitive Analytics Research Lab, School of Computing, Engineering & Intelligent Systems, Ulster University, Derry, UK
| | - Charlotte James
- NIHR Bristol Biomedical Research Centre, University Hospitals Bristol and Weston NHS Foundation Trust and University of Bristol, Bristol, UK
| | - Ahmad Al Khleifat
- Department of Basic and Clinical Neuroscience, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - AmanPreet Badhwar
- Multiomics Investigation of Neurodegenerative Diseases (MIND) Lab, Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, Montréal, Quebec, Canada
- Institut de génie biomédical, Université de Montréal, Montréal, Quebec, Canada
- Département de Pharmacologie et Physiologie, Université de Montréal, Montréal, Quebec, Canada
| | - Natasha Clarke
- Multiomics Investigation of Neurodegenerative Diseases (MIND) Lab, Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal, Montréal, Quebec, Canada
| | - Amir Dehsarvi
- Aberdeen Biomedical Imaging Centre, School of Medicine, Medical Sciences, and Nutrition, University of Aberdeen, Aberdeen, UK
| | | | - Sarah J Marzi
- UK Dementia Research Institute, Imperial College London, London, UK
- Department of Brain Sciences, Imperial College London, London, UK
| | - Cameron Shand
- Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK
| | - Brian M Schilder
- UK Dementia Research Institute, Imperial College London, London, UK
- Department of Brain Sciences, Imperial College London, London, UK
| | - Stefano Tamburin
- Department of Neurosciences, Biomedicine and Movement Sciences, University of Verona, Verona, Italy
| | | | | | - David J Llewellyn
- University of Exeter Medical School, Exeter, UK
- The Alan Turing Institute, London, UK
| | | |
Collapse
|
18
|
Winchester LM, Harshfield EL, Shi L, Badhwar A, Khleifat AA, Clarke N, Dehsarvi A, Lengyel I, Lourida I, Madan CR, Marzi SJ, Proitsi P, Rajkumar AP, Rittman T, Silajdžić E, Tamburin S, Ranson JM, Llewellyn DJ. Artificial intelligence for biomarker discovery in Alzheimer's disease and dementia. Alzheimers Dement 2023; 19:5860-5871. [PMID: 37654029 PMCID: PMC10840606 DOI: 10.1002/alz.13390] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 06/13/2023] [Accepted: 06/19/2023] [Indexed: 09/02/2023]
Abstract
With the increase in large multimodal cohorts and high-throughput technologies, the potential for discovering novel biomarkers is no longer limited by data set size. Artificial intelligence (AI) and machine learning approaches have been developed to detect novel biomarkers and interactions in complex data sets. We discuss exemplar uses and evaluate current applications and limitations of AI to discover novel biomarkers. Remaining challenges include a lack of diversity in the data sets available, the sheer complexity of investigating interactions, the invasiveness and cost of some biomarkers, and poor reporting in some studies. Overcoming these challenges will involve collecting data from underrepresented populations, developing more powerful AI approaches, validating the use of noninvasive biomarkers, and adhering to reporting guidelines. By harnessing rich multimodal data through AI approaches and international collaborative innovation, we are well positioned to identify clinically useful biomarkers that are accurate, generalizable, unbiased, and acceptable in clinical practice. HIGHLIGHTS: Artificial intelligence and machine learning approaches may accelerate dementia biomarker discovery. Remaining challenges include data set suitability due to size and bias in cohort selection. Multimodal data, diverse data sets, improved machine learning approaches, real-world validation, and interdisciplinary collaboration are required.
Collapse
Affiliation(s)
| | - Eric L Harshfield
- Department of Clinical Neurosciences, Stroke Research Group, University of Cambridge, Cambridge, UK
| | - Liu Shi
- Novo Nordisk Research Centre Oxford (NNRCO), Headington, UK
| | - AmanPreet Badhwar
- Département de Pharmacologie et Physiologie, Institut de Génie Biomédical, Faculté de Médecine, Université de Montréal, Montreal, Canada
- Centre de recherche de l'Institut Universitaire de Gériatrie (CRIUGM), Montreal, Canada
| | - Ahmad Al Khleifat
- Department of Basic and Clinical Neuroscience, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Natasha Clarke
- Centre de recherche de l'Institut Universitaire de Gériatrie (CRIUGM), Montreal, Canada
| | - Amir Dehsarvi
- School of Medicine, Medical Sciences, and Nutrition, University of Aberdeen, Aberdeen, UK
| | - Imre Lengyel
- Wellcome-Wolfson Institute of Experimental Medicine, Queen's University, Belfast, UK
| | - Ilianna Lourida
- Health and Community Sciences, University of Exeter Medical School, Exeter, UK
| | | | - Sarah J Marzi
- UK Dementia Research Institute at Imperial College London, London, UK
- Department of Brain Sciences, Imperial College London, London, UK
| | - Petroula Proitsi
- Department of Basic and Clinical Neuroscience, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Anto P Rajkumar
- Institute of Mental Health, Mental Health and Clinical Neurosciences academic unit, University of Nottingham, Nottingham, UK, Mental health services of older people, Nottinghamshire healthcare NHS foundation trust, Nottingham, UK
| | - Timothy Rittman
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK
| | - Edina Silajdžić
- Department of Basic and Clinical Neuroscience, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Stefano Tamburin
- Department of Neurosciences, Biomedicine and Movement Sciences, University of Verona, Verona, Italy
| | - Janice M Ranson
- Health and Community Sciences, University of Exeter Medical School, Exeter, UK
| | - David J Llewellyn
- Health and Community Sciences, University of Exeter Medical School, Exeter, UK
- The Alan Turing Institute, London, UK
| |
Collapse
|
19
|
Casella B, Riviera W, Aldinucci M, Menegaz G. MERGE: A model for multi-input biomedical federated learning. PATTERNS (NEW YORK, N.Y.) 2023; 4:100856. [PMID: 38035188 PMCID: PMC10682752 DOI: 10.1016/j.patter.2023.100856] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 09/11/2023] [Accepted: 09/13/2023] [Indexed: 12/02/2023]
Abstract
Driven by the deep learning (DL) revolution, artificial intelligence (AI) has become a fundamental tool for many biomedical tasks, including analyzing and classifying diagnostic images. Imaging, however, is not the only source of information. Tabular data, such as personal and genomic data and blood test results, are routinely collected but rarely considered in DL pipelines. Nevertheless, DL requires large datasets that often must be pooled from different institutions, raising non-trivial privacy concerns. Federated learning (FL) is a cooperative learning paradigm that aims to address these issues by moving models instead of data across different institutions. Here, we present a federated multi-input architecture using images and tabular data as a methodology to enhance model performance while preserving data privacy. We evaluated it on two showcases: the prognosis of COVID-19 and patients' stratification in Alzheimer's disease, providing evidence of enhanced accuracy and F1 scores against single-input models and improved generalizability against non-federated models.
Collapse
Affiliation(s)
- Bruno Casella
- Department of Computer Science, University of Turin, 10149 Turin, Italy
| | - Walter Riviera
- Department of Computer Science, University of Verona, 37134 Verona, Italy
| | - Marco Aldinucci
- Department of Computer Science, University of Turin, 10149 Turin, Italy
| | - Gloria Menegaz
- Department of Engineering for Innovation Medicine, University of Verona, 37134 Verona, Italy
| |
Collapse
|
20
|
Rashidisabet H, Sethi A, Jindarak P, Edmonds J, Chan RVP, Leiderman YI, Vajaranant TS, Yi D. Validating the Generalizability of Ophthalmic Artificial Intelligence Models on Real-World Clinical Data. Transl Vis Sci Technol 2023; 12:8. [PMID: 37922149 PMCID: PMC10629532 DOI: 10.1167/tvst.12.11.8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Accepted: 08/21/2023] [Indexed: 11/05/2023] Open
Abstract
Purpose This study aims to investigate generalizability of deep learning (DL) models trained on commonly used public fundus images to an instance of real-world data (RWD) for glaucoma diagnosis. Methods We used Illinois Eye and Ear Infirmary fundus data set as an instance of RWD in addition to six publicly available fundus data sets. We compared the performance of DL-trained models on public data and RWD for glaucoma classification and optic disc (OD) segmentation tasks. For each task, we created models trained on each data set, respectively, and each model was tested on both data sets. We further examined each model's decision-making process and learned embeddings for the glaucoma classification task. Results Using public data for the test set, public-trained models outperformed RWD-trained models in OD segmentation and glaucoma classification with a mean intersection over union of 96.3% and mean area under the receiver operating characteristic curve of 95.0%, respectively. Using the RWD test set, the performance of public models decreased by 8.0% and 18.4% to 85.6% and 76.6% for OD segmentation and glaucoma classification tasks, respectively. RWD models outperformed public models on RWD test sets by 2.0% and 9.5%, respectively, in OD segmentation and glaucoma classification tasks. Conclusions DL models trained on commonly used public data have limited ability to generalize to RWD for classifying glaucoma. They perform similarly to RWD models for OD segmentation. Translational Relevance RWD is a potential solution for improving generalizability of DL models and enabling clinical translations in the care of prevalent blinding ophthalmic conditions, such as glaucoma.
Collapse
Affiliation(s)
- Homa Rashidisabet
- Department of Biomedical Engineering, University of Illinois Chicago, Chicago, IL, USA
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
| | - Abhishek Sethi
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| | - Ponpawee Jindarak
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| | - James Edmonds
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| | - R V Paul Chan
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| | - Yannek I Leiderman
- Department of Biomedical Engineering, University of Illinois Chicago, Chicago, IL, USA
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| | - Thasarat Sutabutr Vajaranant
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| | - Darvin Yi
- Department of Biomedical Engineering, University of Illinois Chicago, Chicago, IL, USA
- Artificial Intelligence in Ophthalmology (Ai-O) Center, University of Illinois Chicago, Chicago, IL, USA
- Illinois Eye and Ear Infirmary, Department of Ophthalmology and Visual Sciences, University of Illinois Chicago, Chicago, IL, USA
| |
Collapse
|
21
|
Gentile G, Jenkinson M, Griffanti L, Luchetti L, Leoncini M, Inderyas M, Mortilla M, Cortese R, De Stefano N, Battaglini M. BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation. Hum Brain Mapp 2023; 44:4893-4913. [PMID: 37530598 PMCID: PMC10472913 DOI: 10.1002/hbm.26424] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 05/20/2023] [Accepted: 07/06/2023] [Indexed: 08/03/2023] Open
Abstract
In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.
Collapse
Affiliation(s)
- Giordano Gentile
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| | - Mark Jenkinson
- Welcome Centre for Integrative Neuroimaging (WIN), FMRIB, Nuffield Department of Clinical NeurosciencesUniversity of Oxford, John Radcliffe HospitalOxfordUK
- Australian Institute of Machine Learning (AIML), School of Computer and Mathematical SciencesUniversity of AdelaideAdelaideSouth AustraliaAustralia
- South Australian Health and Medical Research Institute (SAHMRI)AdelaideSouth AustraliaAustralia
| | - Ludovica Griffanti
- Welcome Centre for Integrative Neuroimaging (WIN), FMRIB, Nuffield Department of Clinical NeurosciencesUniversity of Oxford, John Radcliffe HospitalOxfordUK
- Welcome Centre for Integrative Neuroimaging (WIN), OHBA, Department of PsychiatryUniversity of Oxford, Warneford HospitalOxfordUK
| | - Ludovico Luchetti
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
| | - Matteo Leoncini
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| | - Maira Inderyas
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| | | | - Rosa Cortese
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
| | - Nicola De Stefano
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
| | - Marco Battaglini
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| |
Collapse
|
22
|
Sahiner B, Chen W, Samala RK, Petrick N. Data drift in medical machine learning: implications and potential remedies. Br J Radiol 2023; 96:20220878. [PMID: 36971405 PMCID: PMC10546450 DOI: 10.1259/bjr.20220878] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Revised: 02/16/2023] [Accepted: 02/20/2023] [Indexed: 03/29/2023] Open
Abstract
Data drift refers to differences between the data used in training a machine learning (ML) model and that applied to the model in real-world operation. Medical ML systems can be exposed to various forms of data drift, including differences between the data sampled for training and used in clinical operation, differences between medical practices or context of use between training and clinical use, and time-related changes in patient populations, disease patterns, and data acquisition, to name a few. In this article, we first review the terminology used in ML literature related to data drift, define distinct types of drift, and discuss in detail potential causes within the context of medical applications with an emphasis on medical imaging. We then review the recent literature regarding the effects of data drift on medical ML systems, which overwhelmingly show that data drift can be a major cause for performance deterioration. We then discuss methods for monitoring data drift and mitigating its effects with an emphasis on pre- and post-deployment techniques. Some of the potential methods for drift detection and issues around model retraining when drift is detected are included. Based on our review, we find that data drift is a major concern in medical ML deployment and that more research is needed so that ML models can identify drift early, incorporate effective mitigation strategies and resist performance decay.
Collapse
Affiliation(s)
- Berkman Sahiner
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| | - Weijie Chen
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| | - Ravi K. Samala
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| | - Nicholas Petrick
- Center for Devices and Radiological Health, U.S. Food and Drug Administration 10903 New Hampshire Avenue, Silver Spring, MD 20993-0002
| |
Collapse
|
23
|
Jacobs PP, Ehrengut C, Bucher AM, Penzkofer T, Lukas M, Kleesiek J, Denecke T. Challenges in Implementing the Local Node Infrastructure for a National Federated Machine Learning Network in Radiology. Healthcare (Basel) 2023; 11:2377. [PMID: 37685411 PMCID: PMC10487228 DOI: 10.3390/healthcare11172377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 08/20/2023] [Accepted: 08/22/2023] [Indexed: 09/10/2023] Open
Abstract
Data-driven machine learning in medical research and diagnostics needs large-scale datasets curated by clinical experts. The generation of large datasets can be challenging in terms of resource consumption and time effort, while generalizability and validation of the developed models significantly benefit from variety in data sources. Training algorithms on smaller decentralized datasets through federated learning can reduce effort, but require the implementation of a specific and ambitious infrastructure to share data, algorithms and computing time. Additionally, it offers the opportunity of maintaining and keeping the data locally. Thus, data safety issues can be avoided because patient data must not be shared. Machine learning models are trained on local data by sharing the model and through an established network. In addition to commercial applications, there are also numerous academic and customized implementations of network infrastructures available. The configuration of these networks primarily differs, yet adheres to a standard framework composed of fundamental components. In this technical note, we propose basic infrastructure requirements for data governance, data science workflows, and local node set-up, and report on the advantages and experienced pitfalls in implementing the local infrastructure with the German Radiological Cooperative Network initiative as the use case example. We show how the infrastructure can be built upon some base components to reflect the needs of a federated learning network and how they can be implemented considering both local and global network requirements. After analyzing the deployment process in different settings and scenarios, we recommend integrating the local node into an existing clinical IT infrastructure. This approach offers benefits in terms of maintenance and deployment effort compared to external integration in a separate environment (e.g., the radiology department). This proposed groundwork can be taken as an exemplary development guideline for future applications of federated learning networks in clinical and scientific environments.
Collapse
Affiliation(s)
- Paul-Philipp Jacobs
- Department of Diagnostic and Interventional Radiology, University of Leipzig, 04109 Leipzig, Germany
| | - Constantin Ehrengut
- Department of Diagnostic and Interventional Radiology, University of Leipzig, 04109 Leipzig, Germany
| | - Andreas Michael Bucher
- Department of Diagnostic and Interventional Radiology, Johann-Wolfgang-v.-Goethe-Universität, 60629 Frankfurt, Germany
| | - Tobias Penzkofer
- Department of Radiology, Campus Virchow-Klinikum, Charité—Universitätsmedizin Berlin, 10117 Berlin, Germany
| | - Mathias Lukas
- Department of Diagnostic and Interventional Radiology, University of Leipzig, 04109 Leipzig, Germany
| | - Jens Kleesiek
- Institute for Artificial Intelligence in Medicine, University Hospital Essen (AöR), 45131 Essen, Germany
- Medical Faculty, University of Duisburg-Essen, 45122 Essen, Germany
| | - Timm Denecke
- Department of Diagnostic and Interventional Radiology, University of Leipzig, 04109 Leipzig, Germany
| |
Collapse
|
24
|
Hwang HJ, Kim H, Seo JB, Ye JC, Oh G, Lee SM, Jang R, Yun J, Kim N, Park HJ, Lee HY, Yoon SH, Shin KE, Lee JW, Kwon W, Sun JS, You S, Chung MH, Gil BM, Lim JK, Lee Y, Hong SJ, Choi YW. Generative Adversarial Network-Based Image Conversion Among Different Computed Tomography Protocols and Vendors: Effects on Accuracy and Variability in Quantifying Regional Disease Patterns of Interstitial Lung Disease. Korean J Radiol 2023; 24:807-820. [PMID: 37500581 PMCID: PMC10400368 DOI: 10.3348/kjr.2023.0088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 06/12/2023] [Accepted: 06/18/2023] [Indexed: 07/29/2023] Open
Abstract
OBJECTIVE To assess whether computed tomography (CT) conversion across different scan parameters and manufacturers using a routable generative adversarial network (RouteGAN) can improve the accuracy and variability in quantifying interstitial lung disease (ILD) using a deep learning-based automated software. MATERIALS AND METHODS This study included patients with ILD who underwent thin-section CT. Unmatched CT images obtained using scanners from four manufacturers (vendors A-D), standard- or low-radiation doses, and sharp or medium kernels were classified into groups 1-7 according to acquisition conditions. CT images in groups 2-7 were converted into the target CT style (Group 1: vendor A, standard dose, and sharp kernel) using a RouteGAN. ILD was quantified on original and converted CT images using a deep learning-based software (Aview, Coreline Soft). The accuracy of quantification was analyzed using the dice similarity coefficient (DSC) and pixel-wise overlap accuracy metrics against manual quantification by a radiologist. Five radiologists evaluated quantification accuracy using a 10-point visual scoring system. RESULTS Three hundred and fifty CT slices from 150 patients (mean age: 67.6 ± 10.7 years; 56 females) were included. The overlap accuracies for quantifying total abnormalities in groups 2-7 improved after CT conversion (original vs. converted: 0.63 vs. 0.68 for DSC, 0.66 vs. 0.70 for pixel-wise recall, and 0.68 vs. 0.73 for pixel-wise precision; P < 0.002 for all). The DSCs of fibrosis score, honeycombing, and reticulation significantly increased after CT conversion (0.32 vs. 0.64, 0.19 vs. 0.47, and 0.23 vs. 0.54, P < 0.002 for all), whereas those of ground-glass opacity, consolidation, and emphysema did not change significantly or decreased slightly. The radiologists' scores were significantly higher (P < 0.001) and less variable on converted CT. CONCLUSION CT conversion using a RouteGAN can improve the accuracy and variability of CT images obtained using different scan parameters and manufacturers in deep learning-based quantification of ILD.
Collapse
Affiliation(s)
- Hye Jeon Hwang
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Hyunjong Kim
- Robotics Program, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Joon Beom Seo
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| | - Jong Chul Ye
- Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Gyutaek Oh
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Sang Min Lee
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Ryoungwoo Jang
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Jihye Yun
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Namkug Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Hee Jun Park
- Coreline Soft, Co., Ltd, Seoul, Republic of Korea
| | - Ho Yun Lee
- Department of Radiology and Center for Imaging Science, Samsung Medical Center, School of Medicine, Sungkyunkwan University, Seoul, Republic of Korea
- Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul, Republic of Korea
| | - Soon Ho Yoon
- Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Kyung Eun Shin
- Department of Radiology, Soonchunhyang University Bucheon Hospital, Bucheon, Republic of Korea
| | - Jae Wook Lee
- Department of Radiology, Soonchunhyang University Bucheon Hospital, Bucheon, Republic of Korea
| | - Woocheol Kwon
- Department of Radiology, Ewha Womans University Seoul Hospital, Seoul, Republic of Korea
- Department of Radiology, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
| | - Joo Sung Sun
- Department of Radiology, Ajou University School of Medicine, Suwon, Republic of Korea
| | - Seulgi You
- Department of Radiology, Ajou University School of Medicine, Suwon, Republic of Korea
| | - Myung Hee Chung
- Department of Radiology, Bucheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea
| | - Bo Mi Gil
- Department of Radiology, Bucheon St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea
| | - Jae-Kwang Lim
- Department of Radiology, Kyungpook National University School of Medicine, Daegu, Republic of Korea
| | - Youkyung Lee
- Department of Radiology, Hanyang University Guri Hospital, Hanyang University College of Medicine, Guri, Republic of Korea
| | - Su Jin Hong
- Department of Radiology, Hanyang University Guri Hospital, Hanyang University College of Medicine, Guri, Republic of Korea
| | - Yo Won Choi
- Department of Radiology, Hanyang University Seoul Hospital, Hanyang University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
25
|
Azevedo T, Bethlehem RAI, Whiteside DJ, Swaddiwudhipong N, Rowe JB, Lió P, Rittman T. Identifying healthy individuals with Alzheimer's disease neuroimaging phenotypes in the UK Biobank. COMMUNICATIONS MEDICINE 2023; 3:100. [PMID: 37474615 PMCID: PMC10359360 DOI: 10.1038/s43856-023-00313-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Accepted: 06/05/2023] [Indexed: 07/22/2023] Open
Abstract
BACKGROUND Identifying prediagnostic neurodegenerative disease is a critical issue in neurodegenerative disease research, and Alzheimer's disease (AD) in particular, to identify populations suitable for preventive and early disease-modifying trials. Evidence from genetic and other studies suggests the neurodegeneration of Alzheimer's disease measured by brain atrophy starts many years before diagnosis, but it is unclear whether these changes can be used to reliably detect prediagnostic sporadic disease. METHODS We trained a Bayesian machine learning neural network model to generate a neuroimaging phenotype and AD score representing the probability of AD using structural MRI data in the Alzheimer's Disease Neuroimaging Initiative (ADNI) Cohort (cut-off 0.5, AUC 0.92, PPV 0.90, NPV 0.93). We go on to validate the model in an independent real-world dataset of the National Alzheimer's Coordinating Centre (AUC 0.74, PPV 0.65, NPV 0.80) and demonstrate the correlation of the AD-score with cognitive scores in those with an AD-score above 0.5. We then apply the model to a healthy population in the UK Biobank study to identify a cohort at risk for Alzheimer's disease. RESULTS We show that the cohort with a neuroimaging Alzheimer's phenotype has a cognitive profile in keeping with Alzheimer's disease, with strong evidence for poorer fluid intelligence, and some evidence of poorer numeric memory, reaction time, working memory, and prospective memory. We found some evidence in the AD-score positive cohort for modifiable risk factors of hypertension and smoking. CONCLUSIONS This approach demonstrates the feasibility of using AI methods to identify a potentially prediagnostic population at high risk for developing sporadic Alzheimer's disease.
Collapse
Affiliation(s)
- Tiago Azevedo
- Department of Computer Science and Technology, University of Cambridge, Cambridge, UK
| | - Richard A I Bethlehem
- Brain Mapping Unit, Department of Psychiatry, University of Cambridge, Cambridge, UK
- Autism Research Centre, Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - David J Whiteside
- Department of Clinical Neurosciences and Cambridge University Hospitals NHS Trust, University of Cambridge, Cambridge, UK
| | - Nol Swaddiwudhipong
- Department of Clinical Neurosciences and Cambridge University Hospitals NHS Trust, University of Cambridge, Cambridge, UK
| | - James B Rowe
- Department of Clinical Neurosciences and Cambridge University Hospitals NHS Trust, University of Cambridge, Cambridge, UK
| | - Pietro Lió
- Department of Computer Science and Technology, University of Cambridge, Cambridge, UK
| | - Timothy Rittman
- Department of Clinical Neurosciences and Cambridge University Hospitals NHS Trust, University of Cambridge, Cambridge, UK.
| |
Collapse
|
26
|
Hu F, Chen AA, Horng H, Bashyam V, Davatzikos C, Alexander-Bloch A, Li M, Shou H, Satterthwaite TD, Yu M, Shinohara RT. Image harmonization: A review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization. Neuroimage 2023; 274:120125. [PMID: 37084926 PMCID: PMC10257347 DOI: 10.1016/j.neuroimage.2023.120125] [Citation(s) in RCA: 18] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 04/12/2023] [Accepted: 04/19/2023] [Indexed: 04/23/2023] Open
Abstract
Magnetic resonance imaging and computed tomography from multiple batches (e.g. sites, scanners, datasets, etc.) are increasingly used alongside complex downstream analyses to obtain new insights into the human brain. However, significant confounding due to batch-related technical variation, called batch effects, is present in this data; direct application of downstream analyses to the data may lead to biased results. Image harmonization methods seek to remove these batch effects and enable increased generalizability and reproducibility of downstream results. In this review, we describe and categorize current approaches in statistical and deep learning harmonization methods. We also describe current evaluation metrics used to assess harmonization methods and provide a standardized framework to evaluate newly-proposed methods for effective harmonization and preservation of biological information. Finally, we provide recommendations to end-users to advocate for more effective use of current methods and to methodologists to direct future efforts and accelerate development of the field.
Collapse
Affiliation(s)
- Fengling Hu
- Penn Statistics in Imaging and Visualization Endeavor (PennSIVE), Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Dr, Philadelphia, PA 19104, United States.
| | - Andrew A Chen
- Penn Statistics in Imaging and Visualization Endeavor (PennSIVE), Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Dr, Philadelphia, PA 19104, United States
| | - Hannah Horng
- Penn Statistics in Imaging and Visualization Endeavor (PennSIVE), Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Dr, Philadelphia, PA 19104, United States
| | - Vishnu Bashyam
- Center for Biomedical Image Computing and Analytics (CBICA), Perelman School of Medicine, United States
| | - Christos Davatzikos
- Center for Biomedical Image Computing and Analytics (CBICA), Perelman School of Medicine, United States
| | - Aaron Alexander-Bloch
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, United States; Penn-CHOP Lifespan Brain Institute, United States; Department of Child and Adolescent Psychiatry and Behavioral Science, Children's Hospital of Philadelphia, United States
| | - Mingyao Li
- Statistical Center for Single-Cell and Spatial Genomics, Perelman School of Medicine, University of Pennsylvania, United States
| | - Haochang Shou
- Penn Statistics in Imaging and Visualization Endeavor (PennSIVE), Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Dr, Philadelphia, PA 19104, United States; Center for Biomedical Image Computing and Analytics (CBICA), Perelman School of Medicine, United States
| | - Theodore D Satterthwaite
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, United States; Penn-CHOP Lifespan Brain Institute, United States; The Penn Lifespan Informatics and Neuroimaging Center, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, United States
| | - Meichen Yu
- Indiana Alzheimer's Disease Research Center, Indiana University School of Medicine, United States
| | - Russell T Shinohara
- Penn Statistics in Imaging and Visualization Endeavor (PennSIVE), Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Dr, Philadelphia, PA 19104, United States; Center for Biomedical Image Computing and Analytics (CBICA), Perelman School of Medicine, United States
| |
Collapse
|
27
|
Meng M, Li H, Zhang M, He G, Wang L, Shen D. Reducing the number of unnecessary biopsies for mammographic BI-RADS 4 lesions through a deep transfer learning method. BMC Med Imaging 2023; 23:82. [PMID: 37312026 DOI: 10.1186/s12880-023-01023-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 05/23/2023] [Indexed: 06/15/2023] Open
Abstract
BACKGROUND In clinical practice, reducing unnecessary biopsies for mammographic BI-RADS 4 lesions is crucial. The objective of this study was to explore the potential value of deep transfer learning (DTL) based on the different fine-tuning strategies for Inception V3 to reduce the number of unnecessary biopsies that residents need to perform for mammographic BI-RADS 4 lesions. METHODS A total of 1980 patients with breast lesions were included, including 1473 benign lesions (185 women with bilateral breast lesions), and 692 malignant lesions collected and confirmed by clinical pathology or biopsy. The breast mammography images were randomly divided into three subsets, a training set, testing set, and validation set 1, at a ratio of 8:1:1. We constructed a DTL model for the classification of breast lesions based on Inception V3 and attempted to improve its performance with 11 fine-tuning strategies. The mammography images from 362 patients with pathologically confirmed BI-RADS 4 breast lesions were employed as validation set 2. Two images from each lesion were tested, and trials were categorized as correct if the judgement (≥ 1 image) was correct. We used precision (Pr), recall rate (Rc), F1 score (F1), and the area under the receiver operating characteristic curve (AUROC) as the performance metrics of the DTL model with validation set 2. RESULTS The S5 model achieved the best fit for the data. The Pr, Rc, F1 and AUROC of S5 were 0.90, 0.90, 0.90, and 0.86, respectively, for Category 4. The proportions of lesions downgraded by S5 were 90.73%, 84.76%, and 80.19% for categories 4 A, 4B, and 4 C, respectively. The overall proportion of BI-RADS 4 lesions downgraded by S5 was 85.91%. There was no significant difference between the classification results of the S5 model and pathological diagnosis (P = 0.110). CONCLUSION The S5 model we proposed here can be used as an effective approach for reducing the number of unnecessary biopsies that residents need to conduct for mammographic BI-RADS 4 lesions and may have other important clinical uses.
Collapse
Affiliation(s)
- Mingzhu Meng
- Department of Radiology, The Affiliated Changzhou No 2 People's Hospital of Nanjing Medical University, Changzhou, 213164, Jiangsu Province, P. R. China
| | - Hong Li
- Department of Radiology, The Second Affiliated Hospital of Soochow University, Suzhou, 215004, Jiangsu Province, P.R. China
| | - Ming Zhang
- Department of Radiology, The Affiliated Changzhou No 2 People's Hospital of Nanjing Medical University, Changzhou, 213164, Jiangsu Province, P. R. China
| | - Guangyuan He
- Department of Radiology, The Affiliated Changzhou No 2 People's Hospital of Nanjing Medical University, Changzhou, 213164, Jiangsu Province, P. R. China
| | - Long Wang
- Department of Radiology, The Affiliated Changzhou No 2 People's Hospital of Nanjing Medical University, Changzhou, 213164, Jiangsu Province, P. R. China.
| | - Dong Shen
- Department of Radiology, The Affiliated Changzhou No 2 People's Hospital of Nanjing Medical University, Changzhou, 213164, Jiangsu Province, P. R. China.
| |
Collapse
|
28
|
Hindsholm AM, Andersen FL, Cramer SP, Simonsen HJ, Askløf MG, Magyari M, Madsen PN, Hansen AE, Sellebjerg F, Larsson HBW, Langkilde AR, Frederiksen JL, Højgaard L, Ladefoged CN, Lindberg U. Scanner agnostic large-scale evaluation of MS lesion delineation tool for clinical MRI. Front Neurosci 2023; 17:1177540. [PMID: 37274207 PMCID: PMC10235534 DOI: 10.3389/fnins.2023.1177540] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 05/05/2023] [Indexed: 06/06/2023] Open
Abstract
Introduction Patients with MS are MRI scanned continuously throughout their disease course resulting in a large manual workload for radiologists which includes lesion detection and size estimation. Though many models for automatic lesion segmentation have been published, few are used broadly in clinic today, as there is a lack of testing on clinical datasets. By collecting a large, heterogeneous training dataset directly from our MS clinic we aim to present a model which is robust to different scanner protocols and artefacts and which only uses MRI modalities present in routine clinical examinations. Methods We retrospectively included 746 patients from routine examinations at our MS clinic. The inclusion criteria included acquisition at one of seven different scanners and an MRI protocol including 2D or 3D T2-w FLAIR, T2-w and T1-w images. Reference lesion masks on the training (n = 571) and validation (n = 70) datasets were generated using a preliminary segmentation model and subsequent manual correction. The test dataset (n = 100) was manually delineated. Our segmentation model https://github.com/CAAI/AIMS/ was based on the popular nnU-Net, which has won several biomedical segmentation challenges. We tested our model against the published segmentation models HD-MS-Lesions, which is also based on nnU-Net, trained with a more homogenous patient cohort. We furthermore tested model robustness to data from unseen scanners by performing a leave-one-scanner-out experiment. Results We found that our model was able to segment MS white matter lesions with a performance comparable to literature: DSC = 0.68, precision = 0.90, recall = 0.70, f1 = 0.78. Furthermore, the model outperformed HD-MS-Lesions in all metrics except precision = 0.96. In the leave-one-scanner-out experiment there was no significant change in performance (p < 0.05) between any of the models which were only trained on part of the dataset and the full segmentation model. Conclusion In conclusion we have seen, that by including a large, heterogeneous dataset emulating clinical reality, we have trained a segmentation model which maintains a high segmentation performance while being robust to data from unseen scanners. This broadens the applicability of the model in clinic and paves the way for clinical implementation.
Collapse
Affiliation(s)
- Amalie Monberg Hindsholm
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Flemming Littrup Andersen
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Stig Præstekjær Cramer
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Helle Juhl Simonsen
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Mathias Gæde Askløf
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Melinda Magyari
- Department of Neurology, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Poul Nørgaard Madsen
- Center for IT and Medical Technology, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Adam Espe Hansen
- Department of Radiology, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Finn Sellebjerg
- Department of Neurology, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Henrik Bo Wiberg Larsson
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | | | - Jette Lautrup Frederiksen
- Department of Neurology, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Liselotte Højgaard
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Claes Nøhr Ladefoged
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| | - Ulrich Lindberg
- Department of Clinical Physiology and Nuclear Medicine, Copenhagen University Hospital–Rigshospitalet, Copenhagen, Denmark
| |
Collapse
|
29
|
Wen G, Shim V, Holdsworth SJ, Fernandez J, Qiao M, Kasabov N, Wang A. Machine Learning for Brain MRI Data Harmonisation: A Systematic Review. Bioengineering (Basel) 2023; 10:bioengineering10040397. [PMID: 37106584 PMCID: PMC10135601 DOI: 10.3390/bioengineering10040397] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 03/16/2023] [Accepted: 03/21/2023] [Indexed: 04/29/2023] Open
Abstract
BACKGROUND Magnetic Resonance Imaging (MRI) data collected from multiple centres can be heterogeneous due to factors such as the scanner used and the site location. To reduce this heterogeneity, the data needs to be harmonised. In recent years, machine learning (ML) has been used to solve different types of problems related to MRI data, showing great promise. OBJECTIVE This study explores how well various ML algorithms perform in harmonising MRI data, both implicitly and explicitly, by summarising the findings in relevant peer-reviewed articles. Furthermore, it provides guidelines for the use of current methods and identifies potential future research directions. METHOD This review covers articles published through PubMed, Web of Science, and IEEE databases through June 2022. Data from studies were analysed based on the criteria of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Quality assessment questions were derived to assess the quality of the included publications. RESULTS a total of 41 articles published between 2015 and 2022 were identified and analysed. In the review, MRI data has been found to be harmonised either in an implicit (n = 21) or an explicit (n = 20) way. Three MRI modalities were identified: structural MRI (n = 28), diffusion MRI (n = 7) and functional MRI (n = 6). CONCLUSION Various ML techniques have been employed to harmonise different types of MRI data. There is currently a lack of consistent evaluation methods and metrics used across studies, and it is recommended that the issue be addressed in future studies. Harmonisation of MRI data using ML shows promises in improving performance for ML downstream tasks, while caution should be exercised when using ML-harmonised data for direct interpretation.
Collapse
Affiliation(s)
- Grace Wen
- Auckland Bioengineering Institute, University of Auckland, Auckland 1142, New Zealand
| | - Vickie Shim
- Auckland Bioengineering Institute, University of Auckland, Auckland 1142, New Zealand
- Centre for Brain Research, University of Auckland, Auckland 1142, New Zealand
| | - Samantha Jane Holdsworth
- Centre for Brain Research, University of Auckland, Auckland 1142, New Zealand
- Mātai Medical Research Institute, Tairāwhiti-Gisborne 4010, New Zealand
- Department of Anatomy & Medical Imaging, Faculty of Medical and Health Sciences, University of Auckland, Auckland 1142, New Zealand
| | - Justin Fernandez
- Auckland Bioengineering Institute, University of Auckland, Auckland 1142, New Zealand
| | - Miao Qiao
- Department of Computer Science, University of Auckland, Auckland 1142, New Zealand
| | - Nikola Kasabov
- Auckland Bioengineering Institute, University of Auckland, Auckland 1142, New Zealand
- Knowledge Engineering and Discovery Research Institute, Auckland University of Technology, Auckland 1010, New Zealand
- Intelligent Systems Research Centre, Ulster University, Londonderry BT52 1SA, UK
- Institute for Information and Communication Technologies, Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria
| | - Alan Wang
- Auckland Bioengineering Institute, University of Auckland, Auckland 1142, New Zealand
- Centre for Brain Research, University of Auckland, Auckland 1142, New Zealand
- Department of Anatomy & Medical Imaging, Faculty of Medical and Health Sciences, University of Auckland, Auckland 1142, New Zealand
| |
Collapse
|
30
|
Chen Z, Pawar K, Ekanayake M, Pain C, Zhong S, Egan GF. Deep Learning for Image Enhancement and Correction in Magnetic Resonance Imaging-State-of-the-Art and Challenges. J Digit Imaging 2023; 36:204-230. [PMID: 36323914 PMCID: PMC9984670 DOI: 10.1007/s10278-022-00721-9] [Citation(s) in RCA: 18] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 09/09/2022] [Accepted: 10/17/2022] [Indexed: 11/06/2022] Open
Abstract
Magnetic resonance imaging (MRI) provides excellent soft-tissue contrast for clinical diagnoses and research which underpin many recent breakthroughs in medicine and biology. The post-processing of reconstructed MR images is often automated for incorporation into MRI scanners by the manufacturers and increasingly plays a critical role in the final image quality for clinical reporting and interpretation. For image enhancement and correction, the post-processing steps include noise reduction, image artefact correction, and image resolution improvements. With the recent success of deep learning in many research fields, there is great potential to apply deep learning for MR image enhancement, and recent publications have demonstrated promising results. Motivated by the rapidly growing literature in this area, in this review paper, we provide a comprehensive overview of deep learning-based methods for post-processing MR images to enhance image quality and correct image artefacts. We aim to provide researchers in MRI or other research fields, including computer vision and image processing, a literature survey of deep learning approaches for MR image enhancement. We discuss the current limitations of the application of artificial intelligence in MRI and highlight possible directions for future developments. In the era of deep learning, we highlight the importance of a critical appraisal of the explanatory information provided and the generalizability of deep learning algorithms in medical imaging.
Collapse
Affiliation(s)
- Zhaolin Chen
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia.
- Department of Data Science and AI, Monash University, Melbourne, VIC, Australia.
| | - Kamlesh Pawar
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
| | - Mevan Ekanayake
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC, Australia
| | - Cameron Pain
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC, Australia
| | - Shenjun Zhong
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- National Imaging Facility, Brisbane, QLD, Australia
| | - Gary F Egan
- Monash Biomedical Imaging, Monash University, Melbourne, VIC, 3168, Australia
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
31
|
Detecting pediatric wrist fractures using deep-learning-based object detection. Pediatr Radiol 2023; 53:1125-1134. [PMID: 36650360 DOI: 10.1007/s00247-023-05588-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 12/09/2022] [Accepted: 12/30/2022] [Indexed: 01/19/2023]
Abstract
BACKGROUND Missed fractures are the leading cause of diagnostic error in the emergency department, and fractures of pediatric bones, particularly subtle wrist fractures, can be misidentified because of their varying characteristics and responses to injury. OBJECTIVE This study evaluated the utility of an object detection deep learning framework for classifying pediatric wrist fractures as positive or negative for fracture, including subtle buckle fractures of the distal radius, and evaluated the performance of this algorithm as augmentation to trainee radiograph interpretation. MATERIALS AND METHODS We obtained 395 posteroanterior wrist radiographs from unique pediatric patients (65% positive for fracture, 30% positive for distal radial buckle fracture) and divided them into train (n = 229), tune (n = 41) and test (n = 125) sets. We trained a Faster R-CNN (region-based convolutional neural network) deep learning object-detection model. Two pediatric and two radiology residents evaluated radiographs initially without the artificial intelligence (AI) assistance, and then subsequently with access to the bounding box generated by the Faster R-CNN model. RESULTS The Faster R-CNN model demonstrated an area under the curve (AUC) of 0.92 (95% confidence interval [CI] 0.87-0.97), accuracy of 88% (n = 110/125; 95% CI 81-93%), sensitivity of 88% (n = 70/80; 95% CI 78-94%) and specificity of 89% (n = 40/45, 95% CI 76-96%) in identifying any fracture and identified 90% of buckle fractures (n = 35/39, 95% CI 76-97%). Access to Faster R-CNN model predictions significantly improved average resident accuracy from 80 to 93% in detecting any fracture (P < 0.001) and from 69 to 92% in detecting buckle fracture (P < 0.001). After accessing AI predictions, residents significantly outperformed AI in cases of disagreement (73% resident correct vs. 27% AI, P = 0.002). CONCLUSION An object-detection-based deep learning approach trained with only a few hundred examples identified radiographs containing pediatric wrist fractures with high accuracy. Access to model predictions significantly improved resident accuracy in diagnosing these fractures.
Collapse
|
32
|
Pati S, Baid U, Edwards B, Sheller M, Wang SH, Reina GA, Foley P, Gruzdev A, Karkada D, Davatzikos C, Sako C, Ghodasara S, Bilello M, Mohan S, Vollmuth P, Brugnara G, Preetha CJ, Sahm F, Maier-Hein K, Zenk M, Bendszus M, Wick W, Calabrese E, Rudie J, Villanueva-Meyer J, Cha S, Ingalhalikar M, Jadhav M, Pandey U, Saini J, Garrett J, Larson M, Jeraj R, Currie S, Frood R, Fatania K, Huang RY, Chang K, Balaña C, Capellades J, Puig J, Trenkler J, Pichler J, Necker G, Haunschmidt A, Meckel S, Shukla G, Liem S, Alexander GS, Lombardo J, Palmer JD, Flanders AE, Dicker AP, Sair HI, Jones CK, Venkataraman A, Jiang M, So TY, Chen C, Heng PA, Dou Q, Kozubek M, Lux F, Michálek J, Matula P, Keřkovský M, Kopřivová T, Dostál M, Vybíhal V, Vogelbaum MA, Mitchell JR, Farinhas J, Maldjian JA, Yogananda CGB, Pinho MC, Reddy D, Holcomb J, Wagner BC, Ellingson BM, Cloughesy TF, Raymond C, Oughourlian T, Hagiwara A, Wang C, To MS, Bhardwaj S, Chong C, Agzarian M, Falcão AX, Martins SB, Teixeira BCA, Sprenger F, Menotti D, Lucio DR, LaMontagne P, Marcus D, Wiestler B, Kofler F, Ezhov I, Metz M, Jain R, Lee M, Lui YW, McKinley R, Slotboom J, Radojewski P, Meier R, Wiest R, Murcia D, Fu E, Haas R, Thompson J, Ormond DR, Badve C, Sloan AE, Vadmal V, Waite K, Colen RR, Pei L, Ak M, Srinivasan A, Bapuraj JR, Rao A, Wang N, Yoshiaki O, Moritani T, Turk S, Lee J, Prabhudesai S, Morón F, Mandel J, Kamnitsas K, Glocker B, Dixon LVM, Williams M, Zampakis P, Panagiotopoulos V, Tsiganos P, Alexiou S, Haliassos I, Zacharaki EI, Moustakas K, Kalogeropoulou C, Kardamakis DM, Choi YS, Lee SK, Chang JH, Ahn SS, Luo B, Poisson L, Wen N, Tiwari P, Verma R, Bareja R, Yadav I, Chen J, Kumar N, Smits M, van der Voort SR, Alafandi A, Incekara F, Wijnenga MMJ, Kapsas G, Gahrmann R, Schouten JW, Dubbink HJ, Vincent AJPE, van den Bent MJ, French PJ, Klein S, Yuan Y, Sharma S, Tseng TC, Adabi S, Niclou SP, Keunen O, Hau AC, Vallières M, Fortin D, Lepage M, Landman B, Ramadass K, Xu K, Chotai S, Chambless LB, Mistry A, Thompson RC, Gusev Y, Bhuvaneshwar K, Sayah A, Bencheqroun C, Belouali A, Madhavan S, Booth TC, Chelliah A, Modat M, Shuaib H, Dragos C, Abayazeed A, Kolodziej K, Hill M, Abbassy A, Gamal S, Mekhaimar M, Qayati M, Reyes M, Park JE, Yun J, Kim HS, Mahajan A, Muzi M, Benson S, Beets-Tan RGH, Teuwen J, Herrera-Trujillo A, Trujillo M, Escobar W, Abello A, Bernal J, Gómez J, Choi J, Baek S, Kim Y, Ismael H, Allen B, Buatti JM, Kotrotsou A, Li H, Weiss T, Weller M, Bink A, Pouymayou B, Shaykh HF, Saltz J, Prasanna P, Shrestha S, Mani KM, Payne D, Kurc T, Pelaez E, Franco-Maldonado H, Loayza F, Quevedo S, Guevara P, Torche E, Mendoza C, Vera F, Ríos E, López E, Velastin SA, Ogbole G, Soneye M, Oyekunle D, Odafe-Oyibotha O, Osobu B, Shu'aibu M, Dorcas A, Dako F, Simpson AL, Hamghalam M, Peoples JJ, Hu R, Tran A, Cutler D, Moraes FY, Boss MA, Gimpel J, Veettil DK, Schmidt K, Bialecki B, Marella S, Price C, Cimino L, Apgar C, Shah P, Menze B, Barnholtz-Sloan JS, Martin J, Bakas S. Federated learning enables big data for rare cancer boundary detection. Nat Commun 2022; 13:7346. [PMID: 36470898 PMCID: PMC9722782 DOI: 10.1038/s41467-022-33407-5] [Citation(s) in RCA: 59] [Impact Index Per Article: 29.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Accepted: 09/16/2022] [Indexed: 12/12/2022] Open
Abstract
Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.
Collapse
Affiliation(s)
- Sarthak Pati
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Informatics, Technical University of Munich, Munich, Bavaria, Germany
| | - Ujjwal Baid
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | | | | | | | | | | | | | | | - Christos Davatzikos
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Chiharu Sako
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Satyam Ghodasara
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Michel Bilello
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Suyash Mohan
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Philipp Vollmuth
- Department of Neuroradiology, Heidelberg University Hospital, Heidelberg, Germany
| | - Gianluca Brugnara
- Department of Neuroradiology, Heidelberg University Hospital, Heidelberg, Germany
| | | | - Felix Sahm
- Clinical Cooperation Unit Neuropathology, German Cancer Consortium (DKTK) within the German Cancer Research Center (DKFZ), Heidelberg, Germany
- Department of Neuropathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Klaus Maier-Hein
- Division of Medical Image Computing, German Cancer Research Center, Heidelberg, Germany
- Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Maximilian Zenk
- Division of Medical Image Computing, German Cancer Research Center, Heidelberg, Germany
| | - Martin Bendszus
- Department of Neuroradiology, Heidelberg University Hospital, Heidelberg, Germany
| | - Wolfgang Wick
- Clinical Cooperation Unit Neuropathology, German Cancer Consortium (DKTK) within the German Cancer Research Center (DKFZ), Heidelberg, Germany
- Neurology Clinic, Heidelberg University Hospital, Heidelberg, Germany
| | - Evan Calabrese
- Department of Radiology & Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA
| | - Jeffrey Rudie
- Department of Radiology & Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA
| | - Javier Villanueva-Meyer
- Department of Radiology & Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA
| | - Soonmee Cha
- Department of Radiology & Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA
| | - Madhura Ingalhalikar
- Symbiosis Center for Medical Image Analysis, Symbiosis International University, Pune, Maharashtra, India
| | - Manali Jadhav
- Symbiosis Center for Medical Image Analysis, Symbiosis International University, Pune, Maharashtra, India
| | - Umang Pandey
- Symbiosis Center for Medical Image Analysis, Symbiosis International University, Pune, Maharashtra, India
| | - Jitender Saini
- Department of Neuroimaging and Interventional Radiology, National Institute of Mental Health and Neurosciences, Bangalore, Karnataka, India
| | - John Garrett
- Department of Radiology, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
- Department of Medical Physics, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
| | - Matthew Larson
- Department of Radiology, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
| | - Robert Jeraj
- Department of Radiology, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
- Department of Medical Physics, School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
| | - Stuart Currie
- Leeds Teaching Hospitals Trust, Department of Radiology, Leeds, UK
| | - Russell Frood
- Leeds Teaching Hospitals Trust, Department of Radiology, Leeds, UK
| | - Kavi Fatania
- Leeds Teaching Hospitals Trust, Department of Radiology, Leeds, UK
| | - Raymond Y Huang
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Ken Chang
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
| | | | | | - Josep Puig
- Department of Radiology (IDI), Girona Biomedical Research Institute (IdIBGi), Josep Trueta University Hospital, Girona, Spain
| | - Johannes Trenkler
- Institute of Neuroradiology, Neuromed Campus (NMC), Kepler University Hospital Linz, Linz, Austria
| | - Josef Pichler
- Department of Neurooncology, Neuromed Campus (NMC), Kepler University Hospital Linz, Linz, Austria
| | - Georg Necker
- Institute of Neuroradiology, Neuromed Campus (NMC), Kepler University Hospital Linz, Linz, Austria
| | - Andreas Haunschmidt
- Institute of Neuroradiology, Neuromed Campus (NMC), Kepler University Hospital Linz, Linz, Austria
| | - Stephan Meckel
- Institute of Neuroradiology, Neuromed Campus (NMC), Kepler University Hospital Linz, Linz, Austria
- Institute of Diagnostic and Interventional Neuroradiology, RKH Klinikum Ludwigsburg, Ludwigsburg, Germany
| | - Gaurav Shukla
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiation Oncology, Christiana Care Health System, Philadelphia, PA, USA
| | - Spencer Liem
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, PA, USA
| | - Gregory S Alexander
- Department of Radiation Oncology, University of Maryland, Baltimore, MD, USA
| | - Joseph Lombardo
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, PA, USA
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, PA, USA
| | - Joshua D Palmer
- Department of Radiation Oncology, The James Cancer Hospital and Solove Research Institute, The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - Adam E Flanders
- Department of Radiology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, PA, USA
| | - Adam P Dicker
- Department of Radiation Oncology, Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, PA, USA
| | - Haris I Sair
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- The Malone Center for Engineering in Healthcare, The Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Craig K Jones
- The Malone Center for Engineering in Healthcare, The Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Archana Venkataraman
- Department of Electrical and Computer Engineering, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Meirui Jiang
- The Chinese University of Hong Kong, Hong Kong, China
| | - Tiffany Y So
- The Chinese University of Hong Kong, Hong Kong, China
| | - Cheng Chen
- The Chinese University of Hong Kong, Hong Kong, China
| | | | - Qi Dou
- The Chinese University of Hong Kong, Hong Kong, China
| | - Michal Kozubek
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Filip Lux
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Jan Michálek
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Petr Matula
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Miloš Keřkovský
- Department of Radiology and Nuclear Medicine, Faculty of Medicine, Masaryk University, Brno and University Hospital Brno, Brno, Czech Republic
| | - Tereza Kopřivová
- Department of Radiology and Nuclear Medicine, Faculty of Medicine, Masaryk University, Brno and University Hospital Brno, Brno, Czech Republic
| | - Marek Dostál
- Department of Radiology and Nuclear Medicine, Faculty of Medicine, Masaryk University, Brno and University Hospital Brno, Brno, Czech Republic
- Department of Biophysics, Faculty of Medicine, Masaryk University, Brno, Czech Republic
| | - Václav Vybíhal
- Department of Neurosurgery, Faculty of Medicine, Masaryk University, Brno, and University Hospital and Czech Republic, Brno, Czech Republic
| | - Michael A Vogelbaum
- Department of Neuro Oncology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| | - J Ross Mitchell
- University of Alberta, Edmonton, AB, Canada
- Alberta Machine Intelligence Institute, Edmonton, AB, Canada
| | - Joaquim Farinhas
- Department of Radiology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA
| | | | | | - Marco C Pinho
- University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Divya Reddy
- University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - James Holcomb
- University of Texas Southwestern Medical Center, Dallas, TX, USA
| | | | - Benjamin M Ellingson
- UCLA Brain Tumor Imaging Laboratory (BTIL), Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
- UCLA Neuro-Oncology Program, Department of Neurology, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CaA, USA
| | - Timothy F Cloughesy
- UCLA Neuro-Oncology Program, Department of Neurology, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CaA, USA
| | - Catalina Raymond
- UCLA Brain Tumor Imaging Laboratory (BTIL), Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Talia Oughourlian
- UCLA Brain Tumor Imaging Laboratory (BTIL), Center for Computer Vision and Imaging Biomarkers, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
- Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Akifumi Hagiwara
- Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Chencai Wang
- Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, USA
| | - Minh-Son To
- College of Medicine and Public Health, Flinders University, Bedford Park, SA, Australia
- Division of Surgery and Perioperative Medicine, Flinders Medical Centre, Bedford Park, SA, Australia
| | - Sargam Bhardwaj
- College of Medicine and Public Health, Flinders University, Bedford Park, SA, Australia
| | - Chee Chong
- South Australia Medical Imaging, Flinders Medical Centre, Bedford Park, SA, Australia
| | - Marc Agzarian
- South Australia Medical Imaging, Flinders Medical Centre, Bedford Park, SA, Australia
- Department of Neurology, Baylor College of Medicine, Houston, TX, USA
| | | | | | - Bernardo C A Teixeira
- Instituto de Neurologia de Curitiba, Curitiba, Paraná, Brazil
- Department of Radiology, Hospital de Clínicas da Universidade Federal do Paraná, Curitiba, Paraná, Brazil
| | - Flávia Sprenger
- Department of Radiology, Hospital de Clínicas da Universidade Federal do Paraná, Curitiba, Paraná, Brazil
| | - David Menotti
- Department of Informatics, Universidade Federal do Paraná, Curitiba, Paraná, Brazil
| | - Diego R Lucio
- Department of Informatics, Universidade Federal do Paraná, Curitiba, Paraná, Brazil
| | - Pamela LaMontagne
- Department of Radiology, Washington University in St. Louis, St. Louis, MO, USA
| | - Daniel Marcus
- Department of Radiology, Washington University in St. Louis, St. Louis, MO, USA
| | - Benedikt Wiestler
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany
- TranslaTUM (Zentralinstitut für translationale Krebsforschung der Technischen Universität München), Klinikum rechts der Isar, Munich, Germany
| | - Florian Kofler
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany
- TranslaTUM (Zentralinstitut für translationale Krebsforschung der Technischen Universität München), Klinikum rechts der Isar, Munich, Germany
- Image-Based Biomedical Modeling, Department of Informatics, Technical University of Munich, Munich, Germany
| | - Ivan Ezhov
- Department of Informatics, Technical University of Munich, Munich, Bavaria, Germany
- TranslaTUM (Zentralinstitut für translationale Krebsforschung der Technischen Universität München), Klinikum rechts der Isar, Munich, Germany
- Image-Based Biomedical Modeling, Department of Informatics, Technical University of Munich, Munich, Germany
| | - Marie Metz
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany
| | - Rajan Jain
- Department of Radiology, NYU Grossman School of Medicine, New York, NY, USA
- Department of Neurosurgery, NYU Grossman School of Medicine, New York, NY, USA
| | - Matthew Lee
- Department of Radiology, NYU Grossman School of Medicine, New York, NY, USA
| | - Yvonne W Lui
- Department of Radiology, NYU Grossman School of Medicine, New York, NY, USA
| | - Richard McKinley
- Support Center for Advanced Neuroimaging, University Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, Inselspital, University of Bern, Bern, Switzerland
| | - Johannes Slotboom
- Support Center for Advanced Neuroimaging, University Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, Inselspital, University of Bern, Bern, Switzerland
| | - Piotr Radojewski
- Support Center for Advanced Neuroimaging, University Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, Inselspital, University of Bern, Bern, Switzerland
| | - Raphael Meier
- Support Center for Advanced Neuroimaging, University Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, Inselspital, University of Bern, Bern, Switzerland
| | - Roland Wiest
- Support Center for Advanced Neuroimaging, University Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, Inselspital, University of Bern, Bern, Switzerland
| | - Derrick Murcia
- Department of Neurosurgery, Anschutz Medical Campus, University of Colorado, Aurora, CO, USA
| | - Eric Fu
- Department of Neurosurgery, Anschutz Medical Campus, University of Colorado, Aurora, CO, USA
| | - Rourke Haas
- Department of Neurosurgery, Anschutz Medical Campus, University of Colorado, Aurora, CO, USA
| | - John Thompson
- Department of Neurosurgery, Anschutz Medical Campus, University of Colorado, Aurora, CO, USA
| | - David Ryan Ormond
- Department of Neurosurgery, Anschutz Medical Campus, University of Colorado, Aurora, CO, USA
| | - Chaitra Badve
- Department of Radiology, University Hospitals Cleveland, Cleveland, OH, USA
| | - Andrew E Sloan
- Department of Neurological Surgery, University Hospitals-Seidman Cancer Center, Cleveland, OH, USA
- Case Comprehensive Cancer Center, Cleveland, OH, USA
- Department of Neurosurgery, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Vachan Vadmal
- Department of Neurosurgery, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Kristin Waite
- National Cancer Institute, National Institute of Health, Division of Cancer Epidemiology and Genetics, Bethesda, MD, USA
| | - Rivka R Colen
- Department of Radiology, Neuroradiology Division, University of Pittsburgh, Pittsburgh, PA, USA
- Department of Diagnostic Radiology, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Linmin Pei
- University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Murat Ak
- Department of Radiology, Neuroradiology Division, University of Pittsburgh, Pittsburgh, PA, USA
| | - Ashok Srinivasan
- Department of Neuroradiology, University of Michigan, Ann Arbor, MI, USA
| | - J Rajiv Bapuraj
- Department of Neuroradiology, University of Michigan, Ann Arbor, MI, USA
| | - Arvind Rao
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Nicholas Wang
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Ota Yoshiaki
- Department of Neuroradiology, University of Michigan, Ann Arbor, MI, USA
| | - Toshio Moritani
- Department of Neuroradiology, University of Michigan, Ann Arbor, MI, USA
| | - Sevcan Turk
- Department of Neuroradiology, University of Michigan, Ann Arbor, MI, USA
| | - Joonsang Lee
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Snehal Prabhudesai
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Fanny Morón
- Department of Radiology, Baylor College of Medicine, Houston, TX, USA
| | - Jacob Mandel
- Department of Neurology, Baylor College of Medicine, Houston, TX, USA
| | - Konstantinos Kamnitsas
- Department of Computing, Imperial College London, London, UK
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Ben Glocker
- Department of Computing, Imperial College London, London, UK
| | - Luke V M Dixon
- Department of Radiology, Imperial College NHS Healthcare Trust, London, UK
| | - Matthew Williams
- Computational Oncology Group, Institute for Global Health Innovation, Imperial College London, London, UK
| | - Peter Zampakis
- Department of NeuroRadiology, University of Patras, Patras, Greece
| | | | - Panagiotis Tsiganos
- Clinical Radiology Laboratory, Department of Medicine, University of Patras, Patras, Greece
| | - Sotiris Alexiou
- Department of Electrical and Computer Engineering, University of Patras, Patras, Greece
| | - Ilias Haliassos
- Department of Neuro-Oncology, University of Patras, Patras, Greece
| | - Evangelia I Zacharaki
- Department of Electrical and Computer Engineering, University of Patras, Patras, Greece
| | | | | | | | | | | | | | - Sung Soo Ahn
- Yonsei University College of Medicine, Seoul, Korea
| | - Bing Luo
- Department of Radiation Oncology, Henry Ford Health System, Detroit, MI, USA
| | - Laila Poisson
- Public Health Sciences, Henry Ford Health System, Detroit, MI, USA
| | - Ning Wen
- Department of Radiation Oncology, Henry Ford Health System, Detroit, MI, USA
- SJTU-Ruijin-UIH Institute for Medical Imaging Technology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, 200025, Shanghai, China
| | | | - Ruchika Verma
- Alberta Machine Intelligence Institute, Edmonton, AB, Canada
- Case Western Reserve University, Cleveland, OH, USA
| | - Rohan Bareja
- Case Western Reserve University, Cleveland, OH, USA
| | - Ipsa Yadav
- Case Western Reserve University, Cleveland, OH, USA
| | | | - Neeraj Kumar
- University of Alberta, Edmonton, AB, Canada
- Alberta Machine Intelligence Institute, Edmonton, AB, Canada
| | - Marion Smits
- Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Sebastian R van der Voort
- Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Ahmed Alafandi
- Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Fatih Incekara
- Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
- Department of Neurosurgery, Brain Tumor Center, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Maarten M J Wijnenga
- Department of Neurology, Brain Tumor Center, Erasmus MC Cancer Institute, Rotterdam, Netherlands
| | - Georgios Kapsas
- Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Renske Gahrmann
- Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Joost W Schouten
- Department of Neurosurgery, Brain Tumor Center, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Hendrikus J Dubbink
- Department of Pathology, Brain Tumor Center, Erasmus MC Cancer Institute, Rotterdam, Netherlands
| | - Arnaud J P E Vincent
- Department of Neurosurgery, Brain Tumor Center, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Martin J van den Bent
- Department of Neurology, Brain Tumor Center, Erasmus MC Cancer Institute, Rotterdam, Netherlands
| | - Pim J French
- Department of Neurology, Brain Tumor Center, Erasmus MC Cancer Institute, Rotterdam, Netherlands
| | - Stefan Klein
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Centre Rotterdam, Rotterdam, Netherlands
| | - Yading Yuan
- Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Sonam Sharma
- Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Tzu-Chi Tseng
- Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Saba Adabi
- Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Simone P Niclou
- NORLUX Neuro-Oncology Laboratory, Department of Cancer Research, Luxembourg Institute of Health, Luxembourg, Luxembourg
| | - Olivier Keunen
- Translation Radiomics, Department of Cancer Research, Luxembourg Institute of Health, Luxembourg, Luxembourg
| | - Ann-Christin Hau
- NORLUX Neuro-Oncology Laboratory, Department of Cancer Research, Luxembourg Institute of Health, Luxembourg, Luxembourg
- Luxembourg Center of Neuropathology, Laboratoire National De Santé, Luxembourg, Luxembourg
| | - Martin Vallières
- Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC, Canada
- Centre de Recherche du Centre Hospitalière Universitaire de Sherbrooke, Sherbrooke, QC, Canada
| | - David Fortin
- Centre de Recherche du Centre Hospitalière Universitaire de Sherbrooke, Sherbrooke, QC, Canada
- Division of Neurosurgery and Neuro-Oncology, Faculty of Medicine and Health Science, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Martin Lepage
- Centre de Recherche du Centre Hospitalière Universitaire de Sherbrooke, Sherbrooke, QC, Canada
- Department of Nuclear Medicine and Radiobiology, Sherbrooke Molecular Imaging Centre, Université de Sherbrooke, Sherbrooke, QC, Canada
| | - Bennett Landman
- Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Karthik Ramadass
- Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Kaiwen Xu
- Department of Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Silky Chotai
- Department of Neurosurgery, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Lola B Chambless
- Department of Neurosurgery, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Akshitkumar Mistry
- Department of Neurosurgery, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Reid C Thompson
- Department of Neurosurgery, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Yuriy Gusev
- Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC, USA
| | - Krithika Bhuvaneshwar
- Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC, USA
| | - Anousheh Sayah
- Division of Neuroradiology & Neurointerventional Radiology, Department of Radiology, MedStar Georgetown University Hospital, Washington, DC, USA
| | - Camelia Bencheqroun
- Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC, USA
| | - Anas Belouali
- Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC, USA
| | - Subha Madhavan
- Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC, USA
| | - Thomas C Booth
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
- Department of Neuroradiology, Ruskin Wing, King's College Hospital NHS Foundation Trust, London, UK
| | - Alysha Chelliah
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Marc Modat
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Haris Shuaib
- Stoke Mandeville Hospital, Mandeville Road, Aylesbury, UK
- Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada
| | - Carmen Dragos
- Stoke Mandeville Hospital, Mandeville Road, Aylesbury, UK
| | | | | | | | | | - Shady Gamal
- University of Cairo School of Medicine, Giza, Egypt
| | | | | | | | - Ji Eun Park
- Department of Radiology, Asan Medical Center, Seoul, South Korea
| | - Jihye Yun
- Department of Radiology, Asan Medical Center, Seoul, South Korea
| | - Ho Sung Kim
- Department of Radiology, Asan Medical Center, Seoul, South Korea
| | - Abhishek Mahajan
- The Clatterbridge Cancer Centre NHS Foundation Trust Pembroke Place, Liverpool, UK
| | - Mark Muzi
- Department of Radiology, University of Washington, Seattle, WA, USA
| | - Sean Benson
- Netherlands Cancer Institute, Amsterdam, Netherlands
| | - Regina G H Beets-Tan
- Department of Radiology, Netherlands Cancer Institute, Amsterdam, Netherlands
- GROW School of Oncology and Developmental Biology, Maastricht, Netherlands
| | - Jonas Teuwen
- Netherlands Cancer Institute, Amsterdam, Netherlands
| | | | | | - William Escobar
- Clínica Imbanaco Grupo Quirón Salud, Cali, Colombia
- Universidad del Valle, Cali, Colombia
| | | | - Jose Bernal
- Universidad del Valle, Cali, Colombia
- The University of Edinburgh, Edinburgh, UK
| | | | - Joseph Choi
- Department of Industrial and Systems Engineering, University of Iowa, Iowa, USA
| | - Stephen Baek
- Department of Industrial and Systems Engineering, Department of Radiation Oncology, University of Iowa, Iowa City, IA, USA
| | - Yusung Kim
- Department of Radiation Oncology, University of Iowa, Iowa City, IA, USA
| | - Heba Ismael
- Department of Radiation Oncology, University of Iowa, Iowa City, IA, USA
| | - Bryan Allen
- Department of Radiation Oncology, University of Iowa, Iowa City, IA, USA
| | - John M Buatti
- Department of Radiation Oncology, University of Iowa, Iowa City, IA, USA
| | | | - Hongwei Li
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - Tobias Weiss
- Department of Neurology, Clinical Neuroscience Center, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Michael Weller
- Department of Neurology, Clinical Neuroscience Center, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Andrea Bink
- Department of Neuroradiology, Clinical Neuroscience Center, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | - Bertrand Pouymayou
- Department of Neuroradiology, Clinical Neuroscience Center, University Hospital Zurich and University of Zurich, Zurich, Switzerland
| | | | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA
| | - Prateek Prasanna
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA
| | - Sampurna Shrestha
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA
| | - Kartik M Mani
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA
- Department of Radiation Oncology, Stony Brook University, Stony Brook, NY, USA
| | - David Payne
- Department of Radiology, Stony Brook University, Stony Brook, NY, USA
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, New York, USA
- Scientific Data Group, Oak Ridge National Laboratory, Oak Ridge, TN, USA
| | - Enrique Pelaez
- Escuela Superior Politecnica del Litoral, Guayaquil, Guayas, Ecuador
| | | | - Francis Loayza
- Escuela Superior Politecnica del Litoral, Guayaquil, Guayas, Ecuador
| | | | | | | | | | - Franco Vera
- Universidad de Concepción, Concepción, Biobío, Chile
| | - Elvis Ríos
- Universidad de Concepción, Concepción, Biobío, Chile
| | - Eduardo López
- Universidad de Concepción, Concepción, Biobío, Chile
| | - Sergio A Velastin
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK
| | - Godwin Ogbole
- Department of Radiology, University College Hospital Ibadan, Oyo, Nigeria
| | - Mayowa Soneye
- Department of Radiology, University College Hospital Ibadan, Oyo, Nigeria
| | - Dotun Oyekunle
- Department of Radiology, University College Hospital Ibadan, Oyo, Nigeria
| | | | - Babatunde Osobu
- Department of Radiology, University College Hospital Ibadan, Oyo, Nigeria
| | - Mustapha Shu'aibu
- Department of Radiology, Muhammad Abdullahi Wase Teaching Hospital, Kano, Nigeria
| | - Adeleye Dorcas
- Department of Radiology, Obafemi Awolowo University Ile-Ife, Ile-Ife, Osun, Nigeria
| | - Farouk Dako
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Center for Global Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Amber L Simpson
- Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada
- School of Computing, Queen's University, Kingston, ON, Canada
| | - Mohammad Hamghalam
- School of Computing, Queen's University, Kingston, ON, Canada
- Department of Electrical Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran
| | - Jacob J Peoples
- School of Computing, Queen's University, Kingston, ON, Canada
| | - Ricky Hu
- School of Computing, Queen's University, Kingston, ON, Canada
| | - Anh Tran
- School of Computing, Queen's University, Kingston, ON, Canada
| | - Danielle Cutler
- The Faculty of Arts & Sciences, Queen's University, Kingston, ON, Canada
| | - Fabio Y Moraes
- Department of Oncology, Queen's University, Kingston, ON, Canada
| | - Michael A Boss
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | - James Gimpel
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | - Deepak Kattil Veettil
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | - Kendall Schmidt
- Data Science Institute, American College of Radiology, Reston, VA, USA
| | - Brian Bialecki
- Data Science Institute, American College of Radiology, Reston, VA, USA
| | - Sailaja Marella
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | - Cynthia Price
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | - Lisa Cimino
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | - Charles Apgar
- Center for Research and Innovation, American College of Radiology, Philadelphia, PA, USA
| | | | - Bjoern Menze
- Department of Informatics, Technical University of Munich, Munich, Bavaria, Germany
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - Jill S Barnholtz-Sloan
- National Cancer Institute, National Institute of Health, Division of Cancer Epidemiology and Genetics, Bethesda, MD, USA
- Center for Biomedical Informatics and Information Technology, National Cancer Institute (NCI), National Institute of Health, Bethesda, MD, USA
| | | | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA.
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| |
Collapse
|
33
|
Kitaguchi D, Fujino T, Takeshita N, Hasegawa H, Mori K, Ito M. Limited generalizability of single deep neural network for surgical instrument segmentation in different surgical environments. Sci Rep 2022; 12:12575. [PMID: 35869249 PMCID: PMC9307578 DOI: 10.1038/s41598-022-16923-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Accepted: 07/18/2022] [Indexed: 12/05/2022] Open
Abstract
Clarifying the generalizability of deep-learning-based surgical-instrument segmentation networks in diverse surgical environments is important in recognizing the challenges of overfitting in surgical-device development. This study comprehensively evaluated deep neural network generalizability for surgical instrument segmentation using 5238 images randomly extracted from 128 intraoperative videos. The video dataset contained 112 laparoscopic colorectal resection, 5 laparoscopic distal gastrectomy, 5 laparoscopic cholecystectomy, and 6 laparoscopic partial hepatectomy cases. Deep-learning-based surgical-instrument segmentation was performed for test sets with (1) the same conditions as the training set; (2) the same recognition target surgical instrument and surgery type but different laparoscopic recording systems; (3) the same laparoscopic recording system and surgery type but slightly different recognition target laparoscopic surgical forceps; (4) the same laparoscopic recording system and recognition target surgical instrument but different surgery types. The mean average precision and mean intersection over union for test sets 1, 2, 3, and 4 were 0.941 and 0.887, 0.866 and 0.671, 0.772 and 0.676, and 0.588 and 0.395, respectively. Therefore, the recognition accuracy decreased even under slightly different conditions. The results of this study reveal the limited generalizability of deep neural networks in the field of surgical artificial intelligence and caution against deep-learning-based biased datasets and models. Trial Registration Number: 2020-315, date of registration: October 5, 2020.
Collapse
|
34
|
Sharafutdinov K, Bhat JS, Fritsch SJ, Nikulina K, E. Samadi M, Polzin R, Mayer H, Marx G, Bickenbach J, Schuppert A. Application of convex hull analysis for the evaluation of data heterogeneity between patient populations of different origin and implications of hospital bias in downstream machine-learning-based data processing: A comparison of 4 critical-care patient datasets. Front Big Data 2022; 5:603429. [DOI: 10.3389/fdata.2022.603429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2020] [Accepted: 10/03/2022] [Indexed: 11/07/2022] Open
Abstract
Machine learning (ML) models are developed on a learning dataset covering only a small part of the data of interest. If model predictions are accurate for the learning dataset but fail for unseen data then generalization error is considered high. This problem manifests itself within all major sub-fields of ML but is especially relevant in medical applications. Clinical data structures, patient cohorts, and clinical protocols may be highly biased among hospitals such that sampling of representative learning datasets to learn ML models remains a challenge. As ML models exhibit poor predictive performance over data ranges sparsely or not covered by the learning dataset, in this study, we propose a novel method to assess their generalization capability among different hospitals based on the convex hull (CH) overlap between multivariate datasets. To reduce dimensionality effects, we used a two-step approach. First, CH analysis was applied to find mean CH coverage between each of the two datasets, resulting in an upper bound of the prediction range. Second, 4 types of ML models were trained to classify the origin of a dataset (i.e., from which hospital) and to estimate differences in datasets with respect to underlying distributions. To demonstrate the applicability of our method, we used 4 critical-care patient datasets from different hospitals in Germany and USA. We estimated the similarity of these populations and investigated whether ML models developed on one dataset can be reliably applied to another one. We show that the strongest drop in performance was associated with the poor intersection of convex hulls in the corresponding hospitals' datasets and with a high performance of ML methods for dataset discrimination. Hence, we suggest the application of our pipeline as a first tool to assess the transferability of trained models. We emphasize that datasets from different hospitals represent heterogeneous data sources, and the transfer from one database to another should be performed with utmost care to avoid implications during real-world applications of the developed models. Further research is needed to develop methods for the adaptation of ML models to new hospitals. In addition, more work should be aimed at the creation of gold-standard datasets that are large and diverse with data from varied application sites.
Collapse
|
35
|
Foley P, Sheller MJ, Edwards B, Pati S, Riviera W, Sharma M, Narayana Moorthy P, Wang SH, Martin J, Mirhaji P, Shah P, Bakas S. OpenFL: the open federated learning library. Phys Med Biol 2022; 67:214001. [PMID: 36198326 PMCID: PMC9715347 DOI: 10.1088/1361-6560/ac97d9] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 10/05/2022] [Indexed: 11/07/2022]
Abstract
Objective.Federated learning (FL) is a computational paradigm that enables organizations to collaborate on machine learning (ML) and deep learning (DL) projects without sharing sensitive data, such as patient records, financial data, or classified secrets.Approach.Open federated learning (OpenFL) framework is an open-source python-based tool for training ML/DL algorithms using the data-private collaborative learning paradigm of FL, irrespective of the use case. OpenFL works with training pipelines built with both TensorFlow and PyTorch, and can be easily extended to other ML and DL frameworks.Main results.In this manuscript, we present OpenFL and summarize its motivation and development characteristics, with the intention of facilitating its application to existing ML/DL model training in a production environment. We further provide recommendations to secure a federation using trusted execution environments to ensure explicit model security and integrity, as well as maintain data confidentiality. Finally, we describe the first real-world healthcare federations that use the OpenFL library, and highlight how it can be applied to other non-healthcare use cases.Significance.The OpenFL library is designed for real world scalability, trusted execution, and also prioritizes easy migration of centralized ML models into a federated training pipeline. Although OpenFL's initial use case was in healthcare, it is applicable beyond this domain and is now reaching wider adoption both in research and production settings. The tool is open-sourced atgithub.com/intel/openfl.
Collapse
Affiliation(s)
- Patrick Foley
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Micah J Sheller
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Brandon Edwards
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Sarthak Pati
- University of Pennsylvania, 3700 Hamilton Walk, Richards Medical Research Laboratories (7th Fl), Philadelphia, PA 19104, United States of America
| | - Walter Riviera
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Mansi Sharma
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | | | - Shih-Han Wang
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Jason Martin
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Parsa Mirhaji
- Albert Einstein College of Medicine, 1300 Morris Park Ave, Bronx, NY 10461, United States of America
| | - Prashant Shah
- Intel Corporation, Santa Clara, CA 95052, United States of America
| | - Spyridon Bakas
- University of Pennsylvania, 3700 Hamilton Walk, Richards Medical Research Laboratories (7th Fl), Philadelphia, PA 19104, United States of America
| |
Collapse
|
36
|
Mallon DH, McNamara CD, Rahmani GS, O'Regan DP, Amiras DG. Automated detection of enteric tubes misplaced in the respiratory tract on chest radiographs using deep learning with two centre validation. Clin Radiol 2022; 77:e758-e764. [PMID: 35850868 DOI: 10.1016/j.crad.2022.06.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 05/25/2022] [Accepted: 06/17/2022] [Indexed: 11/27/2022]
Abstract
AIM To develop and test a model based on a convolutional neural network that can identify enteric tube position accurately on chest radiography. MATERIALS AND METHODS The chest radiographs of adult patients were classified by radiologists based on enteric tube position as either critically misplaced (within the respiratory tract) or not critically misplaced (misplaced within the oesophagus or safely positioned below the diaphragm). A deep-learning model based on the 121-layer DenseNet architecture was developed using a training and validation set of 4,693 chest radiographs. The model was evaluated on an external test data set from a separate institution that consisted of 1,514 consecutive radiographs with a real-world incidence of critically misplaced enteric tubes. RESULTS The receiver operator characteristic area under the curve was 0.90 and 0.92 for the internal validation and external test data sets, respectively. For the external data set with a prevalence of 4.4% of critically misplaced enteric tubes, the model achieved high accuracy (92%), sensitivity (80%), and specificity (92%) for identifying a critically misplaced enteric tube. The negative predictive value (99%) was higher than the positive predictive value (32%). CONCLUSION The present study describes the development and external testing of a model that accurately identifies an enteric tube misplaced within the respiratory tract. This model could help reduce the risk of the catastrophic consequences of feeding through a misplaced enteric tube.
Collapse
Affiliation(s)
- D H Mallon
- Imperial College Healthcare NHS Trust, London, UK; MRC London Institute of Medical Sciences, Imperial College London, London, UK.
| | - C D McNamara
- Imperial College Healthcare NHS Trust, London, UK
| | - G S Rahmani
- Galway University Hospitals, Galway, Ireland
| | - D P O'Regan
- Imperial College Healthcare NHS Trust, London, UK; MRC London Institute of Medical Sciences, Imperial College London, London, UK
| | - D G Amiras
- Imperial College Healthcare NHS Trust, London, UK
| |
Collapse
|
37
|
Garrucho L, Kushibar K, Jouide S, Diaz O, Igual L, Lekadir K. Domain generalization in deep learning based mass detection in mammography: A large-scale multi-center study. Artif Intell Med 2022; 132:102386. [PMID: 36207090 DOI: 10.1016/j.artmed.2022.102386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 08/07/2022] [Accepted: 08/19/2022] [Indexed: 11/02/2022]
Abstract
Computer-aided detection systems based on deep learning have shown great potential in breast cancer detection. However, the lack of domain generalization of artificial neural networks is an important obstacle to their deployment in changing clinical environments. In this study, we explored the domain generalization of deep learning methods for mass detection in digital mammography and analyzed in-depth the sources of domain shift in a large-scale multi-center setting. To this end, we compared the performance of eight state-of-the-art detection methods, including Transformer based models, trained in a single domain and tested in five unseen domains. Moreover, a single-source mass detection training pipeline was designed to improve the domain generalization without requiring images from the new domain. The results show that our workflow generalized better than state-of-the-art transfer learning based approaches in four out of five domains while reducing the domain shift caused by the different acquisition protocols and scanner manufacturers. Subsequently, an extensive analysis was performed to identify the covariate shifts with the greatest effects on detection performance, such as those due to differences in patient age, breast density, mass size, and mass malignancy. Ultimately, this comprehensive study provides key insights and best practices for future research on domain generalization in deep learning based breast cancer detection.
Collapse
Affiliation(s)
- Lidia Garrucho
- Artificial Intelligence in Medicine Lab (BCN-AIM), Faculty of Mathematics and Computer Science, University of Barcelona, Gran Via de les Corts Catalanes 585, Barcelona, 08007, Barcelona, Spain.
| | - Kaisar Kushibar
- Artificial Intelligence in Medicine Lab (BCN-AIM), Faculty of Mathematics and Computer Science, University of Barcelona, Gran Via de les Corts Catalanes 585, Barcelona, 08007, Barcelona, Spain
| | - Socayna Jouide
- Artificial Intelligence in Medicine Lab (BCN-AIM), Faculty of Mathematics and Computer Science, University of Barcelona, Gran Via de les Corts Catalanes 585, Barcelona, 08007, Barcelona, Spain
| | - Oliver Diaz
- Artificial Intelligence in Medicine Lab (BCN-AIM), Faculty of Mathematics and Computer Science, University of Barcelona, Gran Via de les Corts Catalanes 585, Barcelona, 08007, Barcelona, Spain
| | - Laura Igual
- Artificial Intelligence in Medicine Lab (BCN-AIM), Faculty of Mathematics and Computer Science, University of Barcelona, Gran Via de les Corts Catalanes 585, Barcelona, 08007, Barcelona, Spain
| | - Karim Lekadir
- Artificial Intelligence in Medicine Lab (BCN-AIM), Faculty of Mathematics and Computer Science, University of Barcelona, Gran Via de les Corts Catalanes 585, Barcelona, 08007, Barcelona, Spain
| |
Collapse
|
38
|
Song YH, Yi JY, Noh Y, Jang H, Seo SW, Na DL, Seong JK. On the reliability of deep learning-based classification for Alzheimer's disease: Multi-cohorts, multi-vendors, multi-protocols, and head-to-head validation. Front Neurosci 2022; 16:851871. [PMID: 36161156 PMCID: PMC9490270 DOI: 10.3389/fnins.2022.851871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 07/27/2022] [Indexed: 01/18/2023] Open
Abstract
Structural changes in the brain due to Alzheimer's disease dementia (ADD) can be observed through brain T1-weighted magnetic resonance imaging (MRI) images. Many ADD diagnostic studies using brain MRI images have been conducted with machine-learning and deep-learning models. Although reliability is a key in clinical application and applicability of low-resolution MRI (LRMRI) is a key to broad clinical application, both are not sufficiently studied in the deep-learning area. In this study, we developed a 2-dimensional convolutional neural network-based classification model by adopting several methods, such as using instance normalization layer, Mixup, and sharpness aware minimization. To train the model, MRI images from 2,765 cognitively normal individuals and 1,192 patients with ADD from the Samsung medical center cohort were exploited. To assess the reliability of our classification model, we designed external validation in multiple scenarios: (1) multi-cohort validation using four additional cohort datasets including more than 30 different centers in multiple countries, (2) multi-vendor validation using three different MRI vendor subgroups, (3) LRMRI image validation, and finally, (4) head-to-head validation using ten pairs of MRI images from ten individual subjects scanned in two different centers. For multi-cohort validation, we used the MRI images from 739 subjects from the Alzheimer's Disease Neuroimaging Initiative cohort, 125 subjects from the Dementia Platform of Korea cohort, 234 subjects from the Premier cohort, and 139 subjects from the Gachon University Gil Medical Center. We further assessed classification performance across different vendors and protocols for each dataset. We achieved a mean AUC and classification accuracy of 0.9868 and 0.9482 in 5-fold cross-validation. In external validation, we obtained a comparable AUC of 0.9396 and classification accuracy of 0.8757 to other cross-validation studies in the ADNI cohorts. Furthermore, we observed the possibility of broad clinical application through LRMRI image validation by achieving a mean AUC and classification accuracy of 0.9404 and 0.8765 at cross-validation and AUC and classification accuracy of 0.8749 and 0.8281 at the ADNI cohort external validation.
Collapse
Affiliation(s)
- Yeong-Hun Song
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| | - Jun-Young Yi
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
| | - Young Noh
- Department of Neurology, Gil Medical Center, Gachon University College of Medicine, Incheon, South Korea
| | - Hyemin Jang
- Department of Neurology, Samsung Medical Center, School of Medicine, Sungkyunkwan University, Seoul, South Korea
| | - Sang Won Seo
- Department of Neurology, Samsung Medical Center, School of Medicine, Sungkyunkwan University, Seoul, South Korea
| | - Duk L. Na
- Department of Neurology, Samsung Medical Center, School of Medicine, Sungkyunkwan University, Seoul, South Korea
| | - Joon-Kyung Seong
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
- School of Biomedical Engineering, Korea University, Seoul, South Korea
- Interdisciplinary Program in Precision Public Health, College of Health Science, Korea University, Seoul, South Korea
| |
Collapse
|
39
|
Kamraoui RA, Mansencal B, Manjon JV, Coupé P. Longitudinal detection of new MS lesions using deep learning. FRONTIERS IN NEUROIMAGING 2022; 1:948235. [PMID: 37555158 PMCID: PMC10406205 DOI: 10.3389/fnimg.2022.948235] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 07/11/2022] [Indexed: 08/10/2023]
Abstract
The detection of new multiple sclerosis (MS) lesions is an important marker of the evolution of the disease. The applicability of learning-based methods could automate this task efficiently. However, the lack of annotated longitudinal data with new-appearing lesions is a limiting factor for the training of robust and generalizing models. In this study, we describe a deep-learning-based pipeline addressing the challenging task of detecting and segmenting new MS lesions. First, we propose to use transfer-learning from a model trained on a segmentation task using single time-points. Therefore, we exploit knowledge from an easier task and for which more annotated datasets are available. Second, we propose a data synthesis strategy to generate realistic longitudinal time-points with new lesions using single time-point scans. In this way, we pretrain our detection model on large synthetic annotated datasets. Finally, we use a data-augmentation technique designed to simulate data diversity in MRI. By doing that, we increase the size of the available small annotated longitudinal datasets. Our ablation study showed that each contribution lead to an enhancement of the segmentation accuracy. Using the proposed pipeline, we obtained the best score for the segmentation and the detection of new MS lesions in the MSSEG2 MICCAI challenge.
Collapse
Affiliation(s)
| | - Boris Mansencal
- PICTURA, Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, Talence, France
| | - José V. Manjon
- ITACA, Universitat Politècnica de València, Valencia, Spain
| | - Pierrick Coupé
- PICTURA, Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, Talence, France
| |
Collapse
|
40
|
Bressem KK, Adams LC, Proft F, Hermann KGA, Diekhoff T, Spiller L, Niehues SM, Makowski MR, Hamm B, Protopopov M, Rios Rodriguez V, Haibel H, Rademacher J, Torgutalp M, Lambert RG, Baraliakos X, Maksymowych WP, Vahldiek JL, Poddubnyy D. Deep Learning Detects Changes Indicative of Axial Spondyloarthritis at MRI of Sacroiliac Joints. Radiology 2022; 305:655-665. [PMID: 35943339 DOI: 10.1148/radiol.212526] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Background MRI is frequently used for early diagnosis of axial spondyloarthritis (axSpA). However, evaluation is time-consuming and requires profound expertise because noninflammatory degenerative changes can mimic axSpA, and early signs may therefore be missed. Deep neural networks could function as assistance for axSpA detection. Purpose To create a deep neural network to detect MRI changes in sacroiliac joints indicative of axSpA. Materials and Methods This retrospective multicenter study included MRI examinations of five cohorts of patients with clinical suspicion of axSpA collected at university and community hospitals between January 2006 and September 2020. Data from four cohorts were used as the training set, and data from one cohort as the external test set. Each MRI examination in the training and test sets was scored by six and seven raters, respectively, for inflammatory changes (bone marrow edema, enthesitis) and structural changes (erosions, sclerosis). A deep learning tool to detect changes indicative of axSpA was developed. First, a neural network to homogenize the images, then a classification network were trained. Performance was evaluated with use of area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. P < .05 was considered indicative of statistically significant difference. Results Overall, 593 patients (mean age, 37 years ± 11 [SD]; 302 women) were studied. Inflammatory and structural changes were found in 197 of 477 patients (41%) and 244 of 477 (51%), respectively, in the training set and 25 of 116 patients (22%) and 26 of 116 (22%) in the test set. The AUCs were 0.94 (95% CI: 0.84, 0.97) for all inflammatory changes, 0.88 (95% CI: 0.80, 0.95) for inflammatory changes fulfilling the Assessment of SpondyloArthritis international Society definition, and 0.89 (95% CI: 0.81, 0.96) for structural changes indicative of axSpA. Sensitivity and specificity on the external test set were 22 of 25 patients (88%) and 65 of 91 patients (71%), respectively, for inflammatory changes and 22 of 26 patients (85%) and 70 of 90 patients (78%) for structural changes. Conclusion Deep neural networks can detect inflammatory or structural changes to the sacroiliac joint indicative of axial spondyloarthritis at MRI. © RSNA, 2022 Online supplemental material is available for this article.
Collapse
Affiliation(s)
- Keno K Bressem
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Lisa C Adams
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Fabian Proft
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Kay Geert A Hermann
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Torsten Diekhoff
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Laura Spiller
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Stefan M Niehues
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Marcus R Makowski
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Bernd Hamm
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Mikhail Protopopov
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Valeria Rios Rodriguez
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Hildurn Haibel
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Judith Rademacher
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Murat Torgutalp
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Robert G Lambert
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Xenofon Baraliakos
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Walter P Maksymowych
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Janis L Vahldiek
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| | - Denis Poddubnyy
- From the Institute for Radiology (K.K.B., L.C.A., K.G.A.H., T.D., S.M.N., B.H., J.L.V.) and Department of Gastroenterology, Infectious Diseases and Rheumatology (including Nutrition Medicine) (F.P., L.S., M.P., V.R.R., H.H., J.R., M.T., D.P.), Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany; Berlin Institute of Health at Charité-Universitätsmedizin Berlin, Berlin, Germany (K.K.B., L.C.A., J.R.); Department of Diagnostic and Interventional Radiology, Faculty of Medicine, Technical University of Munich, Munich, Germany (M.R.M.); Department of Medicine, University of Alberta, Edmonton, Alberta, Canada (R.G.L., W.P.M.); Rheumazentrum Ruhrgebiet Herne, Ruhr University Bochum, Germany (X.B.); and Epidemiology Unit, German Rheumatism Research Centre, Berlin, Germany (D.P.)
| |
Collapse
|
41
|
Zhou Y, Wagner SK, Chia MA, Zhao A, Woodward-Court P, Xu M, Struyven R, Alexander DC, Keane PA. AutoMorph: Automated Retinal Vascular Morphology Quantification Via a Deep Learning Pipeline. Transl Vis Sci Technol 2022; 11:12. [PMID: 35833885 PMCID: PMC9290317 DOI: 10.1167/tvst.11.7.12] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Purpose To externally validate a deep learning pipeline (AutoMorph) for automated analysis of retinal vascular morphology on fundus photographs. AutoMorph has been made publicly available, facilitating widespread research in ophthalmic and systemic diseases. Methods AutoMorph consists of four functional modules: image preprocessing, image quality grading, anatomical segmentation (including binary vessel, artery/vein, and optic disc/cup segmentation), and vascular morphology feature measurement. Image quality grading and anatomical segmentation use the most recent deep learning techniques. We employ a model ensemble strategy to achieve robust results and analyze the prediction confidence to rectify false gradable cases in image quality grading. We externally validate the performance of each module on several independent publicly available datasets. Results The EfficientNet-b4 architecture used in the image grading module achieves performance comparable to that of the state of the art for EyePACS-Q, with an F1-score of 0.86. The confidence analysis reduces the number of images incorrectly assessed as gradable by 76%. Binary vessel segmentation achieves an F1-score of 0.73 on AV-WIDE and 0.78 on DR HAGIS. Artery/vein scores are 0.66 on IOSTAR-AV, and disc segmentation achieves 0.94 in IDRID. Vascular morphology features measured from the AutoMorph segmentation map and expert annotation show good to excellent agreement. Conclusions AutoMorph modules perform well even when external validation data show domain differences from training data (e.g., with different imaging devices). This fully automated pipeline can thus allow detailed, efficient, and comprehensive analysis of retinal vascular morphology on color fundus photographs. Translational Relevance By making AutoMorph publicly available and open source, we hope to facilitate ophthalmic and systemic disease research, particularly in the emerging field of oculomics.
Collapse
Affiliation(s)
- Yukun Zhou
- Centre for Medical Image Computing, University College London, London, UK.,NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK.,Department of Medical Physics and Biomedical Engineering, University College London, London, UK
| | - Siegfried K Wagner
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Mark A Chia
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - An Zhao
- Centre for Medical Image Computing, University College London, London, UK.,Department of Computer Science, University College London, London, UK
| | - Peter Woodward-Court
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK.,Institute of Health Informatics, University College London, London, UK
| | - Moucheng Xu
- Centre for Medical Image Computing, University College London, London, UK.,Department of Medical Physics and Biomedical Engineering, University College London, London, UK
| | - Robbert Struyven
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK.,Department of Medical Physics and Biomedical Engineering, University College London, London, UK
| | - Daniel C Alexander
- Centre for Medical Image Computing, University College London, London, UK.,Department of Computer Science, University College London, London, UK
| | - Pearse A Keane
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| |
Collapse
|
42
|
Nan Y, Ser JD, Walsh S, Schönlieb C, Roberts M, Selby I, Howard K, Owen J, Neville J, Guiot J, Ernst B, Pastor A, Alberich-Bayarri A, Menzel MI, Walsh S, Vos W, Flerin N, Charbonnier JP, van Rikxoort E, Chatterjee A, Woodruff H, Lambin P, Cerdá-Alberich L, Martí-Bonmatí L, Herrera F, Yang G. Data harmonisation for information fusion in digital healthcare: A state-of-the-art systematic review, meta-analysis and future research directions. AN INTERNATIONAL JOURNAL ON INFORMATION FUSION 2022; 82:99-122. [PMID: 35664012 PMCID: PMC8878813 DOI: 10.1016/j.inffus.2022.01.001] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2021] [Revised: 12/22/2021] [Accepted: 01/07/2022] [Indexed: 05/13/2023]
Abstract
Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.
Collapse
Affiliation(s)
- Yang Nan
- National Heart and Lung Institute, Imperial College London, London, Northern Ireland UK
| | - Javier Del Ser
- Department of Communications Engineering, University of the Basque Country UPV/EHU, Bilbao 48013, Spain
- TECNALIA, Basque Research and Technology Alliance (BRTA), Derio 48160, Spain
| | - Simon Walsh
- National Heart and Lung Institute, Imperial College London, London, Northern Ireland UK
| | - Carola Schönlieb
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, Northern Ireland UK
| | - Michael Roberts
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, Northern Ireland UK
- Oncology R&D, AstraZeneca, Cambridge, Northern Ireland UK
| | - Ian Selby
- Department of Radiology, University of Cambridge, Cambridge, Northern Ireland UK
| | - Kit Howard
- Clinical Data Interchange Standards Consortium, Austin, TX, United States of America
| | - John Owen
- Clinical Data Interchange Standards Consortium, Austin, TX, United States of America
| | - Jon Neville
- Clinical Data Interchange Standards Consortium, Austin, TX, United States of America
| | - Julien Guiot
- University Hospital of Liège (CHU Liège), Respiratory medicine department, Liège, Belgium
- University of Liege, Department of clinical sciences, Pneumology-Allergology, Liège, Belgium
| | - Benoit Ernst
- University Hospital of Liège (CHU Liège), Respiratory medicine department, Liège, Belgium
- University of Liege, Department of clinical sciences, Pneumology-Allergology, Liège, Belgium
| | | | | | - Marion I. Menzel
- Technische Hochschule Ingolstadt, Ingolstadt, Germany
- GE Healthcare GmbH, Munich, Germany
| | - Sean Walsh
- Radiomics (Oncoradiomics SA), Liège, Belgium
| | - Wim Vos
- Radiomics (Oncoradiomics SA), Liège, Belgium
| | - Nina Flerin
- Radiomics (Oncoradiomics SA), Liège, Belgium
| | | | | | - Avishek Chatterjee
- Department of Precision Medicine, Maastricht University, Maastricht, The Netherlands
| | - Henry Woodruff
- Department of Precision Medicine, Maastricht University, Maastricht, The Netherlands
| | - Philippe Lambin
- Department of Precision Medicine, Maastricht University, Maastricht, The Netherlands
| | - Leonor Cerdá-Alberich
- Medical Imaging Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain
| | - Luis Martí-Bonmatí
- Medical Imaging Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain
| | - Francisco Herrera
- Department of Computer Sciences and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada, Granada, Spain
- Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Guang Yang
- National Heart and Lung Institute, Imperial College London, London, Northern Ireland UK
- Cardiovascular Research Centre, Royal Brompton Hospital, London, Northern Ireland UK
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, Northern Ireland UK
| |
Collapse
|
43
|
De Stefano N, Battaglini M, Pareto D, Cortese R, Zhang J, Oesingmann N, Prados F, Rocca MA, Valsasina P, Vrenken H, Gandini Wheeler-Kingshott CAM, Filippi M, Barkhof F, Rovira À. MAGNIMS recommendations for harmonization of MRI data in MS multicenter studies. Neuroimage Clin 2022; 34:102972. [PMID: 35245791 PMCID: PMC8892169 DOI: 10.1016/j.nicl.2022.102972] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Revised: 02/22/2022] [Accepted: 02/23/2022] [Indexed: 11/24/2022]
Abstract
Sharing data from cooperative studies is essential to develop new biomarkers in MS. Differences in MRI acquisition, analysis, storage represent a substantial constraint. We review the state of the art and developments in the harmonization of MRI. We provide recommendations to harmonize large MRI datasets in the MS field.
There is an increasing need of sharing harmonized data from large, cooperative studies as this is essential to develop new diagnostic and prognostic biomarkers. In the field of multiple sclerosis (MS), the issue has become of paramount importance due to the need to translate into the clinical setting some of the most recent MRI achievements. However, differences in MRI acquisition parameters, image analysis and data storage across sites, with their potential bias, represent a substantial constraint. This review focuses on the state of the art, recent technical advances, and desirable future developments of the harmonization of acquisition, analysis and storage of large-scale multicentre MRI data of MS cohorts. Huge efforts are currently being made to achieve all the requirements needed to provide harmonized MRI datasets in the MS field, as proper management of large imaging datasets is one of our greatest opportunities and challenges in the coming years. Recommendations based on these achievements will be provided here. Despite the advances that have been made, the complexity of these tasks requires further research by specialized academical centres, with dedicated technical and human resources. Such collective efforts involving different professional figures are of crucial importance to offer to MS patients a personalised management while minimizing consumption of resources.
Collapse
Affiliation(s)
- Nicola De Stefano
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy.
| | - Marco Battaglini
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy
| | - Deborah Pareto
- Section of Neuroradiology, Department of Radiology, Hospital Universitari Vall d'Hebron, Universitat Autònoma de Barcelona, Barcelona, Spain
| | - Rosa Cortese
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy
| | - Jian Zhang
- Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy
| | | | - Ferran Prados
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom; Center for Medical Imaging Computing, Medical Physics and Biomedical Engineering, UCL, London, WC1V 6LJ, United Kingdom; e-Health Center, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Maria A Rocca
- Neuroimaging Research Unit, Division of Neuroscience, and Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
| | - Paola Valsasina
- Neuroimaging Research Unit, Division of Neuroscience, and Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Hugo Vrenken
- Amsterdam Neuroscience, MS Center Amsterdam, Department of Radiology and Nuclear Medicine, Amsterdam UMC, Amsterdam, Netherlands
| | - Claudia A M Gandini Wheeler-Kingshott
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom; Brain MRI 3T Research Center, C. Mondino National Neurological Institute, Pavia, Italy; Department of Brain and Behavioural Sciences, University of Pavia, Pavia, Italy
| | - Massimo Filippi
- Neuroimaging Research Unit, Division of Neuroscience, and Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Neurology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy; Neurorehabilitation Unit, and Neurophysiology Service, IRCCS San Raffaele Scientific Institute, Milan, Italy; Neurophysiology Service, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Frederik Barkhof
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom; Center for Medical Imaging Computing, Medical Physics and Biomedical Engineering, UCL, London, WC1V 6LJ, United Kingdom; Amsterdam Neuroscience, MS Center Amsterdam, Department of Radiology and Nuclear Medicine, Amsterdam UMC, Amsterdam, Netherlands
| | - Àlex Rovira
- Section of Neuroradiology, Department of Radiology, Hospital Universitari Vall d'Hebron, Universitat Autònoma de Barcelona, Barcelona, Spain
| |
Collapse
|
44
|
Danilevicz MF, Gill M, Anderson R, Batley J, Bennamoun M, Bayer PE, Edwards D. Plant Genotype to Phenotype Prediction Using Machine Learning. Front Genet 2022; 13:822173. [PMID: 35664329 PMCID: PMC9159391 DOI: 10.3389/fgene.2022.822173] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Accepted: 03/07/2022] [Indexed: 12/13/2022] Open
Abstract
Genomic prediction tools support crop breeding based on statistical methods, such as the genomic best linear unbiased prediction (GBLUP). However, these tools are not designed to capture non-linear relationships within multi-dimensional datasets, or deal with high dimension datasets such as imagery collected by unmanned aerial vehicles. Machine learning (ML) algorithms have the potential to surpass the prediction accuracy of current tools used for genotype to phenotype prediction, due to their capacity to autonomously extract data features and represent their relationships at multiple levels of abstraction. This review addresses the challenges of applying statistical and machine learning methods for predicting phenotypic traits based on genetic markers, environment data, and imagery for crop breeding. We present the advantages and disadvantages of explainable model structures, discuss the potential of machine learning models for genotype to phenotype prediction in crop breeding, and the challenges, including the scarcity of high-quality datasets, inconsistent metadata annotation and the requirements of ML models.
Collapse
Affiliation(s)
- Monica F. Danilevicz
- School of Biological Sciences and Institute of Agriculture, University of Western Australia, Perth, WA, Australia
| | - Mitchell Gill
- School of Biological Sciences and Institute of Agriculture, University of Western Australia, Perth, WA, Australia
| | - Robyn Anderson
- School of Biological Sciences and Institute of Agriculture, University of Western Australia, Perth, WA, Australia
| | - Jacqueline Batley
- School of Biological Sciences and Institute of Agriculture, University of Western Australia, Perth, WA, Australia
| | - Mohammed Bennamoun
- School of Physics, Mathematics and Computing, University of Western Australia, Perth, WA, Australia
| | - Philipp E. Bayer
- School of Biological Sciences and Institute of Agriculture, University of Western Australia, Perth, WA, Australia
| | - David Edwards
- School of Biological Sciences and Institute of Agriculture, University of Western Australia, Perth, WA, Australia
- *Correspondence: David Edwards,
| |
Collapse
|
45
|
Braeker N, Schmitz C, Wagner N, Stanicki BJ, Schröder C, Ehret F, Fürweger C, Zwahlen DR, Förster R, Muacevic A, Windisch P. Classifying the Acquisition Sequence for Brain MRIs Using Neural Networks on Single Slices. Cureus 2022; 14:e22435. [PMID: 35345703 PMCID: PMC8941825 DOI: 10.7759/cureus.22435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/21/2022] [Indexed: 11/13/2022] Open
Abstract
Background Neural networks for analyzing MRIs are oftentimes trained on particular combinations of perspectives and acquisition sequences. Since real-world data are less structured and do not follow a standard denomination of acquisition sequences, this impedes the transition from deep learning research to clinical application. The purpose of this study is therefore to assess the feasibility of classifying the acquisition sequence from a single MRI slice using convolutional neural networks. Methods A total of 113 MRI slices from 52 patients were used in a transfer learning approach to train three convolutional neural networks of different complexities to predict the acquisition sequence, while 27 slices were used for internal validation. The model then underwent external validation on 600 slices from 273 patients belonging to one of four classes (T1-weighted without contrast enhancement, T1-weighted with contrast enhancement, T2-weighted, and diffusion-weighted). Categorical accuracy was noted, and the results of the predictions for the validation set are provided with confusion matrices. Results The neural networks achieved a categorical accuracy of 0.79, 0.81, and 0.84 on the external validation data. The implementation of Grad-CAM showed no clear pattern of focus except for T2-weighted slices, where the network focused on areas containing cerebrospinal fluid. Conclusion Automatically classifying the acquisition sequence using neural networks seems feasible and could be used to facilitate the automatic labelling of MRI data.
Collapse
|
46
|
Srikrishna M, Heckemann RA, Pereira JB, Volpe G, Zettergren A, Kern S, Westman E, Skoog I, Schöll M. Comparison of Two-Dimensional- and Three-Dimensional-Based U-Net Architectures for Brain Tissue Classification in One-Dimensional Brain CT. Front Comput Neurosci 2022; 15:785244. [PMID: 35082608 PMCID: PMC8784554 DOI: 10.3389/fncom.2021.785244] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 12/02/2021] [Indexed: 11/13/2022] Open
Abstract
Brain tissue segmentation plays a crucial role in feature extraction, volumetric quantification, and morphometric analysis of brain scans. For the assessment of brain structure and integrity, CT is a non-invasive, cheaper, faster, and more widely available modality than MRI. However, the clinical application of CT is mostly limited to the visual assessment of brain integrity and exclusion of copathologies. We have previously developed two-dimensional (2D) deep learning-based segmentation networks that successfully classified brain tissue in head CT. Recently, deep learning-based MRI segmentation models successfully use patch-based three-dimensional (3D) segmentation networks. In this study, we aimed to develop patch-based 3D segmentation networks for CT brain tissue classification. Furthermore, we aimed to compare the performance of 2D- and 3D-based segmentation networks to perform brain tissue classification in anisotropic CT scans. For this purpose, we developed 2D and 3D U-Net-based deep learning models that were trained and validated on MR-derived segmentations from scans of 744 participants of the Gothenburg H70 Cohort with both CT and T1-weighted MRI scans acquired timely close to each other. Segmentation performance of both 2D and 3D models was evaluated on 234 unseen datasets using measures of distance, spatial similarity, and tissue volume. Single-task slice-wise processed 2D U-Nets performed better than multitask patch-based 3D U-Nets in CT brain tissue classification. These findings provide support to the use of 2D U-Nets to segment brain tissue in one-dimensional (1D) CT. This could increase the application of CT to detect brain abnormalities in clinical settings.
Collapse
Affiliation(s)
- Meera Srikrishna
- Wallenberg Centre for Molecular and Translational Medicine, University of Gothenburg, Gothenburg, Sweden
- Department of Psychiatry and Neurochemistry, Institute of Physiology and Neuroscience, University of Gothenburg, Gothenburg, Sweden
| | - Rolf A. Heckemann
- Department of Medical Radiation Sciences, Institute of Clinical Sciences, Sahlgrenska Academy, Gothenburg, Sweden
| | - Joana B. Pereira
- Division of Clinical Geriatrics, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
- Memory Research Unit, Department of Clinical Sciences, Malmö Lund University, Mälmo, Sweden
| | - Giovanni Volpe
- Department of Physics, University of Gothenburg, Gothenburg, Sweden
| | - Anna Zettergren
- Neuropsychiatric Epidemiology, Institute of Neuroscience and Physiology, Sahlgrenska Academy, Centre for Ageing and Health (AgeCap), University of Gothenburg, Gothenburg, Sweden
| | - Silke Kern
- Neuropsychiatric Epidemiology, Institute of Neuroscience and Physiology, Sahlgrenska Academy, Centre for Ageing and Health (AgeCap), University of Gothenburg, Gothenburg, Sweden
- Region Västra Götaland, Sahlgrenska University Hospital, Psychiatry, Cognition and Old Age Psychiatry Clinic, Gothenburg, Sweden
| | - Eric Westman
- Division of Clinical Geriatrics, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| | - Ingmar Skoog
- Neuropsychiatric Epidemiology, Institute of Neuroscience and Physiology, Sahlgrenska Academy, Centre for Ageing and Health (AgeCap), University of Gothenburg, Gothenburg, Sweden
- Region Västra Götaland, Sahlgrenska University Hospital, Psychiatry, Cognition and Old Age Psychiatry Clinic, Gothenburg, Sweden
| | - Michael Schöll
- Wallenberg Centre for Molecular and Translational Medicine, University of Gothenburg, Gothenburg, Sweden
- Department of Psychiatry and Neurochemistry, Institute of Physiology and Neuroscience, University of Gothenburg, Gothenburg, Sweden
- Dementia Research Centre, Institute of Neurology, University College London, London, United Kingdom
- Department of Clinical Physiology, Sahlgrenska University Hospital, Gothenburg, Sweden
- *Correspondence: Michael Schöll
| |
Collapse
|
47
|
Kamraoui RA, Ta VT, Tourdias T, Mansencal B, Manjon JV, Coupé P. DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation. Med Image Anal 2021; 76:102312. [PMID: 34894571 DOI: 10.1016/j.media.2021.102312] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 10/18/2021] [Accepted: 11/22/2021] [Indexed: 11/29/2022]
Abstract
Recently, segmentation methods based on Convolutional Neural Networks (CNNs) showed promising performance in automatic Multiple Sclerosis (MS) lesions segmentation. These techniques have even outperformed human experts in controlled evaluation conditions such as Longitudinal MS Lesion Segmentation Challenge (ISBI Challenge). However, state-of-the-art approaches trained to perform well on highly-controlled datasets fail to generalize on clinical data from unseen datasets. Instead of proposing another improvement of the segmentation accuracy, we propose a novel method robust to domain shift and performing well on unseen datasets, called DeepLesionBrain (DLB). This generalization property results from three main contributions. First, DLB is based on a large group of compact 3D CNNs. This spatially distributed strategy aims to produce a robust prediction despite the risk of generalization failure of some individual networks. Second, we propose a hierarchical specialization learning (HSL) by pre-training a generic network over the whole brain, before using its weights as initialization to locally specialized networks. By this end, DLB learns both generic features extracted at global image level and specific features extracted at local image level. Finally, DLB includes a new image quality data augmentation to reduce dependency to training data specificity (e.g., acquisition protocol). DLB generalization was validated in cross-dataset experiments on MSSEG'16, ISBI challenge, and in-house datasets. During experiments, DLB showed higher segmentation accuracy, better segmentation consistency and greater generalization performance compared to state-of-the-art methods. Therefore, DLB offers a robust framework well-suited for clinical practice.
Collapse
Affiliation(s)
| | - Vinh-Thong Ta
- Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, PICTURA, Talence F-33400, France
| | - Thomas Tourdias
- Service de Neuroimagerie Diagnostique et Thérapeutique, Univ. Bordeaux, Bordeaux F-33000, France; Univ. Bordeaux, INSERM, Neurocentre Magendie, U1215, Bordeaux F-3300, France
| | - Boris Mansencal
- Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, PICTURA, Talence F-33400, France
| | - José V Manjon
- ITACA, Universitat Politécnica de Valéncia, Valencia 46022, Spain
| | - Pierrick Coupé
- Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, PICTURA, Talence F-33400, France
| |
Collapse
|
48
|
He B, Yin D, Chen X, Luo H, Xiao D, He M, Wang G, Fang C, Liu L, Jia F. A study of generalization and compatibility performance of 3D U-Net segmentation on multiple heterogeneous liver CT datasets. BMC Med Imaging 2021; 21:178. [PMID: 34819022 PMCID: PMC8611902 DOI: 10.1186/s12880-021-00708-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Accepted: 11/15/2021] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND Most existing algorithms have been focused on the segmentation from several public Liver CT datasets scanned regularly (no pneumoperitoneum and horizontal supine position). This study primarily segmented datasets with unconventional liver shapes and intensities deduced by contrast phases, irregular scanning conditions, different scanning objects of pigs and patients with large pathological tumors, which formed the multiple heterogeneity of datasets used in this study. METHODS The multiple heterogeneous datasets used in this paper includes: (1) One public contrast-enhanced CT dataset and one public non-contrast CT dataset; (2) A contrast-enhanced dataset that has abnormal liver shape with very long left liver lobes and large-sized liver tumors with abnormal presets deduced by microvascular invasion; (3) One artificial pneumoperitoneum dataset under the pneumoperitoneum and three scanning profiles (horizontal/left/right recumbent position); (4) Two porcine datasets of Bama type and domestic type that contains pneumoperitoneum cases but with large anatomy discrepancy with humans. The study aimed to investigate the segmentation performances of 3D U-Net in: (1) generalization ability between multiple heterogeneous datasets by cross-testing experiments; (2) the compatibility when hybrid training all datasets in different sampling and encoder layer sharing schema. We further investigated the compatibility of encoder level by setting separate level for each dataset (i.e., dataset-wise convolutions) while sharing the decoder. RESULTS Model trained on different datasets has different segmentation performance. The prediction accuracy between LiTS dataset and Zhujiang dataset was about 0.955 and 0.958 which shows their good generalization ability due to that they were all contrast-enhanced clinical patient datasets scanned regularly. For the datasets scanned under pneumoperitoneum, their corresponding datasets scanned without pneumoperitoneum showed good generalization ability. Dataset-wise convolution module in high-level can improve the dataset unbalance problem. The experimental results will facilitate researchers making solutions when segmenting those special datasets. CONCLUSIONS (1) Regularly scanned datasets is well generalized to irregularly ones. (2) The hybrid training is beneficial but the dataset imbalance problem always exits due to the multi-domain homogeneity. The higher levels encoded more domain specific information than lower levels and thus were less compatible in terms of our datasets.
Collapse
Affiliation(s)
- Baochun He
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
| | - Dalong Yin
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, Harbin Medical University, Harbin, China
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, University of Science and Technology of China, Hefei, China
| | - Xiaoxia Chen
- Department of Radiology, The Third Medical Center, General Hospital of PLA, Beijing, China
| | - Huoling Luo
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
| | - Deqiang Xiao
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China
| | - Mu He
- First Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Guisheng Wang
- Department of Radiology, The Third Medical Center, General Hospital of PLA, Beijing, China
| | - Chihua Fang
- First Hepatobiliary Surgery, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Lianxin Liu
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, Harbin Medical University, Harbin, China.
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, University of Science and Technology of China, Hefei, China.
| | - Fucang Jia
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
- Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences, Shenzhen, China.
- Pazhou Lab, Guangzhou, China.
| |
Collapse
|
49
|
Yala A, Mikhael PG, Strand F, Lin G, Satuluru S, Kim T, Banerjee I, Gichoya J, Trivedi H, Lehman CD, Hughes K, Sheedy DJ, Matthis LM, Karunakaran B, Hegarty KE, Sabino S, Silva TB, Evangelista MC, Caron RF, Souza B, Mauad EC, Patalon T, Handelman-Gotlib S, Guindy M, Barzilay R. Multi-Institutional Validation of a Mammography-Based Breast Cancer Risk Model. J Clin Oncol 2021; 40:1732-1740. [PMID: 34767469 DOI: 10.1200/jco.21.01337] [Citation(s) in RCA: 71] [Impact Index Per Article: 23.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
PURPOSE Accurate risk assessment is essential for the success of population screening programs in breast cancer. Models with high sensitivity and specificity would enable programs to target more elaborate screening efforts to high-risk populations, while minimizing overtreatment for the rest. Artificial intelligence (AI)-based risk models have demonstrated a significant advance over risk models used today in clinical practice. However, the responsible deployment of novel AI requires careful validation across diverse populations. To this end, we validate our AI-based model, Mirai, across globally diverse screening populations. METHODS We collected screening mammograms and pathology-confirmed breast cancer outcomes from Massachusetts General Hospital, USA; Novant, USA; Emory, USA; Maccabi-Assuta, Israel; Karolinska, Sweden; Chang Gung Memorial Hospital, Taiwan; and Barretos, Brazil. We evaluated Uno's concordance-index for Mirai in predicting risk of breast cancer at one to five years from the mammogram. RESULTS A total of 128,793 mammograms from 62,185 patients were collected across the seven sites, of which 3,815 were followed by a cancer diagnosis within 5 years. Mirai obtained concordance indices of 0.75 (95% CI, 0.72 to 0.78), 0.75 (95% CI, 0.70 to 0.80), 0.77 (95% CI, 0.75 to 0.79), 0.77 (95% CI, 0.73 to 0.81), 0.81 (95% CI, 0.79 to 0.82), 0.79 (95% CI, 0.76 to 0.83), and 0.84 (95% CI, 0.81 to 0.88) at Massachusetts General Hospital, Novant, Emory, Maccabi-Assuta, Karolinska, Chang Gung Memorial Hospital, and Barretos, respectively. CONCLUSION Mirai, a mammography-based risk model, maintained its accuracy across globally diverse test sets from seven hospitals across five countries. This is the broadest validation to date of an AI-based breast cancer model and suggests that the technology can offer broad and equitable improvements in care.
Collapse
Affiliation(s)
- Adam Yala
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA.,Jameel Clinic, Massachusetts Institute of Technology, Cambridge, MA
| | - Peter G Mikhael
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA.,Jameel Clinic, Massachusetts Institute of Technology, Cambridge, MA
| | - Fredrik Strand
- Breast Radiology Unit, Department of Imaging and Physiology, Karolinska University Hospital, Stockholm, Sweden.,Department of Oncology-Pathology, Karolinska Institute, Stockholm, Sweden
| | - Gigin Lin
- Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
| | - Siddharth Satuluru
- Department of Computer Science, University of California Los Angeles, Los Angeles, CA
| | - Thomas Kim
- Department of Computer Science, Georgia Institute of Technology, Atlanta, GA
| | - Imon Banerjee
- Department of Biomedical Informatics, Emory University, Atlanta, GA
| | - Judy Gichoya
- Department of Radiology, Emory University, Atlanta, GA
| | - Hari Trivedi
- Department of Radiology, Emory University, Atlanta, GA
| | - Constance D Lehman
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
| | - Kevin Hughes
- Division of Surgical Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, MA
| | - David J Sheedy
- Analytics and Informatics Department, Novant Health, Winston-Salem, NC
| | - Lisa M Matthis
- Analytics and Informatics Department, Novant Health, Winston-Salem, NC
| | - Bipin Karunakaran
- Analytics and Informatics Department, Novant Health, Winston-Salem, NC
| | - Karen E Hegarty
- Digital Product and Services, Novant Health, Winston-Salem, NC
| | - Silvia Sabino
- Department of Cancer Prevention, Barretos Cancer Hospital, Barretos, Brazil
| | - Thiago B Silva
- Department of Cancer Prevention, Barretos Cancer Hospital, Barretos, Brazil
| | | | - Renato F Caron
- Department of Cancer Prevention, Barretos Cancer Hospital, Barretos, Brazil
| | - Bruno Souza
- Department of Cancer Prevention, Barretos Cancer Hospital, Barretos, Brazil
| | - Edmundo C Mauad
- Department of Cancer Prevention, Barretos Cancer Hospital, Barretos, Brazil
| | - Tal Patalon
- Maccabitech, Maccabi Health Services, Tel Aviv, Israel
| | | | - Michal Guindy
- Department of Imaging, Assuta Medical Centers, Tel Aviv, Israel
| | - Regina Barzilay
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA.,Jameel Clinic, Massachusetts Institute of Technology, Cambridge, MA
| |
Collapse
|
50
|
Oala L, Murchison AG, Balachandran P, Choudhary S, Fehr J, Leite AW, Goldschmidt PG, Johner C, Schörverth EDM, Nakasi R, Meyer M, Cabitza F, Baird P, Prabhu C, Weicken E, Liu X, Wenzel M, Vogler S, Akogo D, Alsalamah S, Kazim E, Koshiyama A, Piechottka S, Macpherson S, Shadforth I, Geierhofer R, Matek C, Krois J, Sanguinetti B, Arentz M, Bielik P, Calderon-Ramirez S, Abbood A, Langer N, Haufe S, Kherif F, Pujari S, Samek W, Wiegand T. Machine Learning for Health: Algorithm Auditing & Quality Control. J Med Syst 2021; 45:105. [PMID: 34729675 PMCID: PMC8562935 DOI: 10.1007/s10916-021-01783-y] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Accepted: 10/11/2021] [Indexed: 01/26/2023]
Abstract
Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editorial, we give a summary of ongoing work towards that vision and announce a call for participation to the special issue Machine Learning for Health: Algorithm Auditing & Quality Control in this journal to advance the practice of ML4H auditing.
Collapse
Affiliation(s)
| | | | | | | | - Jana Fehr
- Hasso-Plattner-Institute of Digital Engineering, Potsdam, Germany
| | - Alixandro Werneck Leite
- Machine Learning Laboratory in Finance and Organizations, Universidade de Brasília, Brasília, Brazil
| | | | | | | | | | | | | | | | | | | | - Xiaoxuan Liu
- University Hospitals Birmingham NHS Foundation Trust & Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
| | | | | | | | - Shada Alsalamah
- Information Systems Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia
- Digital Health and Innovation Department, Science Division, World Health Organization, Winterthur, Switzerland
| | - Emre Kazim
- University College London, London, United Kingdom
| | | | | | | | | | | | | | - Joachim Krois
- Oral Diagnostics Digital Health Health Services Research, Charité-Universitätsmedizin, Berlin, Germany
| | | | - Matthew Arentz
- Department of Global Health, University of Washington, Washington, USA
| | | | | | | | - Nicolas Langer
- Department of Psychology, University of Zurich, Zürich, Switzerland
| | | | - Ferath Kherif
- Laboratory for Research in Neuroimaging, Department of Clinical Neuroscience, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Sameer Pujari
- Digital Health and Innovation Department, Science Division, World Health Organization, Winterthur, Switzerland
| | | | | |
Collapse
|