1
|
Bartnik A, Singh S, Sum C, Smith M, Bergsland N, Zivadinov R, Dwyer MG. An Automated Tool to Classify and Transform Unstructured MRI Data into BIDS Datasets. Neuroinformatics 2024; 22:229-238. [PMID: 38530566 DOI: 10.1007/s12021-024-09659-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/07/2024] [Indexed: 03/28/2024]
Abstract
The increasing use of neuroimaging in clinical research has driven the creation of many large imaging datasets. However, these datasets often rely on inconsistent naming conventions in image file headers to describe acquisition, and time-consuming manual curation is necessary. Therefore, we sought to automate the process of classifying and organizing magnetic resonance imaging (MRI) data according to acquisition types common to the clinical routine, as well as automate the transformation of raw, unstructured images into Brain Imaging Data Structure (BIDS) datasets. To do this, we trained an XGBoost model to classify MRI acquisition types using relatively few acquisition parameters that are automatically stored by the MRI scanner in image file metadata, which are then mapped to the naming conventions prescribed by BIDS to transform the input images to the BIDS structure. The model recognizes MRI types with 99.475% accuracy, as well as a micro/macro-averaged precision of 0.9995/0.994, a micro/macro-averaged recall of 0.9995/0.989, and a micro/macro-averaged F1 of 0.9995/0.991. Our approach accurately and quickly classifies MRI types and transforms unstructured data into standardized structures with little-to-no user intervention, reducing the barrier of entry for clinical scientists and increasing the accessibility of existing neuroimaging data.
Collapse
Affiliation(s)
- Alexander Bartnik
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA
| | - Sujal Singh
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA
| | - Conan Sum
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA
| | - Mackenzie Smith
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA
| | - Niels Bergsland
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA
| | - Robert Zivadinov
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA
| | - Michael G Dwyer
- Buffalo Neuroimaging Analysis Center, Department of Neurology, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell St, Buffalo, NY, 14203, USA.
| |
Collapse
|
2
|
Guo X, Gichoya JW, Trivedi H, Purkayastha S, Banerjee I. MedShift: Automated Identification of Shift Data for Medical Image Dataset Curation. IEEE J Biomed Health Inform 2023; 27:3936-3947. [PMID: 37167055 PMCID: PMC10513895 DOI: 10.1109/jbhi.2023.3275104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
Automated curation of noisy external data in the medical domain has long been in high demand, as AI technologies need to be validated using various sources with clean, annotated data. Identifying the variance between internal and external sources is a fundamental step in curating a high-quality dataset, as the data distributions from different sources can vary significantly and subsequently affect the performance of AI models. The primary challenges for detecting data shifts are - (1) accessing private data across healthcare institutions for manual detection and (2) the lack of automated approaches to learn efficient shift-data representation without training samples. To overcome these problems, we propose an automated pipeline called MedShift to detect top-level shift samples and evaluate the significance of shift data without sharing data between internal and external organizations. MedShift employs unsupervised anomaly detectors to learn the internal distribution and identify samples showing significant shiftness for external datasets, and then compares their performance. To quantify the effects of detected shift data, we train a multi-class classifier that learns internal domain knowledge and evaluates the classification performance for each class in external domains after dropping the shift data. We also propose a data quality metric to quantify the dissimilarity between internal and external datasets. We verify the efficacy of MedShift using musculoskeletal radiographs (MURA) and chest X-ray datasets from multiple external sources. Our experiments show that our proposed shift data detection pipeline can be beneficial for medical centers to curate high-quality datasets more efficiently.
Collapse
|
3
|
Chakrabarty S, Abidi SA, Mousa M, Mokkarala M, Hren I, Yadav D, Kelsey M, LaMontagne P, Wood J, Adams M, Su Y, Thorpe S, Chung C, Sotiras A, Marcus DS. Integrative Imaging Informatics for Cancer Research: Workflow Automation for Neuro-Oncology (I3CR-WANO). JCO Clin Cancer Inform 2023; 7:e2200177. [PMID: 37146265 PMCID: PMC10281444 DOI: 10.1200/cci.22.00177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 01/25/2023] [Accepted: 03/06/2023] [Indexed: 05/07/2023] Open
Abstract
PURPOSE Efforts to use growing volumes of clinical imaging data to generate tumor evaluations continue to require significant manual data wrangling, owing to data heterogeneity. Here, we propose an artificial intelligence-based solution for the aggregation and processing of multisequence neuro-oncology MRI data to extract quantitative tumor measurements. MATERIALS AND METHODS Our end-to-end framework (1) classifies MRI sequences using an ensemble classifier, (2) preprocesses the data in a reproducible manner, (3) delineates tumor tissue subtypes using convolutional neural networks, and (4) extracts diverse radiomic features. Moreover, it is robust to missing sequences and adopts an expert-in-the-loop approach in which the segmentation results may be manually refined by radiologists. After the implementation of the framework in Docker containers, it was applied to two retrospective glioma data sets collected from the Washington University School of Medicine (WUSM; n = 384) and The University of Texas MD Anderson Cancer Center (MDA; n = 30), comprising preoperative MRI scans from patients with pathologically confirmed gliomas. RESULTS The scan-type classifier yielded an accuracy of >99%, correctly identifying sequences from 380 of 384 and 30 of 30 sessions from the WUSM and MDA data sets, respectively. Segmentation performance was quantified using the Dice Similarity Coefficient between the predicted and expert-refined tumor masks. The mean Dice scores were 0.882 (±0.244) and 0.977 (±0.04) for whole-tumor segmentation for WUSM and MDA, respectively. CONCLUSION This streamlined framework automatically curated, processed, and segmented raw MRI data of patients with varying grades of gliomas, enabling the curation of large-scale neuro-oncology data sets and demonstrating high potential for integration as an assistive tool in clinical practice.
Collapse
Affiliation(s)
- Satrajit Chakrabarty
- Department of Electrical and Systems Engineering, Washington University in St Louis, St Louis, MO
| | - Syed Amaan Abidi
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
| | - Mina Mousa
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
| | - Mahati Mokkarala
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
| | - Isabelle Hren
- Department of Computer Science & Engineering, Washington University in St Louis, St Louis, MO
| | - Divya Yadav
- Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Matthew Kelsey
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
| | - Pamela LaMontagne
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
| | - John Wood
- Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Michael Adams
- Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Yuzhuo Su
- Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Sherry Thorpe
- Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Caroline Chung
- Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Aristeidis Sotiras
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
- Institute for Informatics, Washington University School of Medicine, St Louis, MO
| | - Daniel S. Marcus
- Mallinckrodt Institute of Radiology, Washington University School of Medicine, St Louis, MO
| |
Collapse
|
4
|
Lim RP, Kachel S, Villa ADM, Kearney L, Bettencourt N, Young AA, Chiribiri A, Scannell CM. CardiSort: a convolutional neural network for cross vendor automated sorting of cardiac MR images. Eur Radiol 2022; 32:5907-5920. [PMID: 35368227 PMCID: PMC9381634 DOI: 10.1007/s00330-022-08724-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2022] [Revised: 01/22/2022] [Accepted: 03/05/2022] [Indexed: 01/19/2023]
Abstract
OBJECTIVES To develop an image-based automatic deep learning method to classify cardiac MR images by sequence type and imaging plane for improved clinical post-processing efficiency. METHODS Multivendor cardiac MRI studies were retrospectively collected from 4 centres and 3 vendors. A two-head convolutional neural network ('CardiSort') was trained to classify 35 sequences by imaging sequence (n = 17) and plane (n = 10). Single vendor training (SVT) on single-centre images (n = 234 patients) and multivendor training (MVT) with multicentre images (n = 434 patients, 3 centres) were performed. Model accuracy and F1 scores on a hold-out test set were calculated, with ground truth labels by an expert radiologist. External validation of MVT (MVTexternal) was performed on data from 3 previously unseen magnet systems from 2 vendors (n = 80 patients). RESULTS Model sequence/plane/overall accuracy and F1-scores were 85.2%/93.2%/81.8% and 0.82 for SVT and 96.1%/97.9%/94.3% and 0.94 MVT on the hold-out test set. MVTexternal yielded sequence/plane/combined accuracy and F1-scores of 92.7%/93.0%/86.6% and 0.86. There was high accuracy for common sequences and conventional cardiac planes. Poor accuracy was observed for underrepresented classes and sequences where there was greater variability in acquisition parameters across centres, such as perfusion imaging. CONCLUSIONS A deep learning network was developed on multivendor data to classify MRI studies into component sequences and planes, with external validation. With refinement, it has potential to improve workflow by enabling automated sequence selection, an important first step in completely automated post-processing pipelines. KEY POINTS • Deep learning can be applied for consistent and efficient classification of cardiac MR image types. • A multicentre, multivendor study using a deep learning algorithm (CardiSort) showed high classification accuracy on a hold-out test set with good generalisation to images from previously unseen magnet systems. • CardiSort has potential to improve clinical workflows, as a vital first step in developing fully automated post-processing pipelines.
Collapse
Affiliation(s)
- Ruth P Lim
- Austin Health, Melbourne, Australia.
- Departments of Radiology, The University of Melbourne, Melbourne, Australia.
- Department of Surgery (Austin), The University of Melbourne, Melbourne, Australia.
| | - Stefan Kachel
- Austin Health, Melbourne, Australia
- Departments of Radiology, The University of Melbourne, Melbourne, Australia
- Department of Radiology, Columbia University, New York, USA
| | - Adriana D M Villa
- School of Biomedical Engineering and Imaging Sciences, Kings College London, London, UK
| | - Leighton Kearney
- Austin Health, Melbourne, Australia
- I-MED Radiology, Melbourne, Australia
| | | | - Alistair A Young
- School of Biomedical Engineering and Imaging Sciences, Kings College London, London, UK
| | - Amedeo Chiribiri
- School of Biomedical Engineering and Imaging Sciences, Kings College London, London, UK
| | - Cian M Scannell
- School of Biomedical Engineering and Imaging Sciences, Kings College London, London, UK
| |
Collapse
|
5
|
Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset. Sci Rep 2021; 11:8366. [PMID: 33863957 PMCID: PMC8052417 DOI: 10.1038/s41598-021-87762-2] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Accepted: 03/22/2021] [Indexed: 01/16/2023] Open
Abstract
The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in a large chest X-ray dataset. We characterized the effectiveness of data Shapley in identifying low quality versus valuable data for pneumonia detection. We found that removing training data with high Shapley values decreased the pneumonia detection performance, whereas removing data with low Shapley values improved the model performance. Furthermore, there were more mislabeled examples in low Shapley value data and more true pneumonia cases in high Shapley value data. Our results suggest that low Shapley value indicates mislabeled or poor quality images, whereas high Shapley value indicates data that are valuable for pneumonia detection. Our method can serve as a framework for using data Shapley to denoise large-scale medical imaging datasets.
Collapse
|
6
|
Leiner T, Bennink E, Mol CP, Kuijf HJ, Veldhuis WB. Bringing AI to the clinic: blueprint for a vendor-neutral AI deployment infrastructure. Insights Imaging 2021; 12:11. [PMID: 33528677 PMCID: PMC7855120 DOI: 10.1186/s13244-020-00931-1] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Accepted: 10/16/2020] [Indexed: 12/30/2022] Open
Abstract
AI provides tremendous opportunities for improving patient care, but at present there is little evidence of real-world uptake. An important barrier is the lack of well-designed, vendor-neutral and future-proof infrastructures for deployment. Because current AI algorithms are very narrow in scope, it is expected that a typical hospital will deploy many algorithms concurrently. Managing stand-alone point solutions for all of these algorithms will be unmanageable. A solution to this problem is a dedicated platform for deployment of AI. Here we describe a blueprint for such a platform and the high-level design and implementation considerations of such a system that can be used clinically as well as for research and development. Close collaboration between radiologists, data scientists, software developers and experts in hospital IT as well as involvement of patients is crucial in order to successfully bring AI to the clinic.
Collapse
Affiliation(s)
- Tim Leiner
- Department of Radiology
- E.01.132, Utrecht University Medical Center, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands.
| | - Edwin Bennink
- Image Sciences Institute, Utrecht University Medical Center, Utrecht, The Netherlands
| | - Christian P Mol
- Image Sciences Institute, Utrecht University Medical Center, Utrecht, The Netherlands
| | - Hugo J Kuijf
- Image Sciences Institute, Utrecht University Medical Center, Utrecht, The Netherlands
| | - Wouter B Veldhuis
- Department of Radiology
- E.01.132, Utrecht University Medical Center, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| |
Collapse
|
7
|
van der Voort SR, Smits M, Klein S. DeepDicomSort: An Automatic Sorting Algorithm for Brain Magnetic Resonance Imaging Data. Neuroinformatics 2021; 19:159-184. [PMID: 32627144 PMCID: PMC7782469 DOI: 10.1007/s12021-020-09475-7] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
With the increasing size of datasets used in medical imaging research, the need for automated data curation is arising. One important data curation task is the structured organization of a dataset for preserving integrity and ensuring reusability. Therefore, we investigated whether this data organization step can be automated. To this end, we designed a convolutional neural network (CNN) that automatically recognizes eight different brain magnetic resonance imaging (MRI) scan types based on visual appearance. Thus, our method is unaffected by inconsistent or missing scan metadata. It can recognize pre-contrast T1-weighted (T1w),post-contrast T1-weighted (T1wC), T2-weighted (T2w), proton density-weighted (PDw) and derived maps (e.g. apparent diffusion coefficient and cerebral blood flow). In a first experiment,we used scans of subjects with brain tumors: 11065 scans of 719 subjects for training, and 2369 scans of 192 subjects for testing. The CNN achieved an overall accuracy of 98.7%. In a second experiment, we trained the CNN on all 13434 scans from the first experiment and tested it on 7227 scans of 1318 Alzheimer's subjects. Here, the CNN achieved an overall accuracy of 98.5%. In conclusion, our method can accurately predict scan type, and can quickly and automatically sort a brain MRI dataset virtually without the need for manual verification. In this way, our method can assist with properly organizing a dataset, which maximizes the shareability and integrity of the data.
Collapse
Affiliation(s)
- Sebastian R van der Voort
- Biomedical Imaging Group Rotterdam, Departments of Radiology and Nuclear Medicine and Medical Informatics, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, The Netherlands.
| | - Marion Smits
- Department of Radiology and Nuclear Medicine, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, The Netherlands
| | - Stefan Klein
- Biomedical Imaging Group Rotterdam, Departments of Radiology and Nuclear Medicine and Medical Informatics, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, The Netherlands
| |
Collapse
|
8
|
|
9
|
Paranjape K, Schinkel M, Nannan Panday R, Car J, Nanayakkara P. Introducing Artificial Intelligence Training in Medical Education. JMIR MEDICAL EDUCATION 2019; 5:e16048. [PMID: 31793895 PMCID: PMC6918207 DOI: 10.2196/16048] [Citation(s) in RCA: 171] [Impact Index Per Article: 34.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/09/2019] [Revised: 10/16/2019] [Accepted: 10/20/2019] [Indexed: 05/18/2023]
Abstract
Health care is evolving and with it the need to reform medical education. As the practice of medicine enters the age of artificial intelligence (AI), the use of data to improve clinical decision making will grow, pushing the need for skillful medicine-machine interaction. As the rate of medical knowledge grows, technologies such as AI are needed to enable health care professionals to effectively use this knowledge to practice medicine. Medical professionals need to be adequately trained in this new technology, its advantages to improve cost, quality, and access to health care, and its shortfalls such as transparency and liability. AI needs to be seamlessly integrated across different aspects of the curriculum. In this paper, we have addressed the state of medical education at present and have recommended a framework on how to evolve the medical education curriculum to include AI.
Collapse
Affiliation(s)
| | - Michiel Schinkel
- Department of Internal Medicine, Amsterdam University Medical Center, Amsterdam, Netherlands
| | - Rishi Nannan Panday
- Section Acute Medicine, Department of Internal Medicine, Vrije Universiteit University Medical Center, Amsterdam, Netherlands
| | - Josip Car
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Prabath Nanayakkara
- Department of Internal Medicine, Amsterdam University Medical Center, Amsterdam, Netherlands
| |
Collapse
|