1
|
Yu B, Kaku A, Liu K, Parnandi A, Fokas E, Venkatesan A, Pandit N, Ranganath R, Schambra H, Fernandez-Granda C. Quantifying impairment and disease severity using AI models trained on healthy subjects. NPJ Digit Med 2024; 7:180. [PMID: 38969786 PMCID: PMC11226623 DOI: 10.1038/s41746-024-01173-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 06/21/2024] [Indexed: 07/07/2024] Open
Abstract
Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. We propose a framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors (ρ = 0.814, 95% CI [0.700,0.888]) and video (ρ = 0.736, 95% C.I [0.584, 0.838]). To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment (ρ = 0.644, 95% C.I [0.585,0.696]).
Collapse
Affiliation(s)
- Boyang Yu
- Center for Data Science, New York University, 60 Fifth Ave, New York, NY, 10011, USA
| | - Aakash Kaku
- Center for Data Science, New York University, 60 Fifth Ave, New York, NY, 10011, USA
| | - Kangning Liu
- Center for Data Science, New York University, 60 Fifth Ave, New York, NY, 10011, USA
| | - Avinash Parnandi
- Department of Neurology, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA
- Department of Rehabilitation Medicine, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA
| | - Emily Fokas
- Department of Neurology, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA
| | - Anita Venkatesan
- Department of Neurology, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA
| | - Natasha Pandit
- Department of Rehabilitation Medicine, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA
| | - Rajesh Ranganath
- Center for Data Science, New York University, 60 Fifth Ave, New York, NY, 10011, USA
- Courant Institute of Mathematical Sciences, New York University, 251 Mercer St, New York, NY, 10012, USA
| | - Heidi Schambra
- Department of Neurology, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA.
- Department of Rehabilitation Medicine, NYU Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, USA.
| | - Carlos Fernandez-Granda
- Center for Data Science, New York University, 60 Fifth Ave, New York, NY, 10011, USA.
- Courant Institute of Mathematical Sciences, New York University, 251 Mercer St, New York, NY, 10012, USA.
| |
Collapse
|
2
|
Jebril H, Esengönül M, Bogunović H. Anomaly Detection in Optical Coherence Tomography Angiography (OCTA) with a Vector-Quantized Variational Auto-Encoder (VQ-VAE). Bioengineering (Basel) 2024; 11:682. [PMID: 39061764 PMCID: PMC11273395 DOI: 10.3390/bioengineering11070682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2024] [Revised: 06/29/2024] [Accepted: 07/02/2024] [Indexed: 07/28/2024] Open
Abstract
Optical coherence tomography angiography (OCTA) provides detailed information on retinal blood flow and perfusion. Abnormal retinal perfusion indicates possible ocular or systemic disease. We propose a deep learning-based anomaly detection model to identify such anomalies in OCTA. It utilizes two deep learning approaches. First, a representation learning with a Vector-Quantized Variational Auto-Encoder (VQ-VAE) followed by Auto-Regressive (AR) modeling. Second, it exploits epistemic uncertainty estimates from Bayesian U-Net employed to segment the vasculature on OCTA en face images. Evaluation on two large public datasets, DRAC and OCTA-500, demonstrates effective anomaly detection (an AUROC of 0.92 for the DRAC and an AUROC of 0.75 for the OCTA-500) and localization (a mean Dice score of 0.61 for the DRAC) on this challenging task. To our knowledge, this is the first work that addresses anomaly detection in OCTA.
Collapse
Affiliation(s)
- Hana Jebril
- Lab for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, 1090 Vienna, Austria; (H.J.); (M.E.)
| | - Meltem Esengönül
- Lab for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, 1090 Vienna, Austria; (H.J.); (M.E.)
| | - Hrvoje Bogunović
- Lab for Ophthalmic Image Analysis, Department of Ophthalmology and Optometry, Medical University of Vienna, 1090 Vienna, Austria; (H.J.); (M.E.)
- Christian Doppler Lab for Artificial Intelligence in Retina, Department of Ophthalmology and Optometry, Medical University of Vienna, 1090 Vienna, Austria
| |
Collapse
|
3
|
Kim K, Cho K, Jang R, Kyung S, Lee S, Ham S, Choi E, Hong GS, Kim N. Updated Primer on Generative Artificial Intelligence and Large Language Models in Medical Imaging for Medical Professionals. Korean J Radiol 2024; 25:224-242. [PMID: 38413108 PMCID: PMC10912493 DOI: 10.3348/kjr.2023.0818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 11/27/2023] [Accepted: 12/28/2023] [Indexed: 02/29/2024] Open
Abstract
The emergence of Chat Generative Pre-trained Transformer (ChatGPT), a chatbot developed by OpenAI, has garnered interest in the application of generative artificial intelligence (AI) models in the medical field. This review summarizes different generative AI models and their potential applications in the field of medicine and explores the evolving landscape of Generative Adversarial Networks and diffusion models since the introduction of generative AI models. These models have made valuable contributions to the field of radiology. Furthermore, this review also explores the significance of synthetic data in addressing privacy concerns and augmenting data diversity and quality within the medical domain, in addition to emphasizing the role of inversion in the investigation of generative models and outlining an approach to replicate this process. We provide an overview of Large Language Models, such as GPTs and bidirectional encoder representations (BERTs), that focus on prominent representatives and discuss recent initiatives involving language-vision models in radiology, including innovative large language and vision assistant for biomedicine (LLaVa-Med), to illustrate their practical application. This comprehensive review offers insights into the wide-ranging applications of generative AI models in clinical research and emphasizes their transformative potential.
Collapse
Affiliation(s)
- Kiduk Kim
- Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Kyungjin Cho
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | | | - Sunggu Kyung
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Soyoung Lee
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Sungwon Ham
- Healthcare Readiness Institute for Unified Korea, Korea University Ansan Hospital, Korea University College of Medicine, Ansan, Republic of Korea
| | - Edward Choi
- Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Gil-Sun Hong
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea.
| | - Namkug Kim
- Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
- Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea.
| |
Collapse
|
4
|
Khosravi P, Mohammadi S, Zahiri F, Khodarahmi M, Zahiri J. AI-Enhanced Detection of Clinically Relevant Structural and Functional Anomalies in MRI: Traversing the Landscape of Conventional to Explainable Approaches. J Magn Reson Imaging 2024. [PMID: 38243677 DOI: 10.1002/jmri.29247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 01/05/2024] [Accepted: 01/08/2024] [Indexed: 01/21/2024] Open
Abstract
Anomaly detection in medical imaging, particularly within the realm of magnetic resonance imaging (MRI), stands as a vital area of research with far-reaching implications across various medical fields. This review meticulously examines the integration of artificial intelligence (AI) in anomaly detection for MR images, spotlighting its transformative impact on medical diagnostics. We delve into the forefront of AI applications in MRI, exploring advanced machine learning (ML) and deep learning (DL) methodologies that are pivotal in enhancing the precision of diagnostic processes. The review provides a detailed analysis of preprocessing, feature extraction, classification, and segmentation techniques, alongside a comprehensive evaluation of commonly used metrics. Further, this paper explores the latest developments in ensemble methods and explainable AI, offering insights into future directions and potential breakthroughs. This review synthesizes current insights, offering a valuable guide for researchers, clinicians, and medical imaging experts. It highlights AI's crucial role in improving the precision and speed of detecting key structural and functional irregularities in MRI. Our exploration of innovative techniques and trends furthers MRI technology development, aiming to refine diagnostics, tailor treatments, and elevate patient care outcomes. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 1.
Collapse
Affiliation(s)
- Pegah Khosravi
- Department of Biological Sciences, New York City College of Technology, CUNY, New York City, New York, USA
- The CUNY Graduate Center, City University of New York, New York City, New York, USA
| | - Saber Mohammadi
- Department of Biological Sciences, New York City College of Technology, CUNY, New York City, New York, USA
- Department of Biophysics, Tarbiat Modares University, Tehran, Iran
| | - Fatemeh Zahiri
- Department of Cell and Molecular Sciences, Kharazmi University, Tehran, Iran
| | | | - Javad Zahiri
- Department of Neuroscience, University of California San Diego, San Diego, California, USA
| |
Collapse
|
5
|
Park S, Lee ES, Shin KS, Lee JE, Ye JC. Self-supervised multi-modal training from uncurated images and reports enables monitoring AI in radiology. Med Image Anal 2024; 91:103021. [PMID: 37952385 DOI: 10.1016/j.media.2023.103021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 08/14/2023] [Accepted: 10/31/2023] [Indexed: 11/14/2023]
Abstract
The escalating demand for artificial intelligence (AI) systems that can monitor and supervise human errors and abnormalities in healthcare presents unique challenges. Recent advances in vision-language models reveal the challenges of monitoring AI by understanding both visual and textual concepts and their semantic correspondences. However, there has been limited success in the application of vision-language models in the medical domain. Current vision-language models and learning strategies for photographic images and captions call for a web-scale data corpus of image and text pairs which is not often feasible in the medical domain. To address this, we present a model named medical cross-attention vision-language model (Medical X-VL), which leverages key components to be tailored for the medical domain. The model is based on the following components: self-supervised unimodal models in medical domain and a fusion encoder to bridge them, momentum distillation, sentencewise contrastive learning for medical reports, and sentence similarity-adjusted hard negative mining. We experimentally demonstrated that our model enables various zero-shot tasks for monitoring AI, ranging from the zero-shot classification to zero-shot error correction. Our model outperformed current state-of-the-art models in two medical image datasets, suggesting a novel clinical application of our monitoring AI model to alleviate human errors. Our method demonstrates a more specialized capacity for fine-grained understanding, which presents a distinct advantage particularly applicable to the medical domain.
Collapse
Affiliation(s)
- Sangjoon Park
- Department of Radiation Oncology, Yonsei College of Medicine, Seoul, Republic of Korea
| | - Eun Sun Lee
- Chung-Ang University Hospital, Seoul, Republic of Korea.
| | - Kyung Sook Shin
- Department of Radiology, Chungnam National University Hospital, Chungnam National University College of Medicine, Daejeon, Republic of Korea
| | - Jeong Eun Lee
- Department of Radiology, Chungnam National University Hospital, Chungnam National University College of Medicine, Daejeon, Republic of Korea.
| | - Jong Chul Ye
- Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
| |
Collapse
|
6
|
Hong GS, Jang M, Kyung S, Cho K, Jeong J, Lee GY, Shin K, Kim KD, Ryu SM, Seo JB, Lee SM, Kim N. Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning. Korean J Radiol 2023; 24:1061-1080. [PMID: 37724586 PMCID: PMC10613849 DOI: 10.3348/kjr.2023.0393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Revised: 07/01/2023] [Accepted: 07/30/2023] [Indexed: 09/21/2023] Open
Abstract
Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.
Collapse
Affiliation(s)
- Gil-Sun Hong
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Miso Jang
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Sunggu Kyung
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Kyungjin Cho
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Jiheon Jeong
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Grace Yoojin Lee
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Keewon Shin
- Laboratory for Biosignal Analysis and Perioperative Outcome Research, Biomedical Engineering Center, Asan Institute of Lifesciences, Asan Medical Center, Seoul, Republic of Korea
| | - Ki Duk Kim
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Seung Min Ryu
- Department of Orthopedic Surgery, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Joon Beom Seo
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Sang Min Lee
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| | - Namkug Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
7
|
Raad JD, Chinnam RB, Arslanturk S, Tan S, Jeong JW, Mody S. Unsupervised abnormality detection in neonatal MRI brain scans using deep learning. Sci Rep 2023; 13:11489. [PMID: 37460615 PMCID: PMC10352269 DOI: 10.1038/s41598-023-38430-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 07/07/2023] [Indexed: 07/20/2023] Open
Abstract
Analysis of 3D medical imaging data has been a large topic of focus in the area of Machine Learning/Artificial Intelligence, though little work has been done in algorithmic (particularly unsupervised) analysis of neonatal brain MRI's. A myriad of conditions can manifest at an early age, including neonatal encephalopathy (NE), which can result in lifelong physical consequences. As such, there is a dire need for better biomarkers of NE and other conditions. The objective of the study is to improve identification of anomalies and prognostication of neonatal MRI brain scans. We introduce a framework designed to support the analysis and assessment of neonatal MRI brain scans, the results of which can be used as an aid to neuroradiologists. We explored the efficacy of the framework through iterations of several deep convolutional Autoencoder (AE) unsupervised modeling architectures designed to learn normalcy of the neonatal brain structure. We tested this framework on the developing human connectome project (dHCP) dataset with 97 patients that were previously categorized by severity. Our framework demonstrated the model's ability to identify and distinguish subtle morphological signatures present in brain structures. Normal and abnormal neonatal brain scans can be distinguished with reasonable accuracy, correctly categorizing them in up to 83% of cases. Most critically, new brain anomalies originally missed during the radiological reading were identified and corroborated by a neuroradiologist. This framework and our modeling approach demonstrate an ability to improve prognostication of neonatal brain conditions and are able to localize new anomalies.
Collapse
Affiliation(s)
- Jad Dino Raad
- Industrial and Systems Engineering Department, Wayne State University, Detroit, MI, 48201, USA
| | - Ratna Babu Chinnam
- Industrial and Systems Engineering Department, Wayne State University, Detroit, MI, 48201, USA
| | - Suzan Arslanturk
- Computer Science Department, Wayne State University, Detroit, MI, 48201, USA.
| | - Sidhartha Tan
- Department of Pediatrics, Wayne State University, Detroit, MI, 48201, USA
| | - Jeong-Won Jeong
- Department of Pediatrics, Wayne State University, Detroit, MI, 48201, USA
| | - Swati Mody
- Division of Pediatric Radiology, University of Michigan, Ann Arbor, MI, 48109, USA
| |
Collapse
|
8
|
Parnandi A, Kaku A, Venkatesan A, Pandit N, Fokas E, Yu B, Kim G, Nilsen D, Fernandez-Granda C, Schambra H. Data-Driven Quantitation of Movement Abnormality after Stroke. Bioengineering (Basel) 2023; 10:648. [PMID: 37370579 PMCID: PMC10294965 DOI: 10.3390/bioengineering10060648] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Revised: 05/19/2023] [Accepted: 05/23/2023] [Indexed: 06/29/2023] Open
Abstract
Stroke commonly affects the ability of the upper extremities (UEs) to move normally. In clinical settings, identifying and measuring movement abnormality is challenging due to the imprecision and impracticality of available assessments. These challenges interfere with therapeutic tracking, communication, and treatment. We thus sought to develop an approach that blends precision and pragmatism, combining high-dimensional motion capture with out-of-distribution (OOD) detection. We used an array of wearable inertial measurement units to capture upper body motion in healthy and chronic stroke subjects performing a semi-structured, unconstrained 3D tabletop task. After data were labeled by human coders, we trained two deep learning models exclusively on healthy subject data to classify elemental movements (functional primitives). We tested these healthy subject-trained models on previously unseen healthy and stroke motion data. We found that model confidence, indexed by prediction probabilities, was generally high for healthy test data but significantly dropped when encountering OOD stroke data. Prediction probabilities worsened with more severe motor impairment categories and were directly correlated with individual impairment scores. Data inputs from the paretic UE, rather than trunk, most strongly influenced model confidence. We demonstrate for the first time that using OOD detection with high-dimensional motion data can reveal clinically meaningful movement abnormality in subjects with chronic stroke.
Collapse
Affiliation(s)
- Avinash Parnandi
- Department of Neurology, NYU Grossman School of Medicine, New York, NY 10017, USA; (A.P.)
| | - Aakash Kaku
- NYU Center for Data Science, New York, NY 10011, USA; (A.K.)
| | - Anita Venkatesan
- Department of Neurology, NYU Grossman School of Medicine, New York, NY 10017, USA; (A.P.)
| | - Natasha Pandit
- Department of Neurology, NYU Grossman School of Medicine, New York, NY 10017, USA; (A.P.)
| | - Emily Fokas
- Department of Neurology, NYU Grossman School of Medicine, New York, NY 10017, USA; (A.P.)
| | - Boyang Yu
- NYU Center for Data Science, New York, NY 10011, USA; (A.K.)
| | - Grace Kim
- Department of Occupational Therapy, NYU Steinhardt, New York, NY 10011, USA
| | - Dawn Nilsen
- Department of Rehabilitation and Regenerative Medicine, Columbia University, New York, NY 10032, USA
| | - Carlos Fernandez-Granda
- NYU Center for Data Science, New York, NY 10011, USA; (A.K.)
- Courant Institute of Mathematical Sciences, New York, NY 10011, USA
| | - Heidi Schambra
- Department of Neurology, NYU Grossman School of Medicine, New York, NY 10017, USA; (A.P.)
- Department of Rehabilitation Medicine, NYU Grossman School of Medicine, New York, NY 10017, USA
| |
Collapse
|
9
|
Yun C, Eom B, Park S, Kim C, Kim D, Jabeen F, Kim WH, Kim HJ, Kim J. A Study on the Effectiveness of Deep Learning-Based Anomaly Detection Methods for Breast Ultrasonography. SENSORS (BASEL, SWITZERLAND) 2023; 23:2864. [PMID: 36905074 PMCID: PMC10007509 DOI: 10.3390/s23052864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 01/04/2023] [Accepted: 01/12/2023] [Indexed: 06/18/2023]
Abstract
In the medical field, it is delicate to anticipate good performance in using deep learning due to the lack of large-scale training data and class imbalance. In particular, ultrasound, which is a key breast cancer diagnosis method, is delicate to diagnose accurately as the quality and interpretation of images can vary depending on the operator's experience and proficiency. Therefore, computer-aided diagnosis technology can facilitate diagnosis by visualizing abnormal information such as tumors and masses in ultrasound images. In this study, we implemented deep learning-based anomaly detection methods for breast ultrasound images and validated their effectiveness in detecting abnormal regions. Herein, we specifically compared the sliced-Wasserstein autoencoder with two representative unsupervised learning models autoencoder and variational autoencoder. The anomalous region detection performance is estimated with the normal region labels. Our experimental results showed that the sliced-Wasserstein autoencoder model outperformed the anomaly detection performance of others. However, anomaly detection using the reconstruction-based approach may not be effective because of the occurrence of numerous false-positive values. In the following studies, reducing these false positives becomes an important challenge.
Collapse
Affiliation(s)
- Changhee Yun
- National Information Society Agency, Daegu 41068, Republic of Korea
| | - Bomi Eom
- National Information Society Agency, Daegu 41068, Republic of Korea
| | - Sungjun Park
- School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Republic of Korea
| | - Chanho Kim
- School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Republic of Korea
| | - Dohwan Kim
- Department of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea
| | - Farah Jabeen
- School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Republic of Korea
| | - Won Hwa Kim
- Department of Radiology, Kyungpook National University Chilgok Hospital, Kyungpook National University, Daegu 41404, Republic of Korea
| | - Hye Jung Kim
- Department of Radiology, Kyungpook National University Chilgok Hospital, Kyungpook National University, Daegu 41404, Republic of Korea
| | - Jaeil Kim
- School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Republic of Korea
| |
Collapse
|
10
|
Luo G, Xie W, Gao R, Zheng T, Chen L, Sun H. Unsupervised anomaly detection in brain MRI: Learning abstract distribution from massive healthy brains. Comput Biol Med 2023; 154:106610. [PMID: 36708653 DOI: 10.1016/j.compbiomed.2023.106610] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Revised: 01/20/2023] [Accepted: 01/22/2023] [Indexed: 01/27/2023]
Abstract
PURPOSE To develop a general unsupervised anomaly detection method based only on MR images of normal brains to automatically detect various brain abnormalities. MATERIALS AND METHODS In this study, a novel method based on three-dimensional deep autoencoder network is proposed to automatically detect and segment various brain abnormalities without being trained on any abnormal samples. A total of 578 normal T2w MR volumes without obvious abnormalities were used for model training and validation. The proposed 3D autoencoder was evaluated on two different datasets (BraTs dataset and in-house dataset) containing T2w volumes from patients with glioblastoma, multiple sclerosis and cerebral infarction. Lesions detection and segmentation performance were reported as AUC, precision-recall curve, sensitivity, and Dice score. RESULTS In anomaly detection, AUCs for three typical lesions were as follows: glioblastoma, 0.844; multiple sclerosis, 0.858; cerebral infarction, 0.807. In anomaly segmentation, the mean Dice for glioblastomas was 0.462. The proposed network also has the ability to generate an anomaly heatmap for visualization purpose. CONCLUSION Our proposed method was able to automatically detect various brain anomalies such as glioblastoma, multiple sclerosis, and cerebral infarction. This work suggests that unsupervised anomaly detection is a powerful approach to detect arbitrary brain abnormalities without labeled samples. It has the potential to support diagnostic workflow in radiology as an automated tool for computer-aided image analysis.
Collapse
Affiliation(s)
- Guoting Luo
- Huaxi MR Research Center (HMRRC), Department of Radiology, West China Hospital of Sichuan University, Chengdu, China
| | - Wei Xie
- Department of Radiology, West China Hospital of Sichuan University, Chengdu, China
| | - Ronghui Gao
- Department of Radiology, West China Hospital of Sichuan University, Chengdu, China
| | - Tao Zheng
- IT Center, West China Hospital of Sichuan University, Chengdu, China
| | - Lei Chen
- Department of Neurology, West China Hospital of Sichuan University, Chengdu, China.
| | - Huaiqiang Sun
- Huaxi MR Research Center (HMRRC), Department of Radiology, West China Hospital of Sichuan University, Chengdu, China.
| |
Collapse
|
11
|
Fan W, Shangguan W, Bouguila N. Continuous image anomaly detection based on contrastive lifelong learning. APPL INTELL 2023. [DOI: 10.1007/s10489-022-04401-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
12
|
Lebenberg J, Zhang R, Grosset L, Guichard JP, Fernandes F, Jouvent E, Chabriat H. Segmentation of incident lacunes during the course of ischemic cerebral small vessel diseases. Front Neurol 2023; 14:1113644. [PMID: 37034061 PMCID: PMC10076773 DOI: 10.3389/fneur.2023.1113644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 02/22/2023] [Indexed: 04/11/2023] Open
Abstract
Background Lacunes represent key imaging markers of cerebral small vessel diseases (cSVDs). During their progression, incident lacunes are related to stroke manifestations and contribute to progressive cognitive and/or motor decline. Assessing new lesions has become crucial but remains time-consuming and error-prone, even for an expert. We, thus, sought to develop and validate an automatic segmentation method of incident lacunes in CADASIL caused by cysteine mutation in the EGFr domains of the NOTCH3 gene, a severe and progressive monogenic form of cSVD. Methods Incident lacunes were identified based on difference maps of 3D T1-weighted MRIs obtained at the baseline and 2 years later. These maps were thresholded using clustering analysis and compared with results obtained by expert visual analysis, which is considered the gold standard approach. Results The median number of lacunes at the baseline in 30 randomly selected patients was 7 (IQR = [2, 11]). The median number of incident lacunes was 2 (IQR = [0, 3]) using the automatic method (mean time-processing: 25 s/patient) and 0.5 (IQR = [0, 2]) using the standard visual approach (mean time-processing: 8 min/patient). The complementary analysis of segmentation results is enabled to quickly remove false positives detected in specific locations and to identify true incident lesions not previously detected by the standard analysis (2 min/case). A combined approach based on automatic segmentation of incident lacunes followed by quick corrections of false positives allowed to reach high individual sensitivity (median at 0.66, IQR = [0.21, 1.00]) and global specificity scores (0.80). Conclusion The automatic segmentation of incident lacunes followed by quick corrections of false positives appears promising for properly and rapidly quantifying incident lacunes in large cohorts of cSVDs.
Collapse
Affiliation(s)
- Jessica Lebenberg
- APHP, Lariboisière Hospital, Translational Neurovascular Centre, FHU Neurovasc, Université Paris Cité, Paris, France
- U1141, Université Paris Cité, Inserm, Neurodiderot, Paris, France
| | - Ruiting Zhang
- U1141, Université Paris Cité, Inserm, Neurodiderot, Paris, France
- Department of Radiology, School of Medicine, The Second Affiliated Hospital of Zhejiang University, Hangzhou, China
| | - Lina Grosset
- U1141, Université Paris Cité, Inserm, Neurodiderot, Paris, France
- APHP, Lariboisière Hospital, Department of Neurology, FHU NeuroVasc, Université Paris Cité, Paris, France
- Faculté de Santé, Université Paris Cité, Paris, France
| | - Jean Pierre Guichard
- APHP, Lariboisière Hospital, Department of Neuroradiology, Université Paris Cité, Paris, France
| | - Fanny Fernandes
- APHP, Lariboisière Hospital, Translational Neurovascular Centre, FHU Neurovasc, Université Paris Cité, Paris, France
| | - Eric Jouvent
- U1141, Université Paris Cité, Inserm, Neurodiderot, Paris, France
- APHP, Lariboisière Hospital, Department of Neurology, FHU NeuroVasc, Université Paris Cité, Paris, France
- Faculté de Santé, Université Paris Cité, Paris, France
| | - Hugues Chabriat
- APHP, Lariboisière Hospital, Translational Neurovascular Centre, FHU Neurovasc, Université Paris Cité, Paris, France
- U1141, Université Paris Cité, Inserm, Neurodiderot, Paris, France
- Faculté de Santé, Université Paris Cité, Paris, France
- *Correspondence: Hugues Chabriat
| |
Collapse
|
13
|
Multilevel hybrid accurate handcrafted model for myocardial infarction classification using ECG signals. INT J MACH LEARN CYB 2022; 14:1651-1668. [PMID: 36467277 PMCID: PMC9702788 DOI: 10.1007/s13042-022-01718-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Accepted: 11/03/2022] [Indexed: 11/29/2022]
Abstract
Myocardial infarction (MI) is detected using electrocardiography (ECG) signals. Machine learning (ML) models have been used for automated MI detection on ECG signals. Deep learning models generally yield high classification performance but are computationally intensive. We have developed a novel multilevel hybrid feature extraction-based classification model with low time complexity for MI classification. The study dataset comprising 12-lead ECGs belonging to one healthy and 10 MI classes were downloaded from a public ECG signal databank. The model architecture comprised multilevel hybrid feature extraction, iterative feature selection, classification, and iterative majority voting (IMV). In the hybrid handcrafted feature (HHF) generation phase, both textural and statistical feature extraction functions were used to extract features from ECG beats but only at a low level. A new pooling-based multilevel decomposition model was presented to enable them to create features at a high level. This model used average and maximum pooling to create decomposed signals. Using these pooling functions, an unbalanced tree was obtained. Therefore, this model was named multilevel unbalanced pooling tree transformation (MUPTT). On the feature extraction side, two extractors (functions) were used to generate both statistical and textural features. To generate statistical features, 20 commonly used moments were used. A new, improved symmetric binary pattern function was proposed to generate textural features. Both feature extractors were applied to the original MI signal and the decomposed signals generated by the MUPTT. The most valuable features from among the extracted feature vectors were selected using iterative neighborhood component analysis (INCA). In the classification phase, a one-dimensional nearest neighbor classifier with ten-fold cross-validation was used to obtain lead-wise results. The computed lead-wise results derived from all 12 leads of the same beat were input to the IMV algorithm to generate ten voted results. The most representative was chosen using a greedy technique to calculate the overall classification performance of the model. The HHF-MUPTT-based ECG beat classification model attained excellent performance, with the best lead-wise accuracy of 99.85% observed in Lead III and 99.94% classification accuracy using the IMV algorithm. The results confirmed the high MI classification ability of the presented computationally lightweight HHF-MUPTT-based model.
Collapse
|
14
|
Devika K, Mahapatra D, Subramanian R, Ramana Murthy Oruganti V. Dense Attentive GAN-based One-Class Model for Detection of Autism and ADHD. JOURNAL OF KING SAUD UNIVERSITY - COMPUTER AND INFORMATION SCIENCES 2022. [DOI: 10.1016/j.jksuci.2022.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
15
|
Posilović L, Medak D, Milković F, Subašić M, Budimir M, Lončarić S. Deep learning-based anomaly detection from ultrasonic images. ULTRASONICS 2022; 124:106737. [PMID: 35427859 DOI: 10.1016/j.ultras.2022.106737] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 03/18/2022] [Accepted: 03/19/2022] [Indexed: 06/14/2023]
Abstract
Non-destructive testing is a group of methods for evaluating the integrity of components. Among them, ultrasonic inspection stands out due to its ability to visualize both shallow and deep sections of the material in the search for flaws. Testing of the critical components can be a tiring and time-consuming task. Therefore, human experts in analyzing inspection data could use a hand in discarding anomaly-free data and reviewing only suspicious data. Using such a tool, errors would be less common, inspection times would shorten and non-destructive testing would be more efficient. In this work, we evaluate multiple state-of-the-art deep-learning anomaly detection methods on the ultrasonic non-destructive testing dataset. We achieved an average performance of almost 82% of ROC AUC. We discuss in detail the advantages and disadvantages of the presented methods.
Collapse
Affiliation(s)
- Luka Posilović
- University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia.
| | - Duje Medak
- University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia.
| | - Fran Milković
- University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia.
| | - Marko Subašić
- University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia.
| | - Marko Budimir
- INETEC Institute for Nuclear Technology, Zagreb, Croatia.
| | - Sven Lončarić
- University of Zagreb, Faculty of Electrical Engineering and Computing, Zagreb, Croatia.
| |
Collapse
|
16
|
Dogan S, Barua PD, Baygin M, Chakraborty S, Ciaccio E, Tuncer T, Abd Kadir KA, Md Shah MN, Azman RR, Lee CC, Ng KH, Acharya UR. Novel multiple pooling and local phase quantization stable feature extraction techniques for automated classification of brain infarcts. Biocybern Biomed Eng 2022. [DOI: 10.1016/j.bbe.2022.06.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
17
|
Naga D, Muster W, Musvasva E, Ecker GF. Off-targetP ML: an open source machine learning framework for off-target panel safety assessment of small molecules. J Cheminform 2022; 14:27. [PMID: 35525988 PMCID: PMC9077900 DOI: 10.1186/s13321-022-00603-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Accepted: 03/26/2022] [Indexed: 11/10/2022] Open
Abstract
Unpredicted drug safety issues constitute the majority of failures in the pharmaceutical industry according to several studies. Some of these preclinical safety issues could be attributed to the non-selective binding of compounds to targets other than their intended therapeutic target, causing undesired adverse events. Consequently, pharmaceutical companies routinely run in-vitro safety screens to detect off-target activities prior to preclinical and clinical studies. Hereby we present an open source machine learning framework aiming at the prediction of our in-house 50 off-target panel activities for ~ 4000 compounds, directly from their structure. This framework is intended to guide chemists in the drug design process prior to synthesis and to accelerate drug discovery. We also present a set of ML approaches that require minimum programming experience for deployment. The workflow incorporates different ML approaches such as deep learning and automated machine learning. It also accommodates popular issues faced in bioactivity predictions, as data imbalance, inter-target duplicated measurements and duplicated public compound identifiers. Throughout the workflow development, we explore and compare the capability of Neural Networks and AutoML in constructing prediction models for fifty off-targets of different protein classes, different dataset sizes, and high-class imbalance. Outcomes from different methods are compared in terms of efficiency and efficacy. The most important challenges and factors impacting model construction and performance in addition to suggestions on how to overcome such challenges are also discussed.
Collapse
Affiliation(s)
- Doha Naga
- Roche Pharma Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland.,Department of Pharmaceutical Sciences, University of Vienna, Vienna, Austria
| | - Wolfgang Muster
- Roche Pharma Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland
| | - Eunice Musvasva
- Roche Pharma Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland
| | - Gerhard F Ecker
- Department of Pharmaceutical Sciences, University of Vienna, Vienna, Austria.
| |
Collapse
|
18
|
Ma Z, Reich DS, Dembling S, Duyn JH, Koretsky AP. Outlier detection in multimodal MRI identifies rare individual phenotypes among more than 15,000 brains. Hum Brain Mapp 2022; 43:1766-1782. [PMID: 34957633 PMCID: PMC8886649 DOI: 10.1002/hbm.25756] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2021] [Revised: 11/25/2021] [Accepted: 12/04/2021] [Indexed: 01/11/2023] Open
Abstract
Outliers in neuroimaging represent spurious data or the data of unusual phenotypes that deserve special attention such as clinical follow-up. Outliers have usually been detected in a supervised or semi-supervised manner for labeled neuroimaging cohorts. There has been much less work using unsupervised outlier detection on large unlabeled cohorts like the UK Biobank brain imaging dataset. Given its large sample size, rare imaging phenotypes within this unique cohort are of interest, as they are often clinically relevant and could be informative for discovering new processes. Here, we developed a two-level outlier detection and screening methodology to characterize individual outliers from the multimodal MRI dataset of more than 15,000 UK Biobank subjects. In primary screening, using brain ventricles, white matter, cortical thickness, and functional connectivity-based imaging phenotypes, every subject was parameterized with an outlier score per imaging phenotype. Outlier scores of these imaging phenotypes had good-to-excellent test-retest reliability, with the exception of resting-state functional connectivity (RSFC). Due to the low reliability of RSFC outlier scores, RSFC outliers were excluded from further individual-level outlier screening. In secondary screening, the extreme outliers (1,026 subjects) were examined individually, and those arising from data collection/processing errors were eliminated. A representative subgroup of 120 subjects from the remaining non-artifactual outliers were radiologically reviewed, and radiological findings were identified in 97.5% of them. This study establishes an unsupervised framework for investigating rare individual imaging phenotypes within a large neuroimaging cohort.
Collapse
Affiliation(s)
- Zhiwei Ma
- Laboratory of Functional and Molecular ImagingNational Institute of Neurological Disorders and Stroke, National Institutes of HealthBethesdaMarylandUSA
| | - Daniel S. Reich
- Translational Neuroradiology SectionNational Institute of Neurological Disorders and Stroke, National Institutes of HealthBethesdaMarylandUSA
| | - Sarah Dembling
- Laboratory of Functional and Molecular ImagingNational Institute of Neurological Disorders and Stroke, National Institutes of HealthBethesdaMarylandUSA
| | - Jeff H. Duyn
- Laboratory of Functional and Molecular ImagingNational Institute of Neurological Disorders and Stroke, National Institutes of HealthBethesdaMarylandUSA
| | - Alan P. Koretsky
- Laboratory of Functional and Molecular ImagingNational Institute of Neurological Disorders and Stroke, National Institutes of HealthBethesdaMarylandUSA
| |
Collapse
|
19
|
Obuseh M, Yu D, DeLaurentis P. Detecting Unusual Intravenous Infusion Alerting Patterns with Machine Learning Algorithms. Biomed Instrum Technol 2022. [PMID: 35749264 DOI: 10.2345/1943-5967-56.2.58] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE To detect unusual infusion alerting patterns using machine learning (ML) algorithms as a first step to advance safer inpatient intravenous administration of high-alert medications. MATERIALS AND METHODS We used one year of detailed propofol infusion data from a hospital. Interpretable and clinically relevant variables were feature engineered, and data points were aggregated per calendar day. A univariate (maximum times-limit) moving range (mr) control chart was used to simulate clinicians' common approach to identifying unusual infusion alerting patterns. Three different unsupervised multivariate ML-based anomaly detection algorithms (Local Outlier Factor, Isolation Forest, and k-Nearest Neighbors) were used for the same purpose. Results from the control chart and ML algorithms were compared. RESULTS The propofol data had 3,300 infusion alerts, 92% of which were generated during the day shift and seven of which had a times-limit greater than 10. The mr-chart identified 15 alert pattern anomalies. Different thresholds were set to include the top 15 anomalies from each ML algorithm. A total of 31 unique ML anomalies were grouped and ranked by agreeability. All algorithms agreed on 10% of the anomalies, and at least two algorithms agreed on 36%. Each algorithm detected one specific anomaly that the mr-chart did not detect. The anomaly represented a day with 71 propofol alerts (half of which were overridden) generated at an average rate of 1.06 per infusion, whereas the moving alert rate for the week was 0.35 per infusion. DISCUSSION These findings show that ML-based algorithms are more robust than control charts in detecting unusual alerting patterns. However, we recommend using a combination of algorithms, as multiple algorithms serve a benchmarking function and allow researchers to focus on data points with the highest algorithm agreeability. CONCLUSION Unsupervised ML algorithms can assist clinicians in identifying unusual alert patterns as a first step toward achieving safer infusion practices.
Collapse
Affiliation(s)
- Marian Obuseh
- Marian Obuseh is a PhD student in the School of Industrial Engineering at Purdue University in West Lafayette, IN.
| | - Denny Yu
- Denny Yu, PhD, is an assistant professor in the School of Industrial Engineering at Purdue University in West Lafayette, IN
| | - Poching DeLaurentis
- Poching DeLaurentis, PhD, was a research scientist in the Regenstrief Center for Healthcare Engineering at Purdue University in West Lafayette, IN, at the time this study was conducted
| |
Collapse
|
20
|
|