1
|
Ziegenfeuter J, Delbridge C, Bernhardt D, Gempt J, Schmidt-Graf F, Hedderich D, Griessmair M, Thomas M, Meyer HS, Zimmer C, Meyer B, Combs SE, Yakushev I, Metz MC, Wiestler B. Resolving spatial response heterogeneity in glioblastoma. Eur J Nucl Med Mol Imaging 2024:10.1007/s00259-024-06782-y. [PMID: 38837060 DOI: 10.1007/s00259-024-06782-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2024] [Accepted: 05/30/2024] [Indexed: 06/06/2024]
Abstract
PURPOSE Spatial intratumoral heterogeneity poses a significant challenge for accurate response assessment in glioblastoma. Multimodal imaging coupled with advanced image analysis has the potential to unravel this response heterogeneity. METHODS Based on automated tumor segmentation and longitudinal registration with follow-up imaging, we categorized contrast-enhancing voxels of 61 patients with suspected recurrence of glioblastoma into either true tumor progression (TP) or pseudoprogression (PsP). To allow the unbiased analysis of semantically related image regions, adjacent voxels with similar values of cerebral blood volume (CBV), FET-PET, and contrast-enhanced T1w were automatically grouped into supervoxels. We then extracted first-order statistics as well as texture features from each supervoxel. With these features, a Random Forest classifier was trained and validated employing a 10-fold cross-validation scheme. For model evaluation, the area under the receiver operating curve, as well as classification performance metrics were calculated. RESULTS Our image analysis pipeline enabled reliable spatial assessment of tumor response. The predictive model reached an accuracy of 80.0% and a macro-weighted AUC of 0.875, which takes class imbalance into account, in the hold-out samples from cross-validation on supervoxel level. Analysis of feature importances confirmed the significant role of FET-PET-derived features. Accordingly, TP- and PsP-labeled supervoxels differed significantly in their 10th and 90th percentile, as well as the median of tumor-to-background normalized FET-PET. However, CBV- and T1c-related features also relevantly contributed to the model's performance. CONCLUSION Disentangling the intratumoral heterogeneity in glioblastoma holds immense promise for advancing precise local response evaluation and thereby also informing more personalized and localized treatment strategies in the future.
Collapse
Affiliation(s)
- Julian Ziegenfeuter
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany.
| | - Claire Delbridge
- Department of Pathology, Technical University of Munich, 81675, München, Germany
| | - Denise Bernhardt
- Department of Radiation Oncology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Jens Gempt
- Department of Neurosurgery, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
- Department of Neurosurgery, University Medical Center Hamburg-Eppendorf, 20251, Hamburg, Germany
| | - Friederike Schmidt-Graf
- Department of Neurology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Dennis Hedderich
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Michael Griessmair
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Marie Thomas
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Hanno S Meyer
- Department of Neurosurgery, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
- Department of Neurosurgery, University Medical Center Hamburg-Eppendorf, 20251, Hamburg, Germany
| | - Claus Zimmer
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Bernhard Meyer
- Department of Neurosurgery, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Stephanie E Combs
- Department of Radiation Oncology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Igor Yakushev
- Department of Nuclear Medicine, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Marie-Christin Metz
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
| | - Benedikt Wiestler
- Department of Neuroradiology, School of Medicine and Health, Technical University of Munich, 81675, München, Germany
- TranslaTUM, Technical University of Munich, 81675, München, Germany
| |
Collapse
|
2
|
Kantipudi K, Gu J, Bui V, Yu H, Jaeger S, Yaniv Z. Automated Pulmonary Tuberculosis Severity Assessment on Chest X-rays. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01052-7. [PMID: 38587769 DOI: 10.1007/s10278-024-01052-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Revised: 01/18/2024] [Accepted: 02/12/2024] [Indexed: 04/09/2024]
Abstract
According to the 2022 World Health Organization's Global Tuberculosis (TB) report, an estimated 10.6 million people fell ill with TB, and 1.6 million died from the disease in 2021. In addition, 2021 saw a reversal of a decades-long trend of declining TB infections and deaths, with an estimated increase of 4.5% in the number of people who fell ill with TB compared to 2020, and an estimated yearly increase of 450,000 cases of drug resistant TB. Estimating the severity of pulmonary TB using frontal chest X-rays (CXR) can enable better resource allocation in resource constrained settings and monitoring of treatment response, enabling prompt treatment modifications if disease severity does not decrease over time. The Timika score is a clinically used TB severity score based on a CXR reading. This work proposes and evaluates three deep learning-based approaches for predicting the Timika score with varying levels of explainability. The first approach uses two deep learning-based models, one to explicitly detect lesion regions using YOLOV5n and another to predict the presence of cavitation using DenseNet121, which are then utilized in score calculation. The second approach uses a DenseNet121-based regression model to directly predict the affected lung percentage and another to predict cavitation presence using a DenseNet121-based classification model. Finally, the third approach directly predicts the Timika score using a DenseNet121-based regression model. The best performance is achieved by the second approach with a mean absolute error of 13-14% and a Pearson correlation of 0.7-0.84 using three held-out datasets for evaluating generalization.
Collapse
Affiliation(s)
- Karthik Kantipudi
- Office of Cyber Infrastructure and Computational Biology, National Institute of Allergy and Infectious Diseases, Bethesda, 20892, MD, USA.
| | - Jingwen Gu
- Office of Cyber Infrastructure and Computational Biology, National Institute of Allergy and Infectious Diseases, Bethesda, 20892, MD, USA
| | - Vy Bui
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, 20894, MD, USA
| | - Hang Yu
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, 20894, MD, USA
| | - Stefan Jaeger
- Lister Hill National Center for Biomedical Communications, National Library of Medicine, Bethesda, 20894, MD, USA
| | - Ziv Yaniv
- Office of Cyber Infrastructure and Computational Biology, National Institute of Allergy and Infectious Diseases, Bethesda, 20892, MD, USA.
| |
Collapse
|
3
|
Zeng Y, Li J, Zhao Z, Liang W, Zeng P, Shen S, Zhang K, Shen C. WET-UNet: Wavelet integrated efficient transformer networks for nasopharyngeal carcinoma tumor segmentation. Sci Prog 2024; 107:368504241232537. [PMID: 38567422 PMCID: PMC11320696 DOI: 10.1177/00368504241232537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Nasopharyngeal carcinoma is a malignant tumor that occurs in the epithelium and mucosal glands of the nasopharynx, and its pathological type is mostly poorly differentiated squamous cell carcinoma. Since the nasopharynx is located deep in the head and neck, early diagnosis and timely treatment are critical to patient survival. However, nasopharyngeal carcinoma tumors are small in size and vary widely in shape, and it is also a challenge for experienced doctors to delineate tumor contours. In addition, due to the special location of nasopharyngeal carcinoma, complex treatments such as radiotherapy or surgical resection are often required, so accurate pathological diagnosis is also very important for the selection of treatment options. However, the current deep learning segmentation model faces the problems of inaccurate segmentation and unstable segmentation process, which are mainly limited by the accuracy of data sets, fuzzy boundaries, and complex lines. In order to solve these two challenges, this article proposes a hybrid model WET-UNet based on the UNet network as a powerful alternative for nasopharyngeal cancer image segmentation. On the one hand, wavelet transform is integrated into UNet to enhance the lesion boundary information by using low-frequency components to adjust the encoder at low frequencies and optimize the subsequent computational process of the Transformer to improve the accuracy and robustness of image segmentation. On the other hand, the attention mechanism retains the most valuable pixels in the image for us, captures the remote dependencies, and enables the network to learn more representative features to improve the recognition ability of the model. Comparative experiments show that our network structure outperforms other models for nasopharyngeal cancer image segmentation, and we demonstrate the effectiveness of adding two modules to help tumor segmentation. The total data set of this article is 5000, and the ratio of training and verification is 8:2. In the experiment, accuracy = 85.2% and precision = 84.9% can show that our proposed model has good performance in nasopharyngeal cancer image segmentation.
Collapse
Affiliation(s)
- Yan Zeng
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Jun Li
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Zhe Zhao
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Wei Liang
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Penghui Zeng
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Shaodong Shen
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| | - Kun Zhang
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information Science and Technology, Hainan Normal University, Haikou, China
| | - Chong Shen
- State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China
- School of Information and Communication Engineering, Hainan University, Haikou, China
| |
Collapse
|
4
|
Zou X, Ren Y, Yang H, Zou M, Meng P, Zhang L, Gong M, Ding W, Han L, Zhang T. Screening and staging of chronic obstructive pulmonary disease with deep learning based on chest X-ray images and clinical parameters. BMC Pulm Med 2024; 24:153. [PMID: 38532368 DOI: 10.1186/s12890-024-02945-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 03/01/2024] [Indexed: 03/28/2024] Open
Abstract
BACKGROUND Chronic obstructive pulmonary disease (COPD) is underdiagnosed with the current gold standard measure pulmonary function test (PFT). A more sensitive and simple option for early detection and severity evaluation of COPD could benefit practitioners and patients. METHODS In this multicenter retrospective study, frontal chest X-ray (CXR) images and related clinical information of 1055 participants were collected and processed. Different deep learning algorithms and transfer learning models were trained to classify COPD based on clinical data and CXR images from 666 subjects, and validated in internal test set based on 284 participants. External test including 105 participants was also performed to verify the generalization ability of the learning algorithms in diagnosing COPD. Meanwhile, the model was further used to evaluate disease severity of COPD by predicting different grads. RESULTS The Ensemble model showed an AUC of 0.969 in distinguishing COPD by simultaneously extracting fusion features of clinical parameters and CXR images in internal test, better than models that used clinical parameters (AUC = 0.963) or images (AUC = 0.946) only. For the external test set, the AUC slightly declined to 0.934 in predicting COPD based on clinical parameters and CXR images. When applying the Ensemble model to determine disease severity of COPD, the AUC reached 0.894 for three-classification and 0.852 for five-classification respectively. CONCLUSION The present study used DL algorithms to screen COPD and predict disease severity based on CXR imaging and clinical parameters. The models showed good performance and the approach might be an effective case-finding tool with low radiation dose for COPD diagnosis and staging.
Collapse
Affiliation(s)
- XiaoLing Zou
- Department of Pulmonary and Critical Care Medicine, The Third Affiliated Hospital of Sun Yat-sen University, Institute of Respiratory Diseases of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, 510630, China
| | - Yong Ren
- Scientific research project department, Guangdong Artificial Intelligence and Digital Economy Laboratory (Guangzhou), Pazhou Lab, Guangzhou, China
- Shensi lab, Shenzhen Institute for Advanced Study, UESTC, Shenzhen, China
| | - HaiLing Yang
- Department of Pulmonary and Critical Care Medicine, The Third Affiliated Hospital of Sun Yat-sen University, Institute of Respiratory Diseases of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, 510630, China
| | - ManMan Zou
- Department of Pulmonary and Critical Care Medicine, Dongguan People's Hospital, Dongguan, China
| | - Ping Meng
- Department of Pulmonary and Critical Care Medicine, the Six Affiliated Hospital of Guangzhou Medical University, Qingyuan People's Hospital, Qingyuan, China
| | - LiYi Zhang
- Department of Pulmonary and Critical Care Medicine, The Third Affiliated Hospital of Sun Yat-sen University, Institute of Respiratory Diseases of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, 510630, China
| | - MingJuan Gong
- Department of Internal Medicine, Huazhou Hospital of Traditional Chinese Medicine, Huazhou, China
| | - WenWen Ding
- Department of Pulmonary and Critical Care Medicine, The Third Affiliated Hospital of Sun Yat-sen University, Institute of Respiratory Diseases of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, 510630, China
| | - LanQing Han
- Center for artificial intelligence in medicine, Research Institute of Tsinghua, Pearl River Delta, Guangzhou, China.
| | - TianTuo Zhang
- Department of Pulmonary and Critical Care Medicine, The Third Affiliated Hospital of Sun Yat-sen University, Institute of Respiratory Diseases of Sun Yat-Sen University, 600 Tianhe Road, Guangzhou, 510630, China.
| |
Collapse
|
5
|
Desale RP, Patil PS. An efficient multi-class classification of skin cancer using optimized vision transformer. Med Biol Eng Comput 2024; 62:773-789. [PMID: 37996627 DOI: 10.1007/s11517-023-02969-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 11/07/2023] [Indexed: 11/25/2023]
Abstract
Skin cancer is a pervasive and deadly disease, prompting a surge in research efforts towards utilizing computer-based techniques to analyze skin lesion images to identify malignancies. This paper introduces an optimized vision transformer approach for effectively classifying skin tumors. The methodology begins with a pre-processing step aimed at preserving color constancy, eliminating hair artifacts, and reducing image noise. Here, a combination of techniques such as piecewise linear bottom hat filtering, adaptive median filtering, Gaussian filtering, and an enhanced gradient intensity method is used for pre-processing. Afterwards, the segmentation phase is initiated using the self-sparse watershed algorithm on the pre-processed image. Subsequently, the segmented image is passed through a feature extraction stage where the hybrid Walsh-Hadamard Karhunen-Loeve expansion technique is employed. The final step involves the application of an improved vision transformer for skin cancer classification. The entire methodology is implemented using the Python programming language, and the International Skin Imaging Collaboration (ISIC) 2019 database is utilized for experimentation. The experimental results demonstrate remarkable performance with the different performance metrics is accuracy 99.81%, precision 96.65%, sensitivity 98.21%, F-measure 97.42%, specificity 99.88%, recall 98.21%, Jaccard coefficient 98.54%, and Mathew's correlation coefficient (MCC) 98.89%. The proposed methodology outperforms the existing methodology.
Collapse
Affiliation(s)
- R P Desale
- E&TC Engineering Department, SSVPS's Bapusaheb Shivajirao Deore College of Engineering, Dhule, Maharashtra, 424005, India.
| | - P S Patil
- E&TC Engineering Department, SSVPS's Bapusaheb Shivajirao Deore College of Engineering, Dhule, Maharashtra, 424005, India
| |
Collapse
|
6
|
Souid A, Alsubaie N, Soufiene BO, Alqahtani MS, Abbas M, Jambi LK, Sakli H. Improving diagnosis accuracy with an intelligent image retrieval system for lung pathologies detection: a features extractor approach. Sci Rep 2023; 13:16619. [PMID: 37789095 PMCID: PMC10547797 DOI: 10.1038/s41598-023-42366-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 09/09/2023] [Indexed: 10/05/2023] Open
Abstract
Detecting lung pathologies is critical for precise medical diagnosis. In the realm of diagnostic methods, various approaches, including imaging tests, physical examinations, and laboratory tests, contribute to this process. Of particular note, imaging techniques like X-rays, CT scans, and MRI scans play a pivotal role in identifying lung pathologies with their non-invasive insights. Deep learning, a subset of artificial intelligence, holds significant promise in revolutionizing the detection and diagnosis of lung pathologies. By leveraging expansive datasets, deep learning algorithms autonomously discern intricate patterns and features within medical images, such as chest X-rays and CT scans. These algorithms exhibit an exceptional capacity to recognize subtle markers indicative of lung diseases. Yet, while their potential is evident, inherent limitations persist. The demand for abundant labeled data during training and the susceptibility to data biases challenge their accuracy. To address these formidable challenges, this research introduces a tailored computer-assisted system designed for the automatic retrieval of annotated medical images that share similar content. At its core lies an intelligent deep learning-based features extractor, adept at simplifying the retrieval of analogous images from an extensive chest radiograph database. The crux of our innovation rests upon the fusion of YOLOv5 and EfficientNet within the features extractor module. This strategic fusion synergizes YOLOv5's rapid and efficient object detection capabilities with EfficientNet's proficiency in combating noisy predictions. The result is a distinctive amalgamation that redefines the efficiency and accuracy of features extraction. Through rigorous experimentation conducted on an extensive and diverse dataset, our proposed solution decisively surpasses conventional methodologies. The model's achievement of a mean average precision of 0.488 with a threshold of 0.9 stands as a testament to its effectiveness, overshadowing the results of YOLOv5 + ResNet and EfficientDet, which achieved 0.234 and 0.257 respectively. Furthermore, our model demonstrates a marked precision improvement, attaining a value of 0.864 across all pathologies-a noteworthy leap of approximately 0.352 compared to YOLOv5 + ResNet and EfficientDet. This research presents a significant stride toward enhancing radiologists' workflow efficiency, offering a refined and proficient tool for retrieving analogous annotated medical images.
Collapse
Affiliation(s)
- Abdelbaki Souid
- MACS Research Laboratory RL16ES22, National Engineering School of Gabes, Gabes, Tunisia
| | - Najah Alsubaie
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, 11671, Riyadh, Saudi Arabia
| | - Ben Othman Soufiene
- PRINCE Laboratory Research, ISITcom, University of Sousse, Hammam Sousse, Tunisia.
| | - Mohammed S Alqahtani
- Radiological Sciences Department, College of Applied Medical Sciences, King Khalid University, 61421, Abha, Saudi Arabia
- BioImaging Unit, Space Research Centre, Michael Atiyah Building, University of Leicester, Leicester, LE17RH, UK
| | - Mohamed Abbas
- Electrical Engineering Department, College of Engineering, King Khalid University, 61421, Abha, Saudi Arabia
| | - Layal K Jambi
- Radiological Sciences Department, College of Applied Medical Sciences, King Saud University, P.O. Box 10219, 11433, Riyadh, Saudi Arabia
| | - Hedi Sakli
- MACS Research Laboratory RL16ES22, National Engineering School of Gabes, Gabes, Tunisia
- EITA Consulting, 5 Rue Du Chant Des Oiseaux, 78360, Montesson, France
| |
Collapse
|
7
|
Tsuji T, Hirata Y, Kusunose K, Sata M, Kumagai S, Shiraishi K, Kotoku J. Classification of chest X-ray images by incorporation of medical domain knowledge into operation branch networks. BMC Med Imaging 2023; 23:62. [PMID: 37161392 PMCID: PMC10169130 DOI: 10.1186/s12880-023-01019-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 05/02/2023] [Indexed: 05/11/2023] Open
Abstract
BACKGROUND This study was conducted to alleviate a common difficulty in chest X-ray image diagnosis: The attention region in a convolutional neural network (CNN) does not often match the doctor's point of focus. The method presented herein, which guides the area of attention in CNN to a medically plausible region, can thereby improve diagnostic capabilities. METHODS The model is based on an attention branch network, which has excellent interpretability of the classification model. This model has an additional new operation branch that guides the attention region to the lung field and heart in chest X-ray images. We also used three chest X-ray image datasets (Teikyo, Tokushima, and ChestX-ray14) to evaluate the CNN attention area of interest in these fields. Additionally, after devising a quantitative method of evaluating improvement of a CNN's region of interest, we applied it to evaluation of the proposed model. RESULTS Operation branch networks maintain or improve the area under the curve to a greater degree than conventional CNNs do. Furthermore, the network better emphasizes reasonable anatomical parts in chest X-ray images. CONCLUSIONS The proposed network better emphasizes the reasonable anatomical parts in chest X-ray images. This method can enhance capabilities for image interpretation based on judgment.
Collapse
Affiliation(s)
- Takumasa Tsuji
- Graduate School of Medical Care and Technology, Teikyo University, 2-11-1 Kaga, Itabashi-Ku, Tokyo, 173-8605, Japan
| | - Yukina Hirata
- Ultrasound Examination Center, Tokushima University Hospital, 2-50-1, Kuramoto, Tokushima, Japan
| | - Kenya Kusunose
- Department of Cardiovascular Medicine, Tokushima University Hospital, 2-50-1, Kuramoto, Tokushima, Japan
| | - Masataka Sata
- Department of Cardiovascular Medicine, Tokushima University Hospital, 2-50-1, Kuramoto, Tokushima, Japan
| | - Shinobu Kumagai
- Central Radiology Division, Teikyo University Hospital, 2-11-1 Kaga, Itabashi-Ku, Tokyo, 173-8606, Japan
| | - Kenshiro Shiraishi
- Department of Radiology, Teikyo University School of Medicine, 2-11-1 Kaga, Itabashi-Ku, Tokyo, 173-8605, Japan
| | - Jun'ichi Kotoku
- Graduate School of Medical Care and Technology, Teikyo University, 2-11-1 Kaga, Itabashi-Ku, Tokyo, 173-8605, Japan.
- Central Radiology Division, Teikyo University Hospital, 2-11-1 Kaga, Itabashi-Ku, Tokyo, 173-8606, Japan.
| |
Collapse
|
8
|
Han Y, Holste G, Ding Y, Tewfik A, Peng Y, Wang Z. Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:750-761. [PMID: 36288235 PMCID: PMC10081959 DOI: 10.1109/tmi.2022.3217218] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Before the recent success of deep learning methods for automated medical image analysis, practitioners used handcrafted radiomic features to quantitatively describe local patches of medical images. However, extracting discriminative radiomic features relies on accurate pathology localization, which is difficult to acquire in real-world settings. Despite advances in disease classification and localization from chest X-rays, many approaches fail to incorporate clinically-informed domainspecific radiomic features. For these reasons, we propose a Radiomics-Guided Transformer (RGT) that fuses global image information with local radiomics-guided auxiliary information to provide accurate cardiopulmonary pathology localization and classification without any bounding box annotations. RGT consists of an image Transformer branch, a radiomics Transformer branch, and fusion layers that aggregate image and radiomics information. Using the learned self-attention of its image branch, RGT extracts a bounding box for which to compute radiomic features, which are further processed by the radiomics branch; learned image and radiomic features are then fused and mutually interact via cross-attention layers. Thus, RGT utilizes a novel end-to-end feedback loop that can bootstrap accurate pathology localization only using image-level disease labels. Experiments on the NIH ChestXRay dataset demonstrate that RGT outperforms prior works in weakly supervised disease localization (by an average margin of 3.6% over various intersection-over-union thresholds) and classification (by 1.1% in average area under the receiver operating characteristic curve). We publicly release our codes and pre-trained models at https://github.com/VITAGroup/chext.
Collapse
|