1
|
Csore J, Roy TL, Wright G, Karmonik C. Unsupervised classification of multi-contrast magnetic resonance histology of peripheral arterial disease lesions using a convolutional variational autoencoder with a Gaussian mixture model in latent space: A technical feasibility study. Comput Med Imaging Graph 2024; 115:102372. [PMID: 38581959 DOI: 10.1016/j.compmedimag.2024.102372] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Revised: 02/09/2024] [Accepted: 03/18/2024] [Indexed: 04/08/2024]
Abstract
PURPOSE To investigate the feasibility of a deep learning algorithm combining variational autoencoder (VAE) and two-dimensional (2D) convolutional neural networks (CNN) for automatically quantifying hard tissue presence and morphology in multi-contrast magnetic resonance (MR) images of peripheral arterial disease (PAD) occlusive lesions. METHODS Multi-contrast MR images (T2-weighted and ultrashort echo time) were acquired from lesions harvested from six amputated legs with high isotropic spatial resolution (0.078 mm and 0.156 mm, respectively) at 9.4 T. A total of 4014 pseudo-color combined images were generated, with 75% used to train a VAE employing custom 2D CNN layers. A Gaussian mixture model (GMM) was employed to classify the latent space data into four tissue classes: I) concentric calcified (c), II) eccentric calcified (e), III) occluded with hard tissue (h) and IV) occluded with soft tissue (s). Test image probabilities, encoded by the trained VAE were used to evaluate model performance. RESULTS GMM component classification probabilities ranged from 0.92 to 0.97 for class (c), 1.00 for class (e), 0.82-0.95 for class (h) and 0.56-0.93 for the remaining class (s). Due to the complexity of soft-tissue lesions reflected in the heterogeneity of the pseudo-color images, more GMM components (n=17) were attributed to class (s), compared to the other three (c, e and h) (n=6). CONCLUSION Combination of 2D CNN VAE and GMM achieves high classification probabilities for hard tissue-containing lesions. Automatic recognition of these classes may aid therapeutic decision-making and identifying uncrossable lesions prior to endovascular intervention.
Collapse
Affiliation(s)
- Judit Csore
- DeBakey Heart and Vascular Center, Houston Methodist Hospital, 6565 Fannin Street, Houston, TX 77030, USA; Heart and Vascular Center, Semmelweis University, 68 Városmajor Street, Budapest 1122, Hungary.
| | - Trisha L Roy
- DeBakey Heart and Vascular Center, Houston Methodist Hospital, 6565 Fannin Street, Houston, TX 77030, USA
| | - Graham Wright
- Sunnybrook Research Institute, 2075 Bayview Avenue, Toronto, Ontario M4N 3M5, Canada
| | - Christof Karmonik
- MRI Core, Translational Imaging Center, Houston Methodist Research Institute, 6670 Bertner Avenue, Houston, TX 77030, USA
| |
Collapse
|
2
|
M T, S K, Sagayam KM, J A. Detection and Segmentation of Glioma Tumors Utilizing a UNet Convolutional Neural Network Approach with Non-Subsampled Shearlet Transform. J Comput Biol 2024. [PMID: 38934096 DOI: 10.1089/cmb.2023.0339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/28/2024] Open
Abstract
The prompt and precise identification and delineation of tumor regions within glioma brain images are critical for mitigating the risks associated with this life-threatening ailment. In this study, we employ the UNet convolutional neural network (CNN) architecture for glioma tumor detection. Our proposed methodology comprises a transformation module, a feature extraction module, and a tumor segmentation module. The spatial domain representation of brain magnetic resonance imaging images undergoes decomposition into low- and high-frequency subbands via a non-subsampled shearlet transform. Leveraging the selective and directive characteristics of this transform enhances the classification efficacy of our proposed system. Shearlet features are extracted from both low- and high-frequency subbands and subsequently classified using the UNet-CNN architecture to identify tumor regions within glioma brain images. We validate our proposed glioma tumor detection methodology using publicly available datasets, namely Brain Tumor Segmentation (BRATS) 2019 and The Cancer Genome Atlas (TCGA). The mean classification rates achieved by our system are 99.1% for the BRATS 2019 dataset and 97.8% for the TCGA dataset. Furthermore, our system demonstrates notable performance metrics on the BRATS 2019 dataset, including 98.2% sensitivity, 98.7% specificity, 98.9% accuracy, 98.7% intersection over union, and 98.5% disc similarity coefficient. Similarly, on the TCGA dataset, our system achieves 97.7% sensitivity, 98.2% specificity, 98.7% accuracy, 98.6% intersection over union, and 98.4% disc similarity coefficient. Comparative analysis against state-of-the-art methods underscores the efficacy of our proposed glioma brain tumor detection approach.
Collapse
Affiliation(s)
- Tamilarasi M
- Department of Electronics and Communication Engineering, Sasurie College of Engineering, Tirupur, India
| | - Kumarganesh S
- Department of Electronics and Communication Engineering, Knowledge Institute of Technology, Salem, India
| | - K Martin Sagayam
- Department of Electronics and Communication Engineering, Karunya Institute of Technology and Sciences, Coimbatore, India
| | - Andrew J
- Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India
| |
Collapse
|
3
|
Yuan J, Siakallis L, Li HB, Brandner S, Zhang J, Li C, Mancini L, Bisdas S. Structural- and DTI- MRI enable automated prediction of IDH Mutation Status in CNS WHO Grade 2-4 glioma patients: a deep Radiomics Approach. BMC Med Imaging 2024; 24:104. [PMID: 38702613 PMCID: PMC11067215 DOI: 10.1186/s12880-024-01274-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 04/15/2024] [Indexed: 05/06/2024] Open
Abstract
BACKGROUND The role of isocitrate dehydrogenase (IDH) mutation status for glioma stratification and prognosis is established. While structural magnetic resonance image (MRI) is a promising biomarker, it may not be sufficient for non-invasive characterisation of IDH mutation status. We investigated the diagnostic value of combined diffusion tensor imaging (DTI) and structural MRI enhanced by a deep radiomics approach based on convolutional neural networks (CNNs) and support vector machine (SVM), to determine the IDH mutation status in Central Nervous System World Health Organization (CNS WHO) grade 2-4 gliomas. METHODS This retrospective study analyzed the DTI-derived fractional anisotropy (FA) and mean diffusivity (MD) images and structural images including fluid attenuated inversion recovery (FLAIR), non-enhanced T1-, and T2-weighted images of 206 treatment-naïve gliomas, including 146 IDH mutant and 60 IDH-wildtype ones. The lesions were manually segmented by experienced neuroradiologists and the masks were applied to the FA and MD maps. Deep radiomics features were extracted from each subject by applying a pre-trained CNN and statistical description. An SVM classifier was applied to predict IDH status using imaging features in combination with demographic data. RESULTS We comparatively assessed the CNN-SVM classifier performance in predicting IDH mutation status using standalone and combined structural and DTI-based imaging features. Combined imaging features surpassed stand-alone modalities for the prediction of IDH mutation status [area under the curve (AUC) = 0.846; sensitivity = 0.925; and specificity = 0.567]. Importantly, optimal model performance was noted following the addition of demographic data (patients' age) to structural and DTI imaging features [area under the curve (AUC) = 0.847; sensitivity = 0.911; and specificity = 0.617]. CONCLUSIONS Imaging features derived from DTI-based FA and MD maps combined with structural MRI, have superior diagnostic value to that provided by standalone structural or DTI sequences. In combination with demographic information, this CNN-SVM model offers a further enhanced non-invasive prediction of IDH mutation status in gliomas.
Collapse
Affiliation(s)
- Jialin Yuan
- Department of Radiology, Shenzhen People's Hospital, Second Clinical Medical College of Jinan University, First Affiliated Hospital of Southern University of Science and Technology, Shenzhen, China
- Queen Square Institute of Neurology, University College London, London, UK
| | - Loizos Siakallis
- Queen Square Institute of Neurology, University College London, London, UK
| | - Hongwei Bran Li
- Department of Informatics, Technical University of Munich, Munich, Germany
- Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, USA
| | - Sebastian Brandner
- Division of Neuropathology, Queen Square Institute of Neurology, University College London, London, UK
| | - Jianguo Zhang
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
| | - Chenming Li
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
| | - Laura Mancini
- Queen Square Institute of Neurology, University College London, London, UK
- Lysholm Department of Neuroradiology, University College London Hospitals NHS Foundation Trust, London, UK
| | - Sotirios Bisdas
- Queen Square Institute of Neurology, University College London, London, UK.
- Lysholm Department of Neuroradiology, University College London Hospitals NHS Foundation Trust, London, UK.
| |
Collapse
|
4
|
Elhadad A, Jamjoom M, Abulkasim H. Reduction of NIFTI files storage and compression to facilitate telemedicine services based on quantization hiding of downsampling approach. Sci Rep 2024; 14:5168. [PMID: 38431641 PMCID: PMC10908832 DOI: 10.1038/s41598-024-54820-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 02/16/2024] [Indexed: 03/05/2024] Open
Abstract
Magnetic resonance imaging is a medical imaging technique to create comprehensive images of the tissues and organs in the body. This study presents an advanced approach for storing and compressing neuroimaging informatics technology initiative files, a standard format in magnetic resonance imaging. It is designed to enhance telemedicine services by facilitating efficient and high-quality communication between healthcare practitioners and patients. The proposed downsampling approach begins by opening the neuroimaging informatics technology initiative file as volumetric data and then planning it into several slice images. Then, the quantization hiding technique will be applied to each of the two consecutive slice images to generate the stego slice with the same size. This involves the following major steps: normalization, microblock generation, and discrete cosine transformation. Finally, it assembles the resultant stego slice images to produce the final neuroimaging informatics technology initiative file as volumetric data. The upsampling process, designed to be completely blind, reverses the downsampling steps to reconstruct the subsequent image slice accurately. The efficacy of the proposed method was evaluated using a magnetic resonance imaging dataset, focusing on peak signal-to-noise ratio, signal-to-noise ratio, structural similarity index, and Entropy as key performance metrics. The results demonstrate that the proposed approach not only significantly reduces file sizes but also maintains high image quality.
Collapse
Affiliation(s)
- Ahmed Elhadad
- Department of Computer Science, Faculty of Computers and Information, South Valley University, Qena, Egypt
| | - Mona Jamjoom
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia
| | - Hussein Abulkasim
- Department of Mathematics and Computer Science, Faculty of Science, New Valley University, El-Kharja, Egypt.
- College of Engineering and Technology, University of Science and Technology of Fujairah, Fujairah, United Arab Emirates.
| |
Collapse
|
5
|
Zeng Q, Sun J, Wang S. DIC-Transformer: interpretation of plant disease classification results using image caption generation technology. FRONTIERS IN PLANT SCIENCE 2024; 14:1273029. [PMID: 38333041 PMCID: PMC10850568 DOI: 10.3389/fpls.2023.1273029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2023] [Accepted: 12/29/2023] [Indexed: 02/10/2024]
Abstract
Disease image classification systems play a crucial role in identifying disease categories in the field of agricultural diseases. However, current plant disease image classification methods can only predict the disease category and do not offer explanations for the characteristics of the predicted disease images. Due to the current situation, this paper employed image description generation technology to produce distinct descriptions for different plant disease categories. A two-stage model called DIC-Transformer, which encompasses three tasks (detection, interpretation, and classification), was proposed. In the first stage, Faster R-CNN was utilized to detect the diseased area and generate the feature vector of the diseased image, with the Swin Transformer as the backbone. In the second stage, the model utilized the Transformer to generate image captions. It then generated the image feature vector, which is weighted by text features, to improve the performance of image classification in the subsequent classification decoder. Additionally, a dataset containing text and visualizations for agricultural diseases (ADCG-18) was compiled. The dataset contains images of 18 diseases and descriptive information about their characteristics. Then, using the ADCG-18, the DIC-Transformer was compared to 11 existing classical caption generation methods and 10 image classification models. The evaluation indicators for captions include Bleu1-4, CiderD, and Rouge. The values of BLEU-1, CIDEr-D, and ROUGE were 0.756, 450.51, and 0.721. The results of DIC-Transformer were 0.01, 29.55, and 0.014 higher than those of the highest-performing comparison model, Fc. The classification evaluation metrics include accuracy, recall, and F1 score, with accuracy at 0.854, recall at 0.854, and F1 score at 0.853. The results of DIC-Transformer were 0.024, 0.078, and 0.075 higher than those of the highest-performing comparison model, MobileNetV2. The results indicate that the DIC-Transformer outperforms other comparison models in classification and caption generation.
Collapse
Affiliation(s)
| | | | - Shansong Wang
- College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, China
| |
Collapse
|
6
|
Zhou H, Wu S, Xu Z, Sun H. Automatic detection of standing dead trees based on improved YOLOv7 from airborne remote sensing imagery. FRONTIERS IN PLANT SCIENCE 2024; 15:1278161. [PMID: 38318496 PMCID: PMC10839092 DOI: 10.3389/fpls.2024.1278161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 01/05/2024] [Indexed: 02/07/2024]
Abstract
Detecting and localizing standing dead trees (SDTs) is crucial for effective forest management and conservation. Due to challenges posed by mountainous terrain and road conditions, conducting a swift and comprehensive survey of SDTs through traditional manual inventory methods is considerably difficult. In recent years, advancements in deep learning and remote sensing technology have facilitated real-time and efficient detection of dead trees. Nevertheless, challenges persist in identifying individual dead trees in airborne remote sensing images, attributed to factors such as small target size, mutual occlusion and complex backgrounds. These aspects collectively contribute to the increased difficulty of detecting dead trees at a single-tree scale. To address this issue, the paper introduces an improved You Only Look Once version 7 (YOLOv7) model that incorporates the Simple Parameter-Free Attention Module (SimAM), an unparameterized attention mechanism. This improvement aims to enhance the network's feature extraction capabilities and increase the model's sensitivity to small target dead trees. To validate the superiority of SimAM_YOLOv7, we compared it with four widely adopted attention mechanisms. Additionally, a method to enhance model robustness is presented, involving the replacement of the Complete Intersection over Union (CIoU) loss in the original YOLOv7 model with the Wise-IoU (WIoU) loss function. Following these, we evaluated detection accuracy using a self-developed dataset of SDTs in forests. The results indicate that the improved YOLOv7 model can effectively identify dead trees in airborne remote sensing images, achieving precision, recall and mAP@0.5 values of 94.31%, 93.13% and 98.03%, respectively. These values are 3.67%, 2.28% and 1.56% higher than those of the original YOLOv7 model. This improvement model provides a convenient solution for forest management.
Collapse
Affiliation(s)
- Hongwei Zhou
- College of Computer and Control Engineering, Northeast Forestry University, Harbin, China
| | - Shangxin Wu
- College of Computer and Control Engineering, Northeast Forestry University, Harbin, China
| | - Zihan Xu
- College of Computer and Control Engineering, Northeast Forestry University, Harbin, China
| | - Hong Sun
- Key Laboratory of National Forestry and Grassland Administration on Forest and Grassland Pest Monitoring and Warning, Center for Biological Disaster Prevention and Control, National Forestry and Grassland Administration, Shenyang, China
| |
Collapse
|
7
|
Lee JS, Shin K, Ryu SM, Jegal SG, Lee W, Yoon MA, Hong GS, Paik S, Kim N. Screening of adolescent idiopathic scoliosis using generative adversarial network (GAN) inversion method in chest radiographs. PLoS One 2023; 18:e0285489. [PMID: 37216382 DOI: 10.1371/journal.pone.0285489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 04/25/2023] [Indexed: 05/24/2023] Open
Abstract
OBJECTIVE Conventional computer-aided diagnosis using convolutional neural networks (CNN) has limitations in detecting sensitive changes and determining accurate decision boundaries in spectral and structural diseases such as scoliosis. We devised a new method to detect and diagnose adolescent idiopathic scoliosis in chest X-rays (CXRs) employing the latent space's discriminative ability in the generative adversarial network (GAN) and a simple multi-layer perceptron (MLP) to screen adolescent idiopathic scoliosis CXRs. MATERIALS AND METHODS Our model was trained and validated in a two-step manner. First, we trained a GAN using CXRs with various scoliosis severities and utilized the trained network as a feature extractor using the GAN inversion method. Second, we classified each vector from the latent space using a simple MLP. RESULTS The 2-layer MLP exhibited the best classification in the ablation study. With this model, the area under the receiver operating characteristic (AUROC) curves were 0.850 in the internal and 0.847 in the external datasets. Furthermore, when the sensitivity was fixed at 0.9, the model's specificity was 0.697 in the internal and 0.646 in the external datasets. CONCLUSION We developed a classifier for Adolescent idiopathic scoliosis (AIS) through generative representation learning. Our model shows good AUROC under screening chest radiographs in both the internal and external datasets. Our model has learned the spectral severity of AIS, enabling it to generate normal images even when trained solely on scoliosis radiographs.
Collapse
Affiliation(s)
- Jun Soo Lee
- Department of Industrial Engineering, Seoul National University, Seoul, Korea
| | - Keewon Shin
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Seung Min Ryu
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
- Department of Orthopedic Surgery, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Korea
| | - Seong Gyu Jegal
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Woojin Lee
- Department of Radiology, Hanyang University Hospital, Seoul, Korea
| | - Min A Yoon
- Department of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Gil-Sun Hong
- Department of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Korea
| | - Sanghyun Paik
- Department of Radiology, Hanyang University Hospital, Seoul, Korea
| | - Namkug Kim
- Department of Convergence Medicine, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| |
Collapse
|