1
|
Veras Florentino PT, Araújo VDO, Zatti H, Luis CV, Cavalcanti CRS, de Oliveira MHC, Leão AHFF, Bertoldo Junior J, Barbosa GGC, Ravera E, Cebukin A, David RB, de Melo DBV, Machado TM, Bellei NCJ, Boaventura V, Barral-Netto M, Smaili SS. Text mining method to unravel long COVID's clinical condition in hospitalized patients. Cell Death Dis 2024; 15:671. [PMID: 39271699 PMCID: PMC11399332 DOI: 10.1038/s41419-024-07043-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2024] [Revised: 08/28/2024] [Accepted: 08/29/2024] [Indexed: 09/15/2024]
Abstract
Long COVID is characterized by persistent that extends symptoms beyond established timeframes. Its varied presentation across different populations and healthcare systems poses significant challenges in understanding its clinical manifestations and implications. In this study, we present a novel application of text mining technique to automatically extract unstructured data from a long COVID survey conducted at a prominent university hospital in São Paulo, Brazil. Our phonetic text clustering (PTC) method enables the exploration of unstructured Electronic Healthcare Records (EHR) data to unify different written forms of similar terms into a single phonemic representation. We used n-gram text analysis to detect compound words and negated terms in Portuguese-BR, focusing on medical conditions and symptoms related to long COVID. By leveraging text mining, we aim to contribute to a deeper understanding of this chronic condition and its implications for healthcare systems globally. The model developed in this study has the potential for scalability and applicability in other healthcare settings, thereby supporting broader research efforts and informing clinical decision-making for long COVID patients.
Collapse
Affiliation(s)
- Pilar Tavares Veras Florentino
- Laboratório de Medicina e Saúde Pública de Precisão (MeSP2), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
- Centro de Integração de Dados e Conhecimentos para a Saúde (CIDACS), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
| | - Vinícius de Oliveira Araújo
- Centro de Integração de Dados e Conhecimentos para a Saúde (CIDACS), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
- Faculdade de Medicina da Bahia, Universidade Federal da Bahia, Salvador, Brazil
| | - Henrique Zatti
- Centro de Integração de Dados e Conhecimentos para a Saúde (CIDACS), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
| | - Caio Vinícius Luis
- Departamento de Farmacologia, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | | | | | | | - Juracy Bertoldo Junior
- Centro de Integração de Dados e Conhecimentos para a Saúde (CIDACS), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
| | - George G Caique Barbosa
- Centro de Integração de Dados e Conhecimentos para a Saúde (CIDACS), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
| | - Ernesto Ravera
- Departamento de Farmacologia, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Alberto Cebukin
- Departamento de Farmacologia, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Renata Bernardes David
- Departamento de Farmacologia, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | | | - Tales Mota Machado
- Centro de Integração de Dados e Conhecimentos para a Saúde (CIDACS), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
- Diretoria de Tecnologia da Informação, Universidade Federal de Ouro Preto, Ouro Preto, Brazil
| | - Nancy C J Bellei
- Disciplina de Moléstias Infecciosas, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Viviane Boaventura
- Laboratório de Medicina e Saúde Pública de Precisão (MeSP2), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil
- Faculdade de Medicina da Bahia, Universidade Federal da Bahia, Salvador, Brazil
| | - Manoel Barral-Netto
- Laboratório de Medicina e Saúde Pública de Precisão (MeSP2), Instituto Gonçalo Moniz, Fundação Oswaldo Cruz, Salvador, Brazil.
- Faculdade de Medicina da Bahia, Universidade Federal da Bahia, Salvador, Brazil.
| | - Soraya S Smaili
- Departamento de Farmacologia, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil.
| |
Collapse
|
2
|
Huang Q, Li G. Knowledge graph based reasoning in medical image analysis: A scoping review. Comput Biol Med 2024; 182:109100. [PMID: 39244959 DOI: 10.1016/j.compbiomed.2024.109100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2024] [Revised: 08/04/2024] [Accepted: 08/31/2024] [Indexed: 09/10/2024]
Abstract
Automated computer-aided diagnosis (CAD) is becoming more significant in the field of medicine due to advancements in computer hardware performance and the progress of artificial intelligence. The knowledge graph is a structure for visually representing knowledge facts. In the last decade, a large body of work based on knowledge graphs has effectively improved the organization and interpretability of large-scale complex knowledge. Introducing knowledge graph inference into CAD is a research direction with significant potential. In this review, we briefly review the basic principles and application methods of knowledge graphs firstly. Then, we systematically organize and analyze the research and application of knowledge graphs in medical imaging-assisted diagnosis. We also summarize the shortcomings of the current research, such as medical data barriers and deficiencies, low utilization of multimodal information, and weak interpretability. Finally, we propose future research directions with possibilities and potentials to address the shortcomings of current approaches.
Collapse
Affiliation(s)
- Qinghua Huang
- School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, 127 West Youyi Road, Beilin District, Xi'an, 710072, Shaanxi, China.
| | - Guanghui Li
- School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, 127 West Youyi Road, Beilin District, Xi'an, 710072, Shaanxi, China; School of Computer Science, Northwestern Polytechnical University, 1 Dongxiang Road, Chang'an District, Xi'an, 710129, Shaanxi, China.
| |
Collapse
|
3
|
Kamalakannan N, Macharla SR, Kanimozhi M, Sudhakar MS. Exponential Pixelating Integral transform with dual fractal features for enhanced chest X-ray abnormality detection. Comput Biol Med 2024; 182:109093. [PMID: 39232407 DOI: 10.1016/j.compbiomed.2024.109093] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 08/25/2024] [Accepted: 08/29/2024] [Indexed: 09/06/2024]
Abstract
The heightened prevalence of respiratory disorders, particularly exacerbated by a significant upswing in fatalities due to the novel coronavirus, underscores the critical need for early detection and timely intervention. This imperative is paramount, possessing the potential to profoundly impact and safeguard numerous lives. Medically, chest radiography stands out as an essential and economically viable medical imaging approach for diagnosing and assessing the severity of diverse Respiratory Disorders. However, their detection in Chest X-Rays is a cumbersome task even for well-trained radiologists owing to low contrast issues, overlapping of the tissue structures, subjective variability, and the presence of noise. To address these issues, a novel analytical model termed Exponential Pixelating Integral is introduced for the automatic detection of infections in Chest X-Rays in this work. Initially, the presented Exponential Pixelating Integral enhances the pixel intensities to overcome the low-contrast issues that are then polar-transformed followed by their representation using the locally invariant Mandelbrot and Julia fractal geometries for effective distinction of structural features. The collated features labeled Exponential Pixelating Integral with dually characterized fractal features are then classified by the non-parametric multivariate adaptive regression splines to establish an ensemble model between each pair of classes for effective diagnosis of diverse diseases. Rigorous analysis of the proposed classification framework on large medical benchmarked datasets showcases its superiority over its peers by registering a higher classification accuracy and F1 scores ranging from 98.46 to 99.45 % and 96.53-98.10 % respectively, making it a precise and interpretable automated system for diagnosing respiratory disorders.
Collapse
Affiliation(s)
| | | | - M Kanimozhi
- School of Electrical & Electronics, Sathyabama Institute of Science and Technology, Chennai, Tamilnadu, India
| | - M S Sudhakar
- School of Electronics Engineering, Vellore Institute of Technology, Vellore, Tamilnadu, India.
| |
Collapse
|
4
|
Ejiyi CJ, Qin Z, Ejiyi MB, Ukwuoma C, Ejiyi TU, Muoka GW, Gyarteng ESA, Bamisile OO. MACCoM: A multiple attention and convolutional cross-mixer framework for detailed 2D biomedical image segmentation. Comput Biol Med 2024; 179:108847. [PMID: 39004046 DOI: 10.1016/j.compbiomed.2024.108847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Revised: 06/28/2024] [Accepted: 07/02/2024] [Indexed: 07/16/2024]
Abstract
The UNet architecture, which is widely used for biomedical image segmentation, has limitations like blurred feature maps and over- or under-segmented regions. To overcome these limitations, we propose a novel network architecture called MACCoM (Multiple Attention and Convolutional Cross-Mixer) - an end-to-end depthwise encoder-decoder fully convolutional network designed for binary and multi-class biomedical image segmentation built upon deeperUNet. We proposed a multi-scope attention module (MSAM) that allows the model to attend to diverse scale features, preserving fine details and high-level semantic information thus useful at the encoder-decoder connection. As the depth increases, our proposed spatial multi-head attention (SMA) is added to facilitate inter-layer communication and information exchange, enabling the network to effectively capture long-range dependencies and global context. MACCoM is also equipped with a convolutional cross-mixer we proposed to enhance the feature extraction capability of the model. By incorporating these modules, we effectively combine semantically similar features and reduce artifacts during the early stages of training. Experimental results on 4 biomedical datasets crafted from 3 datasets of varying modalities consistently demonstrate that MACCoM outperforms or matches state-of-the-art baselines in the segmentation tasks. With Breast Ultrasound Image (BUSI), MACCoM recorded 99.06 % Jaccard, 77.58 % Dice, and 93.92 % Accuracy, while recording 99.50 %, 98.44 %, and 99.29 % respectively for Jaccard, Dice, and Accuracy on the Chest X-ray (CXR) images used. The Jaccard, Dice, and Accuracy for the High-Resolution Fundus (HRF) images are 95.77 %, 74.35 %, and 95.95 % respectively. The findings here highlight MACCoM's effectiveness in improving segmentation performance and its valuable potential in image analysis.
Collapse
Affiliation(s)
- Chukwuebuka Joseph Ejiyi
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, Sichuan, China; Sichuan Industrial Internet Intelligent Monitoring and Application Engineering Research Center, Chengdu University of Technology, Chengdu, China
| | - Zhen Qin
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, Sichuan, China.
| | | | - Chiagoziem Ukwuoma
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, Sichuan, China
| | - Thomas Ugochukwu Ejiyi
- Department of Pure and Industrial Chemistry, University of Nigeria Nsukka, Nsukka, Enugu State, Nigeria
| | - Gladys Wavinya Muoka
- Network and Data Security Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu, Sichuan, China
| | - Emmanuel S A Gyarteng
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Olusola O Bamisile
- Sichuan Industrial Internet Intelligent Monitoring and Application Engineering Research Center, Chengdu University of Technology, Chengdu, China
| |
Collapse
|
5
|
Yan C, Yan H, Liang W, Yin M, Luo H, Luo J. DP-SSLoRA: A privacy-preserving medical classification model combining differential privacy with self-supervised low-rank adaptation. Comput Biol Med 2024; 179:108792. [PMID: 38964242 DOI: 10.1016/j.compbiomed.2024.108792] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 05/31/2024] [Accepted: 06/18/2024] [Indexed: 07/06/2024]
Abstract
BACKGROUND AND OBJECTIVE Concerns about patient privacy issues have limited the application of medical deep learning models in certain real-world scenarios. Differential privacy (DP) can alleviate this problem by injecting random noise into the model. However, naively applying DP to medical models will not achieve a satisfactory balance between privacy and utility due to the high dimensionality of medical models and the limited labeled samples. METHODS This work proposed the DP-SSLoRA model, a privacy-preserving classification model for medical images combining differential privacy with self-supervised low-rank adaptation. In this work, a self-supervised pre-training method is used to obtain enhanced representations from unlabeled publicly available medical data. Then, a low-rank decomposition method is employed to mitigate the impact of differentially private noise and combined with pre-trained features to conduct the classification task on private datasets. RESULTS In the classification experiments using three real chest-X ray datasets, DP-SSLoRA achieves good performance with strong privacy guarantees. Under the premise of ɛ=2, with the AUC of 0.942 in RSNA, the AUC of 0.9658 in Covid-QU-mini, and the AUC of 0.9886 in Chest X-ray 15k. CONCLUSION Extensive experiments on real chest X-ray datasets show that DP-SSLoRA can achieve satisfactory performance with stronger privacy guarantees. This study provides guidance for studying privacy-preserving in the medical field. Source code is publicly available online. https://github.com/oneheartforone/DP-SSLoRA.
Collapse
Affiliation(s)
- Chaokun Yan
- School of Computer and Information Engineering, Henan University, Kaifeng, 475004, Henan, China; Academy for Advanced Interdisciplinary Studies, Henan University, Kaifeng, 475004, Henan, China; Henan Engineering Research Center of Intelligent Technology and Application, Henan University, Kaifeng, 475004, Henan, China
| | - Haicao Yan
- School of Computer and Information Engineering, Henan University, Kaifeng, 475004, Henan, China
| | - Wenjuan Liang
- School of Computer and Information Engineering, Henan University, Kaifeng, 475004, Henan, China; Academy for Advanced Interdisciplinary Studies, Henan University, Kaifeng, 475004, Henan, China; Henan Engineering Research Center of Intelligent Technology and Application, Henan University, Kaifeng, 475004, Henan, China.
| | - Menghan Yin
- School of Computer and Information Engineering, Henan University, Kaifeng, 475004, Henan, China
| | - Huimin Luo
- School of Computer and Information Engineering, Henan University, Kaifeng, 475004, Henan, China; Academy for Advanced Interdisciplinary Studies, Henan University, Kaifeng, 475004, Henan, China; Henan Engineering Research Center of Intelligent Technology and Application, Henan University, Kaifeng, 475004, Henan, China
| | - Junwei Luo
- School of Software, Henan Polytecgnic University, Jiaozuo, 454000, Henan, China
| |
Collapse
|
6
|
Abdel-Salam M, Hu G, Çelik E, Gharehchopogh FS, El-Hasnony IM. Chaotic RIME optimization algorithm with adaptive mutualism for feature selection problems. Comput Biol Med 2024; 179:108803. [PMID: 38955125 DOI: 10.1016/j.compbiomed.2024.108803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 05/17/2024] [Accepted: 06/24/2024] [Indexed: 07/04/2024]
Abstract
The RIME optimization algorithm is a newly developed physics-based optimization algorithm used for solving optimization problems. The RIME algorithm proved high-performing in various fields and domains, providing a high-performance solution. Nevertheless, like many swarm-based optimization algorithms, RIME suffers from many limitations, including the exploration-exploitation balance not being well balanced. In addition, the likelihood of falling into local optimal solutions is high, and the convergence speed still needs some work. Hence, there is room for enhancement in the search mechanism so that various search agents can discover new solutions. The authors suggest an adaptive chaotic version of the RIME algorithm named ACRIME, which incorporates four main improvements, including an intelligent population initialization using chaotic maps, a novel adaptive modified Symbiotic Organism Search (SOS) mutualism phase, a novel mixed mutation strategy, and the utilization of restart strategy. The main goal of these improvements is to improve the variety of the population, achieve a better balance between exploration and exploitation, and improve RIME's local and global search abilities. The study assesses the effectiveness of ACRIME by using the standard benchmark functions of the CEC2005 and CEC2019 benchmarks. The proposed ACRIME is also applied as a feature selection to fourteen various datasets to test its applicability to real-world problems. Besides, the ACRIME algorithm is applied to the COVID-19 classification real problem to test its applicability and performance further. The suggested algorithm is compared to other sophisticated classical and advanced metaheuristics, and its performance is assessed using statistical tests such as Wilcoxon rank-sum and Friedman rank tests. The study demonstrates that ACRIME exhibits a high level of competitiveness and often outperforms competing algorithms. It discovers the optimal subset of features, enhancing the accuracy of classification and minimizing the number of features employed. This study primarily focuses on enhancing the equilibrium between exploration and exploitation, extending the scope of local search.
Collapse
Affiliation(s)
- Mahmoud Abdel-Salam
- Faculty of Computer and Information Science, Mansoura University, Mansoura, 35516, Egypt.
| | - Gang Hu
- Department of Applied Mathematics, Xi'an University of Technology, Xi'an, 710054, PR China
| | - Emre Çelik
- Department of Electrical and Electronics Engineering, Faculty of Engineering, Düzce University, Düzce, Turkey
| | | | - Ibrahim M El-Hasnony
- Faculty of Computer and Information Science, Mansoura University, Mansoura, 35516, Egypt
| |
Collapse
|
7
|
Ding H, Fan L, Zhang J, Gao G. Deep Learning-Based System Combining Chest X-Ray and Computerized Tomography Images for COVID-19 Diagnosis. Br J Hosp Med (Lond) 2024; 85:1-15. [PMID: 39212565 DOI: 10.12968/hmed.2024.0244] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/04/2024]
Abstract
Aims/Background: The coronavirus disease 2019 (COVID-19) pandemic has highlighted the need for accurate and efficient diagnostic methods. This study aims to improve COVID-19 detection by integrating chest X-ray (CXR) and computerized tomography (CT) images using deep learning techniques, further improving diagnostic accuracy by using a combined imaging approach. Methods: The study used two publicly accessible databases, COVID-19 Questionnaires for Understanding the Exposure (COVID-QU-Ex) and Integrated Clinical and Translational Cancer Foundation (iCTCF), containing CXR and CT images, respectively. The proposed system employed convolutional neural networks (CNNs) for classification, specifically EfficientNet and ResNet architectures. The data underwent preprocessing steps, including image resizing, Gaussian noise addition, and data augmentation. The dataset was divided into training, validation, and test sets. Gradient-weighted Class Activation Mapping (Grad-CAM) was used for model interpretability. Results: The EfficientNet-based models outperformed the ResNet-based models across all metrics. The highest accuracy achieved was 99.44% for CXR images and 99.81% for CT images with EfficientNetB5. The models also demonstrated high precision, recall, and F1 scores. For statistical significance, the p-values were less than 0.05, indicating that the results are significant. Conclusion: Integrating CXR and CT images using deep learning significantly improves the accuracy of COVID-19 diagnosis. The EfficientNet-based models, with their superior feature extraction capabilities, show better performance than ResNet models. Grad-CAM Visualizations provide insights into the model's decision-making process, potentially reducing diagnostic errors and accelerating diagnosis processes. This approach can improve patient care and support healthcare systems in managing the pandemic more effectively.
Collapse
Affiliation(s)
- Hui Ding
- Department of Radiology, Ningbo No.2 Hospital, Ningbo, Zhejiang, China
| | - Lingyan Fan
- Department of Acute Infectious Diseases, Ningbo No.2 Hospital, Ningbo, Zhejiang, China
| | - Jingfeng Zhang
- Department of Radiology, Ningbo No.2 Hospital, Ningbo, Zhejiang, China
| | - Guosheng Gao
- Department of Clinical Laboratory, Ningbo No.2 Hospital, Ningbo, Zhejiang, China
| |
Collapse
|
8
|
Singh T, Mishra S, Kalra R, Satakshi, Kumar M, Kim T. COVID-19 severity detection using chest X-ray segmentation and deep learning. Sci Rep 2024; 14:19846. [PMID: 39191941 PMCID: PMC11349901 DOI: 10.1038/s41598-024-70801-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2024] [Accepted: 08/21/2024] [Indexed: 08/29/2024] Open
Abstract
COVID-19 has resulted in a significant global impact on health, the economy, education, and daily life. The disease can range from mild to severe, with individuals over 65 or those with underlying medical conditions being more susceptible to severe illness. Early testing and isolation are vital due to the virus's variable incubation period. Chest radiographs (CXR) have gained importance as a diagnostic tool due to their efficiency and reduced radiation exposure compared to CT scans. However, the sensitivity of CXR in detecting COVID-19 may be lower. This paper introduces a deep learning framework for accurate COVID-19 classification and severity prediction using CXR images. U-Net is used for lung segmentation, achieving a precision of 0.9924. Classification is performed using a Convulation-capsule network, with high true positive rates of 86% for COVID-19, 93% for pneumonia, and 85% for normal cases. Severity assessment employs ResNet50, VGG-16, and DenseNet201, with DenseNet201 showing superior accuracy. Empirical results, validated with 95% confidence intervals, confirm the framework's reliability and robustness. This integration of advanced deep learning techniques with radiological imaging enhances early detection and severity assessment, improving patient management and resource allocation in clinical settings.
Collapse
Affiliation(s)
- Tinku Singh
- School of Information and Communication Engineering, Chungbuk National University, Cheongju, South Korea
| | - Suryanshi Mishra
- Department of Mathematics & Statistics, SHUATS, Prayagraj, Uttar Pradesh, India
| | - Riya Kalra
- Indian Institute of Information Technology Allahabad, Prayagraj, Uttar Pradesh, India
| | - Satakshi
- Department of Mathematics & Statistics, SHUATS, Prayagraj, Uttar Pradesh, India
| | - Manish Kumar
- Indian Institute of Information Technology Allahabad, Prayagraj, Uttar Pradesh, India
| | - Taehong Kim
- School of Information and Communication Engineering, Chungbuk National University, Cheongju, South Korea.
| |
Collapse
|
9
|
Sannasi Chakravarthy SR, Bharanidharan N, Vinothini C, Vinoth Kumar V, Mahesh TR, Guluwadi S. Adaptive Mish activation and ranger optimizer-based SEA-ResNet50 model with explainable AI for multiclass classification of COVID-19 chest X-ray images. BMC Med Imaging 2024; 24:206. [PMID: 39123118 PMCID: PMC11313131 DOI: 10.1186/s12880-024-01394-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2024] [Accepted: 08/06/2024] [Indexed: 08/12/2024] Open
Abstract
A recent global health crisis, COVID-19 is a significant global health crisis that has profoundly affected lifestyles. The detection of such diseases from similar thoracic anomalies using medical images is a challenging task. Thus, the requirement of an end-to-end automated system is vastly necessary in clinical treatments. In this way, the work proposes a Squeeze-and-Excitation Attention-based ResNet50 (SEA-ResNet50) model for detecting COVID-19 utilizing chest X-ray data. Here, the idea lies in improving the residual units of ResNet50 using the squeeze-and-excitation attention mechanism. For further enhancement, the Ranger optimizer and adaptive Mish activation function are employed to improve the feature learning of the SEA-ResNet50 model. For evaluation, two publicly available COVID-19 radiographic datasets are utilized. The chest X-ray input images are augmented during experimentation for robust evaluation against four output classes namely normal, pneumonia, lung opacity, and COVID-19. Then a comparative study is done for the SEA-ResNet50 model against VGG-16, Xception, ResNet18, ResNet50, and DenseNet121 architectures. The proposed framework of SEA-ResNet50 together with the Ranger optimizer and adaptive Mish activation provided maximum classification accuracies of 98.38% (multiclass) and 99.29% (binary classification) as compared with the existing CNN architectures. The proposed method achieved the highest Kappa validation scores of 0.975 (multiclass) and 0.98 (binary classification) over others. Furthermore, the visualization of the saliency maps of the abnormal regions is represented using the explainable artificial intelligence (XAI) model, thereby enhancing interpretability in disease diagnosis.
Collapse
Affiliation(s)
- S R Sannasi Chakravarthy
- Department of Electronics and Communication Engineering, Bannari Amman Institute of Technology, Sathyamangalam, India
| | - N Bharanidharan
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, 632014, India
| | - C Vinothini
- Department of Computer Science and Engineering, Dayananda Sagar College of Engineering, Bangalore, India
| | - Venkatesan Vinoth Kumar
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, 632014, India
| | - T R Mahesh
- Department of Computer Science and Engineering, JAIN (Deemed-to-Be University), Bengaluru, 562112, India
| | - Suresh Guluwadi
- Adama Science and Technology University, Adama, 302120, Ethiopia.
| |
Collapse
|
10
|
Rahman MF, Tseng TL(B, Pokojovy M, McCaffrey P, Walser E, Moen S, Vo A, Ho JC. Machine-Learning-Enabled Diagnostics with Improved Visualization of Disease Lesions in Chest X-ray Images. Diagnostics (Basel) 2024; 14:1699. [PMID: 39202188 PMCID: PMC11353848 DOI: 10.3390/diagnostics14161699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2024] [Revised: 07/31/2024] [Accepted: 08/02/2024] [Indexed: 09/03/2024] Open
Abstract
The class activation map (CAM) represents the neural-network-derived region of interest, which can help clarify the mechanism of the convolutional neural network's determination of any class of interest. In medical imaging, it can help medical practitioners diagnose diseases like COVID-19 or pneumonia by highlighting the suspicious regions in Computational Tomography (CT) or chest X-ray (CXR) film. Many contemporary deep learning techniques only focus on COVID-19 classification tasks using CXRs, while few attempt to make it explainable with a saliency map. To fill this research gap, we first propose a VGG-16-architecture-based deep learning approach in combination with image enhancement, segmentation-based region of interest (ROI) cropping, and data augmentation steps to enhance classification accuracy. Later, a multi-layer Gradient CAM (ML-Grad-CAM) algorithm is integrated to generate a class-specific saliency map for improved visualization in CXR images. We also define and calculate a Severity Assessment Index (SAI) from the saliency map to quantitatively measure infection severity. The trained model achieved an accuracy score of 96.44% for the three-class CXR classification task, i.e., COVID-19, pneumonia, and normal (healthy patients), outperforming many existing techniques in the literature. The saliency maps generated from the proposed ML-GRAD-CAM algorithm are compared with the original Gran-CAM algorithm.
Collapse
Affiliation(s)
- Md Fashiar Rahman
- Department of Industrial, Manufacturing and Systems Engineering, The University of Texas, El Paso, TX 79968, USA
| | - Tzu-Liang (Bill) Tseng
- Department of Industrial, Manufacturing and Systems Engineering, The University of Texas, El Paso, TX 79968, USA
| | - Michael Pokojovy
- Department of Mathematics and Statistics, Old Dominion University, Norfolk, VA 23529, USA;
| | - Peter McCaffrey
- Department of Radiology, The University of Texas Medical Branch, Galveston, TX 77550, USA; (P.M.); (E.W.); (S.M.); (A.V.)
| | - Eric Walser
- Department of Radiology, The University of Texas Medical Branch, Galveston, TX 77550, USA; (P.M.); (E.W.); (S.M.); (A.V.)
| | - Scott Moen
- Department of Radiology, The University of Texas Medical Branch, Galveston, TX 77550, USA; (P.M.); (E.W.); (S.M.); (A.V.)
| | - Alex Vo
- Department of Radiology, The University of Texas Medical Branch, Galveston, TX 77550, USA; (P.M.); (E.W.); (S.M.); (A.V.)
| | - Johnny C. Ho
- Department of Management and Marketing, Turner College of Business, Columbus State University, Columbus, GA 31907, USA;
| |
Collapse
|
11
|
Lamprou V, Kallipolitis A, Maglogiannis I. On the evaluation of deep learning interpretability methods for medical images under the scope of faithfulness. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 253:108238. [PMID: 38823117 DOI: 10.1016/j.cmpb.2024.108238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 05/01/2024] [Accepted: 05/21/2024] [Indexed: 06/03/2024]
Abstract
BACKGROUND AND OBJECTIVE Evaluating the interpretability of Deep Learning models is crucial for building trust and gaining insights into their decision-making processes. In this work, we employ class activation map based attribution methods in a setting where only High-Resolution Class Activation Mapping (HiResCAM) is known to produce faithful explanations. The objective is to evaluate the quality of the attribution maps using quantitative metrics and investigate whether faithfulness aligns with the metrics results. METHODS We fine-tune pre-trained deep learning architectures over four medical image datasets in order to calculate attribution maps. The maps are evaluated on a threefold metrics basis utilizing well-established evaluation scores. RESULTS Our experimental findings suggest that the Area Over Perturbation Curve (AOPC) and Max-Sensitivity scores favor the HiResCAM maps. On the other hand, the Heatmap Assisted Accuracy Score (HAAS) does not provide insights to our comparison as it evaluates almost all maps as inaccurate. To this purpose we further compare our calculated values against values obtained over a diverse group of models which are trained on non-medical benchmark datasets, to eventually achieve more responsive results. CONCLUSION This study develops a series of experiments to discuss the connection between faithfulness and quantitative metrics over medical attribution maps. HiResCAM preserves the gradient effect on a pixel level ultimately producing high-resolution, informative and resilient mappings. In turn, this is depicted in the results of AOPC and Max-Sensitivity metrics, successfully identifying the faithful algorithm. In regards to HAAS, our experiments yield that it is sensitive over complex medical patterns, commonly characterized by strong color dependency and multiple attention areas.
Collapse
Affiliation(s)
- Vangelis Lamprou
- Department of Digital Systems, University of Piraeus, 80, M. Karaoli & A. Dimitriou St, Piraeus 18534, Greece
| | - Athanasios Kallipolitis
- Department of Digital Systems, University of Piraeus, 80, M. Karaoli & A. Dimitriou St, Piraeus 18534, Greece.
| | - Ilias Maglogiannis
- Department of Digital Systems, University of Piraeus, 80, M. Karaoli & A. Dimitriou St, Piraeus 18534, Greece
| |
Collapse
|
12
|
Ye RZ, Lipatov K, Diedrich D, Bhattacharyya A, Erickson BJ, Pickering BW, Herasevich V. Automatic ARDS surveillance with chest X-ray recognition using convolutional neural networks. J Crit Care 2024; 82:154794. [PMID: 38552452 DOI: 10.1016/j.jcrc.2024.154794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 11/20/2023] [Accepted: 12/01/2023] [Indexed: 06/01/2024]
Abstract
OBJECTIVE This study aims to design, validate and assess the accuracy a deep learning model capable of differentiation Chest X-Rays between pneumonia, acute respiratory distress syndrome (ARDS) and normal lungs. MATERIALS AND METHODS A diagnostic performance study was conducted using Chest X-Ray images from adult patients admitted to a medical intensive care unit between January 2003 and November 2014. X-ray images from 15,899 patients were assigned one of three prespecified categories: "ARDS", "Pneumonia", or "Normal". RESULTS A two-step convolutional neural network (CNN) pipeline was developed and tested to distinguish between the three patterns with sensitivity ranging from 91.8% to 97.8% and specificity ranging from 96.6% to 98.8%. The CNN model was validated with a sensitivity of 96.3% and specificity of 96.6% using a previous dataset of patients with Acute Lung Injury (ALI)/ARDS. DISCUSSION The results suggest that a deep learning model based on chest x-ray pattern recognition can be a useful tool in distinguishing patients with ARDS from patients with normal lungs, providing faster results than digital surveillance tools based on text reports. CONCLUSION A CNN-based deep learning model showed clinically significant performance, providing potential for faster ARDS identification. Future research should prospectively evaluate these tools in a clinical setting.
Collapse
Affiliation(s)
- Run Zhou Ye
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA.; Division of Endocrinology, Department of Medicine, Centre de Recherche du CHUS, Sherbrooke QC J1H 5N4, Canada
| | - Kirill Lipatov
- Critical Care Medicine, Mayo Clinic, Eau Claire, WI, United States
| | - Daniel Diedrich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | | | - Bradley J Erickson
- Department of Diagnostic Radiology, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA..
| |
Collapse
|
13
|
Li Y, Xin Y, Li X, Zhang Y, Liu C, Cao Z, Du S, Wang L. Omni-dimensional dynamic convolution feature coordinate attention network for pneumonia classification. Vis Comput Ind Biomed Art 2024; 7:17. [PMID: 38976189 PMCID: PMC11231110 DOI: 10.1186/s42492-024-00168-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Accepted: 06/22/2024] [Indexed: 07/09/2024] Open
Abstract
Pneumonia is a serious disease that can be fatal, particularly among children and the elderly. The accuracy of pneumonia diagnosis can be improved by combining artificial-intelligence technology with X-ray imaging. This study proposes X-ODFCANet, which addresses the issues of low accuracy and excessive parameters in existing deep-learning-based pneumonia-classification methods. This network incorporates a feature coordination attention module and an omni-dimensional dynamic convolution (ODConv) module, leveraging the residual module for feature extraction from X-ray images. The feature coordination attention module utilizes two one-dimensional feature encoding processes to aggregate feature information from different spatial directions. Additionally, the ODConv module extracts and fuses feature information in four dimensions: the spatial dimension of the convolution kernel, input and output channel quantities, and convolution kernel quantity. The experimental results demonstrate that the proposed method can effectively improve the accuracy of pneumonia classification, which is 3.77% higher than that of ResNet18. The model parameters are 4.45M, which was reduced by approximately 2.5 times. The code is available at https://github.com/limuni/X-ODFCANET .
Collapse
Affiliation(s)
- Yufei Li
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China
| | - Yufei Xin
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China
| | - Xinni Li
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China
| | - Yinrui Zhang
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China
| | - Cheng Liu
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China
| | - Zhengwen Cao
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China
| | - Shaoyi Du
- Department of Ultrasound, the Second Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi Province, 710004, China.
- National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, Shaanxi Province, 710049, China.
| | - Lin Wang
- School of Information Science and Technology, Northwest University, Xi'an, 710127, Shaanxi Province, China.
| |
Collapse
|
14
|
Azevedo KS, de Souza LC, Coutinho MGF, de M Barbosa R, Fernandes MAC. Deepvirusclassifier: a deep learning tool for classifying SARS-CoV-2 based on viral subtypes within the coronaviridae family. BMC Bioinformatics 2024; 25:231. [PMID: 38969970 PMCID: PMC11225326 DOI: 10.1186/s12859-024-05754-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 03/19/2024] [Indexed: 07/07/2024] Open
Abstract
PURPOSE In this study, we present DeepVirusClassifier, a tool capable of accurately classifying Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) viral sequences among other subtypes of the coronaviridae family. This classification is achieved through a deep neural network model that relies on convolutional neural networks (CNNs). Since viruses within the same family share similar genetic and structural characteristics, the classification process becomes more challenging, necessitating more robust models. With the rapid evolution of viral genomes and the increasing need for timely classification, we aimed to provide a robust and efficient tool that could increase the accuracy of viral identification and classification processes. Contribute to advancing research in viral genomics and assist in surveilling emerging viral strains. METHODS Based on a one-dimensional deep CNN, the proposed tool is capable of training and testing on the Coronaviridae family, including SARS-CoV-2. Our model's performance was assessed using various metrics, including F1-score and AUROC. Additionally, artificial mutation tests were conducted to evaluate the model's generalization ability across sequence variations. We also used the BLAST algorithm and conducted comprehensive processing time analyses for comparison. RESULTS DeepVirusClassifier demonstrated exceptional performance across several evaluation metrics in the training and testing phases. Indicating its robust learning capacity. Notably, during testing on more than 10,000 viral sequences, the model exhibited a more than 99% sensitivity for sequences with fewer than 2000 mutations. The tool achieves superior accuracy and significantly reduced processing times compared to the Basic Local Alignment Search Tool algorithm. Furthermore, the results appear more reliable than the work discussed in the text, indicating that the tool has great potential to revolutionize viral genomic research. CONCLUSION DeepVirusClassifier is a powerful tool for accurately classifying viral sequences, specifically focusing on SARS-CoV-2 and other subtypes within the Coronaviridae family. The superiority of our model becomes evident through rigorous evaluation and comparison with existing methods. Introducing artificial mutations into the sequences demonstrates the tool's ability to identify variations and significantly contributes to viral classification and genomic research. As viral surveillance becomes increasingly critical, our model holds promise in aiding rapid and accurate identification of emerging viral strains.
Collapse
Affiliation(s)
- Karolayne S Azevedo
- InovAI Lab, nPITI/IMD, Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil
| | - Luísa C de Souza
- InovAI Lab, nPITI/IMD, Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil
| | - Maria G F Coutinho
- InovAI Lab, nPITI/IMD, Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil
| | - Raquel de M Barbosa
- InovAI Lab, nPITI/IMD, Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil.
- Department of Pharmacy and Pharmaceutical Technology, University of Seville, 41012, Seville, Spain.
| | - Marcelo A C Fernandes
- InovAI Lab, nPITI/IMD, Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil.
- Bioinformatics Multidisciplinary Environment (BioME), Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil.
- Department of Computer Engineering and Automation (DCA), Federal University of Rio Grande do Norte, Natal, RN, 59078-970, Brazil.
| |
Collapse
|
15
|
Guan H, Yap PT, Bozoki A, Liu M. Federated learning for medical image analysis: A survey. PATTERN RECOGNITION 2024; 151:110424. [PMID: 38559674 PMCID: PMC10976951 DOI: 10.1016/j.patcog.2024.110424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. We have systematically gathered research papers on federated learning and its applications in medical image analysis published between 2017 and 2023. Our search and compilation were conducted using databases from IEEE Xplore, ACM Digital Library, Science Direct, Springer Link, Web of Science, Google Scholar, and PubMed. In this survey, we first introduce the background of federated learning for dealing with privacy protection and collaborative learning issues. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field.
Collapse
Affiliation(s)
- Hao Guan
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Pew-Thian Yap
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Andrea Bozoki
- Department of Neurology, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
16
|
Lee SB. Development of a chest X-ray machine learning convolutional neural network model on a budget and using artificial intelligence explainability techniques to analyze patterns of machine learning inference. JAMIA Open 2024; 7:ooae035. [PMID: 38699648 PMCID: PMC11064095 DOI: 10.1093/jamiaopen/ooae035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 04/03/2024] [Accepted: 04/10/2024] [Indexed: 05/05/2024] Open
Abstract
Objective Machine learning (ML) will have a large impact on medicine and accessibility is important. This study's model was used to explore various concepts including how varying features of a model impacted behavior. Materials and Methods This study built an ML model that classified chest X-rays as normal or abnormal by using ResNet50 as a base with transfer learning. A contrast enhancement mechanism was implemented to improve performance. After training with a dataset of publicly available chest radiographs, performance metrics were determined with a test set. The ResNet50 base was substituted with deeper architectures (ResNet101/152) and visualization methods used to help determine patterns of inference. Results Performance metrics were an accuracy of 79%, recall 69%, precision 96%, and area under the curve of 0.9023. Accuracy improved to 82% and recall to 74% with contrast enhancement. When visualization methods were applied and the ratio of pixels used for inference measured, deeper architectures resulted in the model using larger portions of the image for inference as compared to ResNet50. Discussion The model performed on par with many existing models despite consumer-grade hardware and smaller datasets. Individual models vary thus a single model's explainability may not be generalizable. Therefore, this study varied architecture and studied patterns of inference. With deeper ResNet architectures, the machine used larger portions of the image to make decisions. Conclusion An example using a custom model showed that AI (Artificial Intelligence) can be accessible on consumer-grade hardware, and it also demonstrated an example of studying themes of ML explainability by varying ResNet architectures.
Collapse
Affiliation(s)
- Stephen B Lee
- Division of Infectious Diseases, Department of Medicine, College of Medicine, University of Saskatchewan, Regina, S4P 0W5, Canada
| |
Collapse
|
17
|
Kanwal K, Asif M, Khalid SG, Liu H, Qurashi AG, Abdullah S. Current Diagnostic Techniques for Pneumonia: A Scoping Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:4291. [PMID: 39001069 PMCID: PMC11244398 DOI: 10.3390/s24134291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/29/2024] [Revised: 06/22/2024] [Accepted: 06/28/2024] [Indexed: 07/16/2024]
Abstract
Community-acquired pneumonia is one of the most lethal infectious diseases, especially for infants and the elderly. Given the variety of causative agents, the accurate early detection of pneumonia is an active research area. To the best of our knowledge, scoping reviews on diagnostic techniques for pneumonia are lacking. In this scoping review, three major electronic databases were searched and the resulting research was screened. We categorized these diagnostic techniques into four classes (i.e., lab-based methods, imaging-based techniques, acoustic-based techniques, and physiological-measurement-based techniques) and summarized their recent applications. Major research has been skewed towards imaging-based techniques, especially after COVID-19. Currently, chest X-rays and blood tests are the most common tools in the clinical setting to establish a diagnosis; however, there is a need to look for safe, non-invasive, and more rapid techniques for diagnosis. Recently, some non-invasive techniques based on wearable sensors achieved reasonable diagnostic accuracy that could open a new chapter for future applications. Consequently, further research and technology development are still needed for pneumonia diagnosis using non-invasive physiological parameters to attain a better point of care for pneumonia patients.
Collapse
Affiliation(s)
- Kehkashan Kanwal
- College of Speech, Language, and Hearing Sciences, Ziauddin University, Karachi 75000, Pakistan
| | - Muhammad Asif
- Faculty of Computing and Applied Sciences, Sir Syed University of Engineering and Technology, Karachi 75300, Pakistan
| | - Syed Ghufran Khalid
- Department of Engineering, Faculty of Science and Technology, Nottingham Trent University, Nottingham B15 3TN, UK
| | - Haipeng Liu
- Research Centre for Intelligent Healthcare, Coventry University, Coventry CV1 5FB, UK
| | | | - Saad Abdullah
- School of Innovation, Design and Engineering, Mälardalen University, 721 23 Västerås, Sweden
| |
Collapse
|
18
|
Brima Y, Atemkeng M. Saliency-driven explainable deep learning in medical imaging: bridging visual explainability and statistical quantitative analysis. BioData Min 2024; 17:18. [PMID: 38909228 PMCID: PMC11193223 DOI: 10.1186/s13040-024-00370-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 06/10/2024] [Indexed: 06/24/2024] Open
Abstract
Deep learning shows great promise for medical image analysis but often lacks explainability, hindering its adoption in healthcare. Attribution techniques that explain model reasoning can potentially increase trust in deep learning among clinical stakeholders. In the literature, much of the research on attribution in medical imaging focuses on visual inspection rather than statistical quantitative analysis.In this paper, we proposed an image-based saliency framework to enhance the explainability of deep learning models in medical image analysis. We use adaptive path-based gradient integration, gradient-free techniques, and class activation mapping along with its derivatives to attribute predictions from brain tumor MRI and COVID-19 chest X-ray datasets made by recent deep convolutional neural network models.The proposed framework integrates qualitative and statistical quantitative assessments, employing Accuracy Information Curves (AICs) and Softmax Information Curves (SICs) to measure the effectiveness of saliency methods in retaining critical image information and their correlation with model predictions. Visual inspections indicate that methods such as ScoreCAM, XRAI, GradCAM, and GradCAM++ consistently produce focused and clinically interpretable attribution maps. These methods highlighted possible biomarkers, exposed model biases, and offered insights into the links between input features and predictions, demonstrating their ability to elucidate model reasoning on these datasets. Empirical evaluations reveal that ScoreCAM and XRAI are particularly effective in retaining relevant image regions, as reflected in their higher AUC values. However, SICs highlight variability, with instances of random saliency masks outperforming established methods, emphasizing the need for combining visual and empirical metrics for a comprehensive evaluation.The results underscore the importance of selecting appropriate saliency methods for specific medical imaging tasks and suggest that combining qualitative and quantitative approaches can enhance the transparency, trustworthiness, and clinical adoption of deep learning models in healthcare. This study advances model explainability to increase trust in deep learning among healthcare stakeholders by revealing the rationale behind predictions. Future research should refine empirical metrics for stability and reliability, include more diverse imaging modalities, and focus on improving model explainability to support clinical decision-making.
Collapse
Affiliation(s)
- Yusuf Brima
- Computer Vision, Institute of Cognitive Science, Osnabrück University, Osnabrueck, D-49090, Lower Saxony, Germany.
| | - Marcellin Atemkeng
- Department of Mathematics, Rhodes University, Grahamstown, 6140, Eastern Cape, South Africa.
| |
Collapse
|
19
|
Qiu Y, Liu Y, Li S, Xu J. MiniSeg: An Extremely Minimum Network Based on Lightweight Multiscale Learning for Efficient COVID-19 Segmentation. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:8570-8584. [PMID: 37015641 DOI: 10.1109/tnnls.2022.3230821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
The rapid spread of the new pandemic, i.e., coronavirus disease 2019 (COVID-19), has severely threatened global health. Deep-learning-based computer-aided screening, e.g., COVID-19 infected area segmentation from computed tomography (CT) image, has attracted much attention by serving as an adjunct to increase the accuracy of COVID-19 screening and clinical diagnosis. Although lesion segmentation is a hot topic, traditional deep learning methods are usually data-hungry with millions of parameters, easy to overfit under limited available COVID-19 training data. On the other hand, fast training/testing and low computational cost are also necessary for quick deployment and development of COVID-19 screening systems, but traditional methods are usually computationally intensive. To address the above two problems, we propose MiniSeg, a lightweight model for efficient COVID-19 segmentation from CT images. Our efforts start with the design of an attentive hierarchical spatial pyramid (AHSP) module for lightweight, efficient, effective multiscale learning that is essential for image segmentation. Then, we build a two-path (TP) encoder for deep feature extraction, where one path uses AHSP modules for learning multiscale contextual features and the other is a shallow convolutional path for capturing fine details. The two paths interact with each other for learning effective representations. Based on the extracted features, a simple decoder is added for COVID-19 segmentation. For comparing MiniSeg to previous methods, we build a comprehensive COVID-19 segmentation benchmark. Extensive experiments demonstrate that the proposed MiniSeg achieves better accuracy because its only 83k parameters make it less prone to overfitting. Its high efficiency also makes it easy to deploy and develop. The code has been released at https://github.com/yun-liu/MiniSeg.
Collapse
|
20
|
Zhou Z, Islam MT, Xing L. Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:7351-7362. [PMID: 37028335 DOI: 10.1109/tnnls.2023.3250490] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.
Collapse
|
21
|
Aasem M, Javed Iqbal M. Toward explainable AI in radiology: Ensemble-CAM for effective thoracic disease localization in chest X-ray images using weak supervised learning. Front Big Data 2024; 7:1366415. [PMID: 38756502 PMCID: PMC11096460 DOI: 10.3389/fdata.2024.1366415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2024] [Accepted: 04/08/2024] [Indexed: 05/18/2024] Open
Abstract
Chest X-ray (CXR) imaging is widely employed by radiologists to diagnose thoracic diseases. Recently, many deep learning techniques have been proposed as computer-aided diagnostic (CAD) tools to assist radiologists in minimizing the risk of incorrect diagnosis. From an application perspective, these models have exhibited two major challenges: (1) They require large volumes of annotated data at the training stage and (2) They lack explainable factors to justify their outcomes at the prediction stage. In the present study, we developed a class activation mapping (CAM)-based ensemble model, called Ensemble-CAM, to address both of these challenges via weakly supervised learning by employing explainable AI (XAI) functions. Ensemble-CAM utilizes class labels to predict the location of disease in association with interpretable features. The proposed work leverages ensemble and transfer learning with class activation functions to achieve three objectives: (1) minimizing the dependency on strongly annotated data when locating thoracic diseases, (2) enhancing confidence in predicted outcomes by visualizing their interpretable features, and (3) optimizing cumulative performance via fusion functions. Ensemble-CAM was trained on three CXR image datasets and evaluated through qualitative and quantitative measures via heatmaps and Jaccard indices. The results reflect the enhanced performance and reliability in comparison to existing standalone and ensembled models.
Collapse
Affiliation(s)
- Muhammad Aasem
- Department of Computer Science, University of Engineering and Technology, Taxila, Pakistan
| | | |
Collapse
|
22
|
Liu Z, Shen L. CECT: Controllable ensemble CNN and transformer for COVID-19 image classification. Comput Biol Med 2024; 173:108388. [PMID: 38569235 DOI: 10.1016/j.compbiomed.2024.108388] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 03/15/2024] [Accepted: 03/25/2024] [Indexed: 04/05/2024]
Abstract
The COVID-19 pandemic has resulted in hundreds of million cases and numerous deaths worldwide. Here, we develop a novel classification network CECT by controllable ensemble convolutional neural network and transformer to provide a timely and accurate COVID-19 diagnosis. The CECT is composed of a parallel convolutional encoder block, an aggregate transposed-convolutional decoder block, and a windowed attention classification block. Each block captures features at different scales from 28 × 28 to 224 × 224 from the input, composing enriched and comprehensive information. Different from existing methods, our CECT can capture features at both multi-local and global scales without any sophisticated module design. Moreover, the contribution of local features at different scales can be controlled with the proposed ensemble coefficients. We evaluate CECT on two public COVID-19 datasets and it reaches the highest accuracy of 98.1% in the intra-dataset evaluation, outperforming existing state-of-the-art methods. Moreover, the developed CECT achieves an accuracy of 90.9% on the unseen dataset in the inter-dataset evaluation, showing extraordinary generalization ability. With remarkable feature capture ability and generalization ability, we believe CECT can be extended to other medical scenarios as a powerful diagnosis tool. Code is available at https://github.com/NUS-Tim/CECT.
Collapse
Affiliation(s)
- Zhaoshan Liu
- Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117575, Singapore.
| | - Lei Shen
- Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117575, Singapore.
| |
Collapse
|
23
|
Rana S, Hosen MJ, Tonni TJ, Rony MAH, Fatema K, Hasan MZ, Rahman MT, Khan RT, Jan T, Whaiduzzaman M. DeepChestGNN: A Comprehensive Framework for Enhanced Lung Disease Identification through Advanced Graphical Deep Features. SENSORS (BASEL, SWITZERLAND) 2024; 24:2830. [PMID: 38732936 PMCID: PMC11086108 DOI: 10.3390/s24092830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/02/2024] [Revised: 04/06/2024] [Accepted: 04/16/2024] [Indexed: 05/13/2024]
Abstract
Lung diseases are the third-leading cause of mortality in the world. Due to compromised lung function, respiratory difficulties, and physiological complications, lung disease brought on by toxic substances, pollution, infections, or smoking results in millions of deaths every year. Chest X-ray images pose a challenge for classification due to their visual similarity, leading to confusion among radiologists. To imitate those issues, we created an automated system with a large data hub that contains 17 datasets of chest X-ray images for a total of 71,096, and we aim to classify ten different disease classes. For combining various resources, our large datasets contain noise and annotations, class imbalances, data redundancy, etc. We conducted several image pre-processing techniques to eliminate noise and artifacts from images, such as resizing, de-annotation, CLAHE, and filtering. The elastic deformation augmentation technique also generates a balanced dataset. Then, we developed DeepChestGNN, a novel medical image classification model utilizing a deep convolutional neural network (DCNN) to extract 100 significant deep features indicative of various lung diseases. This model, incorporating Batch Normalization, MaxPooling, and Dropout layers, achieved a remarkable 99.74% accuracy in extensive trials. By combining graph neural networks (GNNs) with feedforward layers, the architecture is very flexible when it comes to working with graph data for accurate lung disease classification. This study highlights the significant impact of combining advanced research with clinical application potential in diagnosing lung diseases, providing an optimal framework for precise and efficient disease identification and classification.
Collapse
Affiliation(s)
- Shakil Rana
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka 1207, Bangladesh; (S.R.); (M.J.H.); (T.J.T.); (M.A.H.R.); (K.F.); (M.Z.H.)
| | - Md Jabed Hosen
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka 1207, Bangladesh; (S.R.); (M.J.H.); (T.J.T.); (M.A.H.R.); (K.F.); (M.Z.H.)
| | - Tasnim Jahan Tonni
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka 1207, Bangladesh; (S.R.); (M.J.H.); (T.J.T.); (M.A.H.R.); (K.F.); (M.Z.H.)
| | - Md. Awlad Hossen Rony
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka 1207, Bangladesh; (S.R.); (M.J.H.); (T.J.T.); (M.A.H.R.); (K.F.); (M.Z.H.)
| | - Kaniz Fatema
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka 1207, Bangladesh; (S.R.); (M.J.H.); (T.J.T.); (M.A.H.R.); (K.F.); (M.Z.H.)
| | - Md. Zahid Hasan
- Health Informatics Research Laboratory (HIRL), Department of Computer Science and Engineering, Daffodil International University, Dhaka 1207, Bangladesh; (S.R.); (M.J.H.); (T.J.T.); (M.A.H.R.); (K.F.); (M.Z.H.)
| | - Md. Tanvir Rahman
- School of Health and Rehabilitation Sciences, The University of Queensland, St. Lucia, QLD 4072, Australia
- Department of Information and Communication Technology, Mawlana Bhashani Science and Technology University, Tangail 1902, Bangladesh
| | - Risala Tasin Khan
- Institute of Information Technology, Jahangirnagar University, Dhaka 1342, Bangladesh;
| | - Tony Jan
- Centre for Artificial Intelligence Research and Optimisation (AIRO), Torrens University, Ultimo, NSW 2007, Australia;
| | - Md Whaiduzzaman
- Centre for Artificial Intelligence Research and Optimisation (AIRO), Torrens University, Ultimo, NSW 2007, Australia;
- School of Information Systems, Queensland University of Technology, Brisbane, QLD 4000, Australia
| |
Collapse
|
24
|
Althenayan AS, AlSalamah SA, Aly S, Nouh T, Mahboub B, Salameh L, Alkubeyyer M, Mirza A. COVID-19 Hierarchical Classification Using a Deep Learning Multi-Modal. SENSORS (BASEL, SWITZERLAND) 2024; 24:2641. [PMID: 38676257 PMCID: PMC11053684 DOI: 10.3390/s24082641] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 04/15/2024] [Accepted: 04/18/2024] [Indexed: 04/28/2024]
Abstract
Coronavirus disease 2019 (COVID-19), originating in China, has rapidly spread worldwide. Physicians must examine infected patients and make timely decisions to isolate them. However, completing these processes is difficult due to limited time and availability of expert radiologists, as well as limitations of the reverse-transcription polymerase chain reaction (RT-PCR) method. Deep learning, a sophisticated machine learning technique, leverages radiological imaging modalities for disease diagnosis and image classification tasks. Previous research on COVID-19 classification has encountered several limitations, including binary classification methods, single-feature modalities, small public datasets, and reliance on CT diagnostic processes. Additionally, studies have often utilized a flat structure, disregarding the hierarchical structure of pneumonia classification. This study aims to overcome these limitations by identifying pneumonia caused by COVID-19, distinguishing it from other types of pneumonia and healthy lungs using chest X-ray (CXR) images and related tabular medical data, and demonstrate the value of incorporating tabular medical data in achieving more accurate diagnoses. Resnet-based and VGG-based pre-trained convolutional neural network (CNN) models were employed to extract features, which were then combined using early fusion for the classification of eight distinct classes. We leveraged the hierarchal structure of pneumonia classification within our approach to achieve improved classification outcomes. Since an imbalanced dataset is common in this field, a variety of versions of generative adversarial networks (GANs) were used to generate synthetic data. The proposed approach tested in our private datasets of 4523 patients achieved a macro-avg F1-score of 95.9% and an F1-score of 87.5% for COVID-19 identification using a Resnet-based structure. In conclusion, in this study, we were able to create an accurate deep learning multi-modal to diagnose COVID-19 and differentiate it from other kinds of pneumonia and normal lungs, which will enhance the radiological diagnostic process.
Collapse
Affiliation(s)
- Albatoul S. Althenayan
- Information Systems Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia; (S.A.A.); (A.M.)
- Information Systems Department, College of Computer and Information Sciences, Imam Mohammed Bin Saud Islamic University, Riyadh 11432, Saudi Arabia
| | - Shada A. AlSalamah
- Information Systems Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia; (S.A.A.); (A.M.)
- National Health Information Center, Saudi Health Council, Riyadh 13315, Saudi Arabia
- Digital Health and Innovation Department, Science Division, World Health Organization, 1211 Geneva, Switzerland
| | - Sherin Aly
- Institute of Graduate Studies and Research, Alexandria University, Alexandria 21526, Egypt;
| | - Thamer Nouh
- Trauma and Acute Care Surgery Unit, College of Medicine, King Saud University, Riyadh 12271, Saudi Arabia;
| | - Bassam Mahboub
- Clinical Sciences Department, College of Medicine, University of Sharjah, Sharjah P.O. Box 27272, United Arab Emirates;
| | - Laila Salameh
- Sharjah Institute for Medical Research, University of Sharjah, Sharjah P.O. Box 27272, United Arab Emirates;
| | - Metab Alkubeyyer
- Department of Radiology and Medical Imaging, King Khalid University Hospital, King Saud University, Riyadh 12372, Saudi Arabia;
| | - Abdulrahman Mirza
- Information Systems Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia; (S.A.A.); (A.M.)
| |
Collapse
|
25
|
Arefin MS, Rahman MM, Hasan MT, Mahmud M. A Topical Review on Enabling Technologies for the Internet of Medical Things: Sensors, Devices, Platforms, and Applications. MICROMACHINES 2024; 15:479. [PMID: 38675290 PMCID: PMC11051832 DOI: 10.3390/mi15040479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 03/17/2024] [Accepted: 03/22/2024] [Indexed: 04/28/2024]
Abstract
The Internet of Things (IoT) is still a relatively new field of research, and its potential to be used in the healthcare and medical sectors is enormous. In the last five years, IoT has been a go-to option for various applications such as using sensors for different features, machine-to-machine communication, etc., but precisely in the medical sector, it is still lagging far behind compared to other sectors. Hence, this study emphasises IoT applications in medical fields, Medical IoT sensors and devices, IoT platforms for data visualisation, and artificial intelligence in medical applications. A systematic review considering PRISMA guidelines on research articles as well as the websites on IoMT sensors and devices has been carried out. After the year 2001, an integrated outcome of 986 articles was initially selected, and by applying the inclusion-exclusion criterion, a total of 597 articles were identified. 23 new studies have been finally found, including records from websites and citations. This review then analyses different sensor monitoring circuits in detail, considering an Intensive Care Unit (ICU) scenario, device applications, and the data management system, including IoT platforms for the patients. Lastly, detailed discussion and challenges have been outlined, and possible prospects have been presented.
Collapse
Affiliation(s)
- Md. Shamsul Arefin
- Department of Electrical and Electronic Engineering (EEE), Bangladesh University of Business & Technology, Dhaka 1216, Bangladesh;
| | | | - Md. Tanvir Hasan
- Department of Electrical and Electronic Engineering (EEE), Jashore University of Science & Technology, Jashore 7408, Bangladesh;
- Department of Electrical Engineering, University of South Carolina, Columbia, SC 29208, USA
| | - Mufti Mahmud
- Department of Computer Science, Nottingham Trent University, Nottingham NG11 8NS, UK
- Computing and Informatics Research Centre, Nottingham Trent University, Nottingham NG11 8NS, UK
- Medical Technologies Innovation Facility, Nottingham Trent University, Nottingham NG11 8NS, UK
| |
Collapse
|
26
|
Kabir MM, Mridha M, Rahman A, Hamid MA, Monowar MM. Detection of COVID-19, pneumonia, and tuberculosis from radiographs using AI-driven knowledge distillation. Heliyon 2024; 10:e26801. [PMID: 38444490 PMCID: PMC10912466 DOI: 10.1016/j.heliyon.2024.e26801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Revised: 01/30/2024] [Accepted: 02/20/2024] [Indexed: 03/07/2024] Open
Abstract
Chest radiography is an essential diagnostic tool for respiratory diseases such as COVID-19, pneumonia, and tuberculosis because it accurately depicts the structures of the chest. However, accurate detection of these diseases from radiographs is a complex task that requires the availability of medical imaging equipment and trained personnel. Conventional deep learning models offer a viable automated solution for this task. However, the high complexity of these models often poses a significant obstacle to their practical deployment within automated medical applications, including mobile apps, web apps, and cloud-based platforms. This study addresses and resolves this dilemma by reducing the complexity of neural networks using knowledge distillation techniques (KDT). The proposed technique trains a neural network on an extensive collection of chest X-ray images and propagates the knowledge to a smaller network capable of real-time detection. To create a comprehensive dataset, we have integrated three popular chest radiograph datasets with chest radiographs for COVID-19, pneumonia, and tuberculosis. Our experiments show that this knowledge distillation approach outperforms conventional deep learning methods in terms of computational complexity and performance for real-time respiratory disease detection. Specifically, our system achieves an impressive average accuracy of 0.97, precision of 0.94, and recall of 0.97.
Collapse
Affiliation(s)
- Md Mohsin Kabir
- Department of Computer Science & Engineering, Bangladesh University of Business & Technology, Dhaka-1216, Bangladesh
| | - M.F. Mridha
- Department of Computer Science, American International University-Bangladesh, Dhaka-1229, Bangladesh
| | - Ashifur Rahman
- Department of Computer Science & Engineering, Bangladesh University of Business & Technology, Dhaka-1216, Bangladesh
| | - Md. Abdul Hamid
- Department of Information Technology, Faculty of Computing & Information Technology, King Abdulaziz University, Jeddah-21589, Kingdom of Saudi Arabia
| | - Muhammad Mostafa Monowar
- Department of Information Technology, Faculty of Computing & Information Technology, King Abdulaziz University, Jeddah-21589, Kingdom of Saudi Arabia
| |
Collapse
|
27
|
Rainio O, Teuho J, Klén R. Evaluation metrics and statistical tests for machine learning. Sci Rep 2024; 14:6086. [PMID: 38480847 PMCID: PMC10937649 DOI: 10.1038/s41598-024-56706-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Accepted: 03/09/2024] [Indexed: 03/17/2024] Open
Abstract
Research on different machine learning (ML) has become incredibly popular during the past few decades. However, for some researchers not familiar with statistics, it might be difficult to understand how to evaluate the performance of ML models and compare them with each other. Here, we introduce the most common evaluation metrics used for the typical supervised ML tasks including binary, multi-class, and multi-label classification, regression, image segmentation, object detection, and information retrieval. We explain how to choose a suitable statistical test for comparing models, how to obtain enough values of the metric for testing, and how to perform the test and interpret its results. We also present a few practical examples about comparing convolutional neural networks used to classify X-rays with different lung infections and detect cancer tumors in positron emission tomography images.
Collapse
Affiliation(s)
- Oona Rainio
- Turku PET Centre, University of Turku and Turku University Hospital, Turku, Finland.
| | - Jarmo Teuho
- Turku PET Centre, University of Turku and Turku University Hospital, Turku, Finland
| | - Riku Klén
- Turku PET Centre, University of Turku and Turku University Hospital, Turku, Finland
| |
Collapse
|
28
|
Malik H, Anees T. Multi-modal deep learning methods for classification of chest diseases using different medical imaging and cough sounds. PLoS One 2024; 19:e0296352. [PMID: 38470893 DOI: 10.1371/journal.pone.0296352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Accepted: 12/11/2023] [Indexed: 03/14/2024] Open
Abstract
Chest disease refers to a wide range of conditions affecting the lungs, such as COVID-19, lung cancer (LC), consolidation lung (COL), and many more. When diagnosing chest disorders medical professionals may be thrown off by the overlapping symptoms (such as fever, cough, sore throat, etc.). Additionally, researchers and medical professionals make use of chest X-rays (CXR), cough sounds, and computed tomography (CT) scans to diagnose chest disorders. The present study aims to classify the nine different conditions of chest disorders, including COVID-19, LC, COL, atelectasis (ATE), tuberculosis (TB), pneumothorax (PNEUTH), edema (EDE), pneumonia (PNEU). Thus, we suggested four novel convolutional neural network (CNN) models that train distinct image-level representations for nine different chest disease classifications by extracting features from images. Furthermore, the proposed CNN employed several new approaches such as a max-pooling layer, batch normalization layers (BANL), dropout, rank-based average pooling (RBAP), and multiple-way data generation (MWDG). The scalogram method is utilized to transform the sounds of coughing into a visual representation. Before beginning to train the model that has been developed, the SMOTE approach is used to calibrate the CXR and CT scans as well as the cough sound images (CSI) of nine different chest disorders. The CXR, CT scan, and CSI used for training and evaluating the proposed model come from 24 publicly available benchmark chest illness datasets. The classification performance of the proposed model is compared with that of seven baseline models, namely Vgg-19, ResNet-101, ResNet-50, DenseNet-121, EfficientNetB0, DenseNet-201, and Inception-V3, in addition to state-of-the-art (SOTA) classifiers. The effectiveness of the proposed model is further demonstrated by the results of the ablation experiments. The proposed model was successful in achieving an accuracy of 99.01%, making it superior to both the baseline models and the SOTA classifiers. As a result, the proposed approach is capable of offering significant support to radiologists and other medical professionals.
Collapse
Affiliation(s)
- Hassaan Malik
- Department of Computer Science, School of Systems and Technology, University of Management and Technology, Lahore, Pakistan
| | - Tayyaba Anees
- Department of Software Engineering, School of Systems and Technology, University of Management and Technology, Lahore, Pakistan
| |
Collapse
|
29
|
Abad M, Casas-Roma J, Prados F. Generalizable disease detection using model ensemble on chest X-ray images. Sci Rep 2024; 14:5890. [PMID: 38467705 PMCID: PMC10928229 DOI: 10.1038/s41598-024-56171-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 03/03/2024] [Indexed: 03/13/2024] Open
Abstract
In the realm of healthcare, the demand for swift and precise diagnostic tools has been steadily increasing. This study delves into a comprehensive performance analysis of three pre-trained convolutional neural network (CNN) architectures: ResNet50, DenseNet121, and Inception-ResNet-v2. To ensure the broad applicability of our approach, we curated a large-scale dataset comprising a diverse collection of chest X-ray images, that included both positive and negative cases of COVID-19. The models' performance was evaluated using separate datasets for internal validation (from the same source as the training images) and external validation (from different sources). Our examination uncovered a significant drop in network efficacy, registering a 10.66% reduction for ResNet50, a 36.33% decline for DenseNet121, and a 19.55% decrease for Inception-ResNet-v2 in terms of accuracy. Best results were obtained with DenseNet121 achieving the highest accuracy at 96.71% in internal validation and Inception-ResNet-v2 attaining 76.70% accuracy in external validation. Furthermore, we introduced a model ensemble approach aimed at improving network performance when making inferences on images from diverse sources beyond their training data. The proposed method uses uncertainty-based weighting by calculating the entropy in order to assign appropriate weights to the outputs of each network. Our results showcase the effectiveness of the ensemble method in enhancing accuracy up to 97.38% for internal validation and 81.18% for external validation, while maintaining a balanced ability to detect both positive and negative cases.
Collapse
Affiliation(s)
- Maider Abad
- Universitat Oberta de Catalunya, e-Health Center, Barcelona, Spain.
| | - Jordi Casas-Roma
- Universitat Oberta de Catalunya, e-Health Center, Barcelona, Spain
- Department of Computer Science, Universitat Autònoma de Barcelona (UAB), Bellaterra, Spain
- Computer Vision Center (CVC), Universitat Autònoma de Barcelona (UAB), Bellaterra, Spain
| | - Ferran Prados
- Universitat Oberta de Catalunya, e-Health Center, Barcelona, Spain
- Queen Square MS Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, Faculty of Brain Science, University College of London, London, WC1N 3BG, UK
- Centre for Medical Image Computing (CMIC), Department of Medical Physics and Bioengineering, University College London, London, WC1V 6LJ, UK
| |
Collapse
|
30
|
Ju H, Cui Y, Su Q, Juan L, Manavalan B. CODENET: A deep learning model for COVID-19 detection. Comput Biol Med 2024; 171:108229. [PMID: 38447500 DOI: 10.1016/j.compbiomed.2024.108229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 02/20/2024] [Accepted: 02/25/2024] [Indexed: 03/08/2024]
Abstract
Conventional COVID-19 testing methods have some flaws: they are expensive and time-consuming. Chest X-ray (CXR) diagnostic approaches can alleviate these flaws to some extent. However, there is no accurate and practical automatic diagnostic framework with good interpretability. The application of artificial intelligence (AI) technology to medical radiography can help to accurately detect the disease, reduce the burden on healthcare organizations, and provide good interpretability. Therefore, this study proposes a new deep neural network (CNN) based on CXR for COVID-19 diagnosis - CodeNet. This method uses contrastive learning to make full use of latent image data to enhance the model's ability to extract features and generalize across different data domains. On the evaluation dataset, the proposed method achieves an accuracy as high as 94.20%, outperforming several other existing methods used for comparison. Ablation studies validate the efficacy of the proposed method, while interpretability analysis shows that the method can effectively guide clinical professionals. This work demonstrates the superior detection performance of a CNN using contrastive learning techniques on CXR images, paving the way for computer vision and artificial intelligence technologies to leverage massive medical data for disease diagnosis.
Collapse
Affiliation(s)
- Hong Ju
- Heilongjiang Agricultural Engineering Vocational College, China
| | - Yanyan Cui
- Beidahuang Industry Group General Hospital, Harbin, China
| | - Qiaosen Su
- Department of Integrative Biotechnology, College of Biotechnology and Bioengineering, Sungkyunkwan University, Suwon, 16419, Gyeonggi-do, Republic of Korea
| | - Liran Juan
- School of Life Science and Technology, Harbin Institute of Technology, Harbin, 150001, China.
| | - Balachandran Manavalan
- Department of Integrative Biotechnology, College of Biotechnology and Bioengineering, Sungkyunkwan University, Suwon, 16419, Gyeonggi-do, Republic of Korea.
| |
Collapse
|
31
|
Ali MU, Zafar A, Tanveer J, Khan MA, Kim SH, Alsulami MM, Lee SW. Deep learning network selection and optimized information fusion for enhanced COVID‐19 detection. INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOLOGY 2024; 34. [DOI: 10.1002/ima.23001] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/09/2023] [Accepted: 11/12/2023] [Indexed: 08/25/2024]
Abstract
AbstractThis study proposes a wrapper‐based technique to improve the classification performance of chest infection (including COVID‐19) detection using X‐rays. Deep features were extracted using pretrained deep learning models. Ten optimization techniques, including poor and rich optimization, path finder algorithm, Henry gas solubility optimization, Harris hawks optimization, atom search optimization, manta‐ray foraging optimization, equilibrium optimizer, slime mold algorithm, generalized normal distribution optimization, and marine predator algorithm, were used to determine the optimal features using a support vector machine. Moreover, a network selection technique was used to select the deep learning models. An online chest infection detection X‐ray scan dataset was used to validate the proposed approach. The results suggest that the proposed wrapper‐based automatic deep learning network selection and feature optimization framework has a high classification rate of 97.7%. The comparative analysis further validates the credibility of the framework in COVID‐19 and other chest infection classifications, suggesting that the proposed approach can help doctors in clinical practice.
Collapse
Affiliation(s)
- Muhammad Umair Ali
- Department of Intelligent Mechatronics Engineering Sejong University Seoul Republic of Korea
| | - Amad Zafar
- Department of Intelligent Mechatronics Engineering Sejong University Seoul Republic of Korea
| | - Jawad Tanveer
- Department of Computer Science and Engineering Sejong University Seoul Republic of Korea
| | | | - Seong Han Kim
- Department of Intelligent Mechatronics Engineering Sejong University Seoul Republic of Korea
| | - Mashael M. Alsulami
- Department of Information Technology, College of Computers and Information Technology Taif University Taif Saudi Arabia
| | - Seung Won Lee
- Department of Precision Medicine Sungkyunkwan University School of Medicine Suwon Republic of Korea
| |
Collapse
|
32
|
Liao H, Li Y. LFC-UNet: learned lossless medical image fast compression with U-Net. PeerJ Comput Sci 2024; 10:e1924. [PMID: 38435602 PMCID: PMC10909184 DOI: 10.7717/peerj-cs.1924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 02/13/2024] [Indexed: 03/05/2024]
Abstract
In the field of medicine, the rapid advancement of medical technology has significantly increased the speed of medical image generation, compelling us to seek efficient methods for image compression. Neural networks, owing to their outstanding image estimation capabilities, have provided new avenues for lossless compression. In recent years, learning-based lossless image compression methods, combining neural network predictions with residuals, have achieved performance comparable to traditional non-learning algorithms. However, existing methods have not taken into account that residuals often concentrate excessively, hindering the neural network's ability to learn accurate residual probability estimation. To address this issue, this study employs a weighted cross-entropy method to handle the imbalance in residual categories. In terms of network architecture, we introduce skip connections from U-Net to better capture image features, thereby obtaining accurate probability estimates. Furthermore, our framework boasts excellent encoding speed, as the model is able to acquire all residuals and residual probabilities in a single inference pass. The experimental results demonstrate that the proposed method achieves state-of-the-art performance on medical datasets while also offering the fastest processing speed. As illustrated by an instance using head CT data, our approach achieves a compression efficiency of 2.30 bits per pixel, with a processing time of only 0.320 seconds per image.
Collapse
Affiliation(s)
- Hengrui Liao
- School of Computer, University of South China, Hengyang, Hunan, China
| | - Yue Li
- School of Computer, University of South China, Hengyang, Hunan, China
| |
Collapse
|
33
|
Kumar S, Kumar H, Kumar G, Singh SP, Bijalwan A, Diwakar M. A methodical exploration of imaging modalities from dataset to detection through machine learning paradigms in prominent lung disease diagnosis: a review. BMC Med Imaging 2024; 24:30. [PMID: 38302883 PMCID: PMC10832080 DOI: 10.1186/s12880-024-01192-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Accepted: 01/03/2024] [Indexed: 02/03/2024] Open
Abstract
BACKGROUND Lung diseases, both infectious and non-infectious, are the most prevalent cause of mortality overall in the world. Medical research has identified pneumonia, lung cancer, and Corona Virus Disease 2019 (COVID-19) as prominent lung diseases prioritized over others. Imaging modalities, including X-rays, computer tomography (CT) scans, magnetic resonance imaging (MRIs), positron emission tomography (PET) scans, and others, are primarily employed in medical assessments because they provide computed data that can be utilized as input datasets for computer-assisted diagnostic systems. Imaging datasets are used to develop and evaluate machine learning (ML) methods to analyze and predict prominent lung diseases. OBJECTIVE This review analyzes ML paradigms, imaging modalities' utilization, and recent developments for prominent lung diseases. Furthermore, the research also explores various datasets available publically that are being used for prominent lung diseases. METHODS The well-known databases of academic studies that have been subjected to peer review, namely ScienceDirect, arXiv, IEEE Xplore, MDPI, and many more, were used for the search of relevant articles. Applied keywords and combinations used to search procedures with primary considerations for review, such as pneumonia, lung cancer, COVID-19, various imaging modalities, ML, convolutional neural networks (CNNs), transfer learning, and ensemble learning. RESULTS This research finding indicates that X-ray datasets are preferred for detecting pneumonia, while CT scan datasets are predominantly favored for detecting lung cancer. Furthermore, in COVID-19 detection, X-ray datasets are prioritized over CT scan datasets. The analysis reveals that X-rays and CT scans have surpassed all other imaging techniques. It has been observed that using CNNs yields a high degree of accuracy and practicability in identifying prominent lung diseases. Transfer learning and ensemble learning are complementary techniques to CNNs to facilitate analysis. Furthermore, accuracy is the most favored metric for assessment.
Collapse
Affiliation(s)
- Sunil Kumar
- Department of Computer Engineering, J. C. Bose University of Science and Technology, YMCA, Faridabad, India
- Department of Information Technology, School of Engineering and Technology (UIET), CSJM University, Kanpur, India
| | - Harish Kumar
- Department of Computer Engineering, J. C. Bose University of Science and Technology, YMCA, Faridabad, India
| | - Gyanendra Kumar
- Department of Computer and Communication Engineering, Manipal University Jaipur, Jaipur, India
| | | | - Anchit Bijalwan
- Faculty of Electrical and Computer Engineering, Arba Minch University, Arba Minch, Ethiopia.
| | - Manoj Diwakar
- Department of Computer Science and Engineering, Graphic Era Deemed to Be University, Dehradun, India
| |
Collapse
|
34
|
Chen S, Ren S, Wang G, Huang M, Xue C. Interpretable CNN-Multilevel Attention Transformer for Rapid Recognition of Pneumonia From Chest X-Ray Images. IEEE J Biomed Health Inform 2024; 28:753-764. [PMID: 37027681 DOI: 10.1109/jbhi.2023.3247949] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Abstract
Chest imaging plays an essential role in diagnosing and predicting patients with COVID-19 with evidence of worsening respiratory status. Many deep learning-based approaches for pneumonia recognition have been developed to enable computer-aided diagnosis. However, the long training and inference time makes them inflexible, and the lack of interpretability reduces their credibility in clinical medical practice. This paper aims to develop a pneumonia recognition framework with interpretability, which can understand the complex relationship between lung features and related diseases in chest X-ray (CXR) images to provide high-speed analytics support for medical practice. To reduce the computational complexity to accelerate the recognition process, a novel multi-level self-attention mechanism within Transformer has been proposed to accelerate convergence and emphasize the task-related feature regions. Moreover, a practical CXR image data augmentation has been adopted to address the scarcity of medical image data problems to boost the model's performance. The effectiveness of the proposed method has been demonstrated on the classic COVID-19 recognition task using the widespread pneumonia CXR image dataset. In addition, abundant ablation experiments validate the effectiveness and necessity of all of the components of the proposed method.
Collapse
|
35
|
Prince R, Niu Z, Khan ZY, Emmanuel M, Patrick N. COVID-19 detection from chest X-ray images using CLAHE-YCrCb, LBP, and machine learning algorithms. BMC Bioinformatics 2024; 25:28. [PMID: 38233764 PMCID: PMC10792799 DOI: 10.1186/s12859-023-05427-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Accepted: 07/20/2023] [Indexed: 01/19/2024] Open
Abstract
BACKGROUND COVID-19 is a disease that caused a contagious respiratory ailment that killed and infected hundreds of millions. It is necessary to develop a computer-based tool that is fast, precise, and inexpensive to detect COVID-19 efficiently. Recent studies revealed that machine learning and deep learning models accurately detect COVID-19 using chest X-ray (CXR) images. However, they exhibit notable limitations, such as a large amount of data to train, larger feature vector sizes, enormous trainable parameters, expensive computational resources (GPUs), and longer run-time. RESULTS In this study, we proposed a new approach to address some of the above-mentioned limitations. The proposed model involves the following steps: First, we use contrast limited adaptive histogram equalization (CLAHE) to enhance the contrast of CXR images. The resulting images are converted from CLAHE to YCrCb color space. We estimate reflectance from chrominance using the Illumination-Reflectance model. Finally, we use a normalized local binary patterns histogram generated from reflectance (Cr) and YCb as the classification feature vector. Decision tree, Naive Bayes, support vector machine, K-nearest neighbor, and logistic regression were used as the classification algorithms. The performance evaluation on the test set indicates that the proposed approach is superior, with accuracy rates of 99.01%, 100%, and 98.46% across three different datasets, respectively. Naive Bayes, a probabilistic machine learning algorithm, emerged as the most resilient. CONCLUSION Our proposed method uses fewer handcrafted features, affordable computational resources, and less runtime than existing state-of-the-art approaches. Emerging nations where radiologists are in short supply can adopt this prototype. We made both coding materials and datasets accessible to the general public for further improvement. Check the manuscript's availability of the data and materials under the declaration section for access.
Collapse
Affiliation(s)
- Rukundo Prince
- Department of Computer Science and Technology, Beijing Institute of Technology, Beijing, China
| | - Zhendong Niu
- Department of Computer Science and Technology, Beijing Institute of Technology, Beijing, China.
| | - Zahid Younas Khan
- Computer Science and Information Technology, University of Azad Jammu and Kashmir, Kashmir, Pakistan
| | - Masabo Emmanuel
- Software Engineering, African Center of Excellence in Data Science(ACE-DS), and the African Center of Excellence in Internet of Things(ACEIoT), University of Rwanda, Kigali, Rwanda
| | - Niyishaka Patrick
- Computer and Information Sciences, University of Hyderabad, Hyderabad, India
| |
Collapse
|
36
|
Kaur BP, Singh H, Hans R, Sharma SK, Kaushal C, Hassan MM, Shah MA. An augmentation aided concise CNN based architecture for COVID-19 diagnosis in real time. Sci Rep 2024; 14:1136. [PMID: 38212647 PMCID: PMC10784465 DOI: 10.1038/s41598-024-51317-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 01/03/2024] [Indexed: 01/13/2024] Open
Abstract
Over 6.5 million people around the world have lost their lives due to the highly contagious COVID 19 virus. The virus increases the danger of fatal health effects by damaging the lungs severely. The only method to reduce mortality and contain the spread of this disease is by promptly detecting it. Recently, deep learning has become one of the most prominent approaches to CAD, helping surgeons make more informed decisions. But deep learning models are computation hungry and devices with TPUs and GPUs are needed to run these models. The current focus of machine learning research is on developing models that can be deployed on mobile and edge devices. To this end, this research aims to develop a concise convolutional neural network-based computer-aided diagnostic system for detecting the COVID 19 virus in X-ray images, which may be deployed on devices with limited processing resources, such as mobile phones and tablets. The proposed architecture aspires to use the image enhancement in first phase and data augmentation in the second phase for image pre-processing, additionally hyperparameters are also optimized to obtain the optimal parameter settings in the third phase that provide the best results. The experimental analysis has provided empirical evidence of the impact of image enhancement, data augmentation, and hyperparameter tuning on the proposed convolutional neural network model, which increased accuracy from 94 to 98%. Results from the evaluation show that the suggested method gives an accuracy of 98%, which is better than popular transfer learning models like Xception, Resnet50, and Inception.
Collapse
Affiliation(s)
- Balraj Preet Kaur
- Department of Computer Science and Engineering, DAV University, Jalandhar, India
| | - Harpreet Singh
- Department of Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, India
| | - Rahul Hans
- Department of Computer Science and Engineering, DAV University, Jalandhar, India
| | - Sanjeev Kumar Sharma
- Department of Computer Science and Applications, DAV University, Jalandhar, India
| | - Chetna Kaushal
- Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, 140401, India
| | - Md Mehedi Hassan
- Computer Science and Engineering Discipline, Khulna University, Khulna, 9208, Bangladesh
| | - Mohd Asif Shah
- Department of Economics, Kebri Dehar University, Kebri Dehar, 250, Ethiopia.
- Centre of Research Impact and Outcome, Chitkara University Institute of Engineering and Technology, Chitkara University, Rajpura, 140401, Punjab, India.
- Division of Research and Development, Lovely Professional University, Phagwara, 144001, Punjab, India.
| |
Collapse
|
37
|
Chauhan S, Edla DR, Boddu V, Rao MJ, Cheruku R, Nayak SR, Martha S, Lavanya K, Nigat TD. Detection of COVID-19 using edge devices by a light-weight convolutional neural network from chest X-ray images. BMC Med Imaging 2024; 24:1. [PMID: 38166813 PMCID: PMC10759384 DOI: 10.1186/s12880-023-01155-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 11/14/2023] [Indexed: 01/05/2024] Open
Abstract
Deep learning is a highly significant technology in clinical treatment and diagnostics nowadays. Convolutional Neural Network (CNN) is a new idea in deep learning that is being used in the area of computer vision. The COVID-19 detection is the subject of our medical study. Researchers attempted to increase the detection accuracy but at the cost of high model complexity. In this paper, we desire to achieve better accuracy with little training space and time so that this model easily deployed in edge devices. In this paper, a new CNN design is proposed that has three stages: pre-processing, which removes the black padding on the side initially; convolution, which employs filter banks; and feature extraction, which makes use of deep convolutional layers with skip connections. In order to train the model, chest X-ray images are partitioned into three sets: learning(0.7), validation(0.1), and testing(0.2). The models are then evaluated using the test and training data. The LMNet, CoroNet, CVDNet, and Deep GRU-CNN models are the other four models used in the same experiment. The propose model achieved 99.47% & 98.91% accuracy on training and testing respectively. Additionally, it achieved 97.54%, 98.19%, 99.49%, and 97.86% scores for precision, recall, specificity, and f1-score respectively. The proposed model obtained nearly equivalent accuracy and other similar metrics when compared with other models but greatly reduced the model complexity. Moreover, it is found that proposed model is less prone to over fitting as compared to other models.
Collapse
Affiliation(s)
- Sohamkumar Chauhan
- Department of Computer Science and Engineering, National Institute of Technology Goa, Ponda, 403401, Goa, India
| | - Damoder Reddy Edla
- Department of Computer Science and Engineering, National Institute of Technology Goa, Ponda, 403401, Goa, India
| | - Vijayasree Boddu
- Department of Electronics and Communication Engineering, National Institute of Technology Warangal, Hanamkonda, 506004, Telangana, India
| | - M Jayanthi Rao
- Department of CSE, Aditya Institute of Technology and Management, Kotturu, Tekkali, Andhra Pradesh, India
| | - Ramalingaswamy Cheruku
- Department of Computer Science and Engineering, National Institute of Technology Warangal, Hanamkonda, 506004, Telangana, India
| | - Soumya Ranjan Nayak
- School of Computer Engineering, KIIT Deemed to be University, Bhubaneswar, 751024, Odisha, India
| | - Sheshikala Martha
- School of Computer Science and Artificial Intelligence, SR University, Warangal, 506004, Telangana, India
| | - Kamppa Lavanya
- University College of Sciences, Acharya Nagarjuna Univesity, Guntur, Andhra Pradesh, India
| | - Tsedenya Debebe Nigat
- Faculty of Computing and Informatics, Jimma Institute of Technology, Jimma, Oromia, Ethiopia.
| |
Collapse
|
38
|
Ouis MY, A Akhloufi M. Deep learning for report generation on chest X-ray images. Comput Med Imaging Graph 2024; 111:102320. [PMID: 38134726 DOI: 10.1016/j.compmedimag.2023.102320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 11/13/2023] [Accepted: 11/29/2023] [Indexed: 12/24/2023]
Abstract
Medical imaging, specifically chest X-ray image analysis, is a crucial component of early disease detection and screening in healthcare. Deep learning techniques, such as convolutional neural networks (CNNs), have emerged as powerful tools for computer-aided diagnosis (CAD) in chest X-ray image analysis. These techniques have shown promising results in automating tasks such as classification, detection, and segmentation of abnormalities in chest X-ray images, with the potential to surpass human radiologists. In this review, we provide an overview of the importance of chest X-ray image analysis, historical developments, impact of deep learning techniques, and availability of labeled databases. We specifically focus on advancements and challenges in radiology report generation using deep learning, highlighting potential future advancements in this area. The use of deep learning for report generation has the potential to reduce the burden on radiologists, improve patient care, and enhance the accuracy and efficiency of chest X-ray image analysis in medical imaging.
Collapse
Affiliation(s)
- Mohammed Yasser Ouis
- Perception, Robotics and Intelligent Machines Lab(PRIME), Department of Computer Science, Université de Moncton, Moncton, NB E1C 3E9, Canada.
| | - Moulay A Akhloufi
- Perception, Robotics and Intelligent Machines Lab(PRIME), Department of Computer Science, Université de Moncton, Moncton, NB E1C 3E9, Canada.
| |
Collapse
|
39
|
Azad R, Kazerouni A, Heidari M, Aghdam EK, Molaei A, Jia Y, Jose A, Roy R, Merhof D. Advances in medical image analysis with vision Transformers: A comprehensive review. Med Image Anal 2024; 91:103000. [PMID: 37883822 DOI: 10.1016/j.media.2023.103000] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 09/30/2023] [Accepted: 10/11/2023] [Indexed: 10/28/2023]
Abstract
The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.
Collapse
Affiliation(s)
- Reza Azad
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Amirhossein Kazerouni
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Moein Heidari
- School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran
| | | | - Amirali Molaei
- School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran
| | - Yiwei Jia
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Abin Jose
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Rijo Roy
- Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen, Germany
| | - Dorit Merhof
- Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany; Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.
| |
Collapse
|
40
|
Talukder MA, Layek MA, Kazi M, Uddin MA, Aryal S. Empowering COVID-19 detection: Optimizing performance through fine-tuned EfficientNet deep learning architecture. Comput Biol Med 2024; 168:107789. [PMID: 38042105 DOI: 10.1016/j.compbiomed.2023.107789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Revised: 11/21/2023] [Accepted: 11/28/2023] [Indexed: 12/04/2023]
Abstract
The worldwide COVID-19 pandemic has profoundly influenced the health and everyday experiences of individuals across the planet. It is a highly contagious respiratory disease requiring early and accurate detection to curb its rapid transmission. Initial testing methods primarily revolved around identifying the genetic composition of the coronavirus, exhibiting a relatively low detection rate and requiring a time-intensive procedure. To address this challenge, experts have suggested using radiological imagery, particularly chest X-rays, as a valuable approach within the diagnostic protocol. This study investigates the potential of leveraging radiographic imaging (X-rays) with deep learning algorithms to swiftly and precisely identify COVID-19 patients. The proposed approach elevates the detection accuracy by fine-tuning with appropriate layers on various established transfer learning models. The experimentation was conducted on a COVID-19 X-ray dataset containing 2000 images. The accuracy rates achieved were impressive of 99.55%, 97.32%, 99.11%, 99.55%, 99.11% and 100% for Xception, InceptionResNetV2, ResNet50 , ResNet50V2, EfficientNetB0 and EfficientNetB4 respectively. The fine-tuned EfficientNetB4 achieved an excellent accuracy score, showcasing its potential as a robust COVID-19 detection model. Furthermore, EfficientNetB4 excelled in identifying Lung disease using Chest X-ray dataset containing 4,350 Images, achieving remarkable performance with an accuracy of 99.17%, precision of 99.13%, recall of 99.16%, and f1-score of 99.14%. These results highlight the promise of fine-tuned transfer learning for efficient lung detection through medical imaging, especially with X-ray images. This research offers radiologists an effective means of aiding rapid and precise COVID-19 diagnosis and contributes valuable assistance for healthcare professionals in accurately identifying affected patients.
Collapse
Affiliation(s)
- Md Alamin Talukder
- Department of Computer Science and Engineering, Jagannath University, Dhaka, Bangladesh.
| | - Md Abu Layek
- Department of Computer Science and Engineering, Jagannath University, Dhaka, Bangladesh.
| | - Mohsin Kazi
- Department of Pharmaceutics, College of Pharmacy, King Saud University, P.O. Box-2457, Riyadh 11451, Saudi Arabia.
| | - Md Ashraf Uddin
- School of Information Technology, Deakin University, Waurn Ponds Campus, Geelong, Australia.
| | - Sunil Aryal
- School of Information Technology, Deakin University, Waurn Ponds Campus, Geelong, Australia.
| |
Collapse
|
41
|
Gopatoti A, Jayakumar R, Billa P, Patteeswaran V. DDA-SSNets: Dual decoder attention-based semantic segmentation networks for COVID-19 infection segmentation and classification using chest X-Ray images. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024; 32:623-649. [PMID: 38607728 DOI: 10.3233/xst-230421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/14/2024]
Abstract
BACKGROUND COVID-19 needs to be diagnosed and staged to be treated accurately. However, prior studies' diagnostic and staging abilities for COVID-19 infection needed to be improved. Therefore, new deep learning-based approaches are required to aid radiologists in detecting and quantifying COVID-19-related lung infections. OBJECTIVE To develop deep learning-based models to classify and quantify COVID-19-related lung infections. METHODS Initially, Dual Decoder Attention-based Semantic Segmentation Networks (DDA-SSNets) such as Dual Decoder Attention-UNet (DDA-UNet) and Dual Decoder Attention-SegNet (DDA-SegNet) are proposed to facilitate the dual segmentation tasks such as lung lobes and infection segmentation in chest X-ray (CXR) images. The lung lobe and infection segmentations are mapped to grade the severity of COVID-19 infection in both the lungs of CXRs. Later, a Genetic algorithm-based Deep Convolutional Neural Network classifier with the optimum number of layers, namely GADCNet, is proposed to classify the extracted regions of interest (ROI) from the CXR lung lobes into COVID-19 and non-COVID-19. RESULTS The DDA-SegNet shows better segmentation with an average BCSSDC of 99.53% and 99.97% for lung lobes and infection segmentations, respectively, compared with DDA-UNet with an average BCSSDC of 99.14% and 99.92%. The proposed DDA-SegNet with GADCNet classifier offered excellent classification results with an average BCCAC of 99.98%, followed by the GADCNet with DDA-UNet with an average BCCAC of 99.92% after extensive testing and analysis. CONCLUSIONS The results show that the proposed DDA-SegNet has superior performance in the segmentation of lung lobes and COVID-19-infected regions in CXRs, along with improved severity grading compared to the DDA-UNet and improved accuracy of the GADCNet classifier in classifying the CXRs into COVID-19, and non-COVID-19.
Collapse
Affiliation(s)
- Anandbabu Gopatoti
- Department of Electronics and Communication Engineering, Hindusthan College of Engineering and Technology, Coimbatore, Tamil Nadu, India
| | - Ramya Jayakumar
- Department of Electronics and Communication Engineering, Hindusthan College of Engineering and Technology, Coimbatore, Tamil Nadu, India
| | - Poornaiah Billa
- Department of Electronics and Communication Engineering, Lakireddy Bali Reddy College of Engineering, Mylavaram, Andhra Pradesh, India
| | - Vijayalakshmi Patteeswaran
- Department of Electronics and Communication Engineering, Hindusthan College of Engineering and Technology, Coimbatore, Tamil Nadu, India
| |
Collapse
|
42
|
Islam MT, Zhou Z, Ren H, Khuzani MB, Kapp D, Zou J, Tian L, Liao JC, Xing L. Revealing hidden patterns in deep neural network feature space continuum via manifold learning. Nat Commun 2023; 14:8506. [PMID: 38129376 PMCID: PMC10739971 DOI: 10.1038/s41467-023-43958-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 11/24/2023] [Indexed: 12/23/2023] Open
Abstract
Deep neural networks (DNNs) extract thousands to millions of task-specific features during model training for inference and decision-making. While visualizing these features is critical for comprehending the learning process and improving the performance of the DNNs, existing visualization techniques work only for classification tasks. For regressions, the feature points lie on a high dimensional continuum having an inherently complex shape, making a meaningful visualization of the features intractable. Given that the majority of deep learning applications are regression-oriented, developing a conceptual framework and computational method to reliably visualize the regression features is of great significance. Here, we introduce a manifold discovery and analysis (MDA) method for DNN feature visualization, which involves learning the manifold topology associated with the output and target labels of a DNN. MDA leverages the acquired topological information to preserve the local geometry of the feature space manifold and provides insightful visualizations of the DNN features, highlighting the appropriateness, generalizability, and adversarial robustness of a DNN. The performance and advantages of the MDA approach compared to the existing methods are demonstrated in different deep learning applications.
Collapse
Affiliation(s)
- Md Tauhidul Islam
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA
| | - Zixia Zhou
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA
| | - Hongyi Ren
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA
| | | | - Daniel Kapp
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA
| | - James Zou
- Department of Biomedical Data Science, Stanford University, Stanford, CA, 94305, USA
| | - Lu Tian
- Department of Biomedical Data Science, Stanford University, Stanford, CA, 94305, USA
| | - Joseph C Liao
- Department of Urology, Stanford University, Stanford, CA, 94305, USA.
| | - Lei Xing
- Department of Radiation Oncology, Stanford University, Stanford, CA, 94305, USA.
| |
Collapse
|
43
|
Phumkuea T, Wongsirichot T, Damkliang K, Navasakulpong A, Andritsch J. MSTAC: A Multi-Stage Automated Classification of COVID-19 Chest X-ray Images Using Stacked CNN Models. Tomography 2023; 9:2233-2246. [PMID: 38133077 PMCID: PMC10747997 DOI: 10.3390/tomography9060173] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 12/08/2023] [Accepted: 12/08/2023] [Indexed: 12/23/2023] Open
Abstract
This study introduces a Multi-Stage Automated Classification (MSTAC) system for COVID-19 chest X-ray (CXR) images, utilizing stacked Convolutional Neural Network (CNN) models. Suspected COVID-19 patients often undergo CXR imaging, making it valuable for disease classification. The study collected CXR images from public datasets and aimed to differentiate between COVID-19, non-COVID-19, and healthy cases. MSTAC employs two classification stages: the first distinguishes healthy from unhealthy cases, and the second further classifies COVID-19 and non-COVID-19 cases. Compared to a single CNN-Multiclass model, MSTAC demonstrated superior classification performance, achieving 97.30% accuracy and sensitivity. In contrast, the CNN-Multiclass model showed 94.76% accuracy and sensitivity. MSTAC's effectiveness is highlighted in its promising results over the CNN-Multiclass model, suggesting its potential to assist healthcare professionals in efficiently diagnosing COVID-19 cases. The system outperformed similar techniques, emphasizing its accuracy and efficiency in COVID-19 diagnosis. This research underscores MSTAC as a valuable tool in medical image analysis for enhanced disease classification.
Collapse
Affiliation(s)
- Thanakorn Phumkuea
- College of Digital Science, Prince of Songkla University, Songkhla 90110, Thailand
| | - Thakerng Wongsirichot
- Division of Computational Science, Faculty of Science, Prince of Songkla University, Songkhla 90110, Thailand;
| | - Kasikrit Damkliang
- Division of Computational Science, Faculty of Science, Prince of Songkla University, Songkhla 90110, Thailand;
| | - Asma Navasakulpong
- Division of Respiratory and Respiratory Critical Care Medicine, Prince of Songkla University, Songkhla 90110, Thailand;
| | - Jarutas Andritsch
- Faculty of Business, Law and Digital Technologies, Solent University, Southampton SO14 0YN, UK;
| |
Collapse
|
44
|
Wu JCH, Yu HW, Tsai TH, Lu HHS. Dynamically Synthetic Images for Federated Learning of medical images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107845. [PMID: 37852147 DOI: 10.1016/j.cmpb.2023.107845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 09/28/2023] [Accepted: 10/03/2023] [Indexed: 10/20/2023]
Abstract
BACKGROUND To develop deep learning models for medical diagnosis, it is important to collect more medical data from several medical institutions. Due to the regulations for privacy concerns, it is infeasible to collect data from various medical institutions to one institution for centralized learning. Federated Learning (FL) provides a feasible approach to jointly train the deep learning model with data stored in various medical institutions instead of collected together. However, the resulting FL models could be biased towards institutions with larger training datasets. METHODOLOGY In this study, we propose the applicable method of Dynamically Synthetic Images for Federated Learning (DSIFL) that aims to integrate the information of local institutions with heterogeneous types of data. The main technique of DSIFL is to develop a synthetic method that can dynamically adjust the number of synthetic images similar to local data that are misclassified by the current model. The resulting global model can handle the diversity in heterogeneous types of data collected in local medical institutions by including the training of synthetic images similar to misclassified cases in local collections. RESULTS In model performance evaluation metrics, we focus on the accuracy of each client's dataset. Finally, the accuracy of the model of DSIFL in the experiments can achieve the higher accuracy of the FL approach. CONCLUSION In this study, we propose the framework of DSIFL that achieves improvements over the conventional FL approach. We conduct empirical studies with two kinds of medical images. We compare the performance by variants of FL vs. DSIFL approaches. The performance by individual training is used as the baseline, whereas the performance by centralized learning is used as the target for the comparison studies. The empirical findings suggest that the DSIFL has improved performance over the FL via the technique of dynamically synthetic images in training.
Collapse
Affiliation(s)
- Jacky Chung-Hao Wu
- Institute of Statistics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan, ROC
| | - Hsuan-Wen Yu
- Institute of Statistics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan, ROC
| | - Tsung-Hung Tsai
- Institute of Statistics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan, ROC
| | - Henry Horng-Shing Lu
- Institute of Statistics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan, ROC; Department of Statistics and Data Science, Cornell University, New York, USA.
| |
Collapse
|
45
|
Zhang Z, Wu H, Zhao H, Shi Y, Wang J, Bai H, Sun B. A Novel Deep Learning Model for Medical Image Segmentation with Convolutional Neural Network and Transformer. Interdiscip Sci 2023; 15:663-677. [PMID: 37665496 DOI: 10.1007/s12539-023-00585-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Revised: 07/26/2023] [Accepted: 08/01/2023] [Indexed: 09/05/2023]
Abstract
Accurate segmentation of medical images is essential for clinical decision-making, and deep learning techniques have shown remarkable results in this area. However, existing segmentation models that combine transformer and convolutional neural networks often use skip connections in U-shaped networks, which may limit their ability to capture contextual information in medical images. To address this limitation, we propose a coordinated mobile and residual transformer UNet (MRC-TransUNet) that combines the strengths of transformer and UNet architectures. Our approach uses a lightweight MR-ViT to address the semantic gap and a reciprocal attention module to compensate for the potential loss of details. To better explore long-range contextual information, we use skip connections only in the first layer and add MR-ViT and RPA modules in the subsequent downsampling layers. In our study, we evaluated the effectiveness of our proposed method on three different medical image segmentation datasets, namely, breast, brain, and lung. Our proposed method outperformed state-of-the-art methods in terms of various evaluation metrics, including the Dice coefficient and Hausdorff distance. These results demonstrate that our proposed method can significantly improve the accuracy of medical image segmentation and has the potential for clinical applications. Illustration of the proposed MRC-TransUNet. For the input medical images, we first subject them to an intrinsic downsampling operation and then replace the original jump connection structure using MR-ViT. The output feature representations at different scales are fused by the RPA module. Finally, an upsampling operation is performed to fuse the features to restore them to the same resolution as the input image.
Collapse
Affiliation(s)
- Zhuo Zhang
- Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, School of Electronic and Information Engineering, Tiangong University, Tianjin, 300387, China
| | - Hongbing Wu
- School of Computer Science and Technology, Tiangong University, Tianjin, 300387, China
| | - Huan Zhao
- Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, School of Electronic and Information Engineering, Tiangong University, Tianjin, 300387, China
| | - Yicheng Shi
- College of Management and Economics, Tianjin University, Tianjin, 300072, China
| | - Jifang Wang
- Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, School of Electronic and Information Engineering, Tiangong University, Tianjin, 300387, China
| | - Hua Bai
- Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, School of Electronic and Information Engineering, Tiangong University, Tianjin, 300387, China.
| | - Baoshan Sun
- School of Computer Science and Technology, Tiangong University, Tianjin, 300387, China.
| |
Collapse
|
46
|
Fan R, Bowd C, Brye N, Christopher M, Weinreb RN, Kriegman DJ, Zangwill LM. One-Vote Veto: Semi-Supervised Learning for Low-Shot Glaucoma Diagnosis. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3764-3778. [PMID: 37610903 PMCID: PMC11214580 DOI: 10.1109/tmi.2023.3307689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/25/2023]
Abstract
Convolutional neural networks (CNNs) are a promising technique for automated glaucoma diagnosis from images of the fundus, and these images are routinely acquired as part of an ophthalmic exam. Nevertheless, CNNs typically require a large amount of well-labeled data for training, which may not be available in many biomedical image classification applications, especially when diseases are rare and where labeling by experts is costly. This article makes two contributions to address this issue: 1) It extends the conventional Siamese network and introduces a training method for low-shot learning when labeled data are limited and imbalanced, and 2) it introduces a novel semi-supervised learning strategy that uses additional unlabeled training data to achieve greater accuracy. Our proposed multi-task Siamese network (MTSN) can employ any backbone CNN, and we demonstrate with four backbone CNNs that its accuracy with limited training data approaches the accuracy of backbone CNNs trained with a dataset that is 50 times larger. We also introduce One-Vote Veto (OVV) self-training, a semi-supervised learning strategy that is designed specifically for MTSNs. By taking both self-predictions and contrastive predictions of the unlabeled training data into account, OVV self-training provides additional pseudo labels for fine-tuning a pre-trained MTSN. Using a large (imbalanced) dataset with 66,715 fundus photographs acquired over 15 years, extensive experimental results demonstrate the effectiveness of low-shot learning with MTSN and semi-supervised learning with OVV self-training. Three additional, smaller clinical datasets of fundus images acquired under different conditions (cameras, instruments, locations, populations) are used to demonstrate the generalizability of the proposed methods.
Collapse
|
47
|
Yue G, Yang C, Zhao Z, An Z, Yang Y. ERGPNet: lesion segmentation network for COVID-19 chest X-ray images based on embedded residual convolution and global perception. Front Physiol 2023; 14:1296185. [PMID: 38028767 PMCID: PMC10679680 DOI: 10.3389/fphys.2023.1296185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 11/02/2023] [Indexed: 12/01/2023] Open
Abstract
The Segmentation of infected areas from COVID-19 chest X-ray (CXR) images is of great significance for the diagnosis and treatment of patients. However, accurately and effectively segmenting infected areas of CXR images is still challenging due to the inherent ambiguity of CXR images and the cross-scale variations in infected regions. To address these issues, this article proposes a ERGPNet based on embedded residuals and global perception, to segment lesion regions in COVID-19 CXR images. First, aiming at the inherent fuzziness of CXR images, an embedded residual convolution structure is proposed to enhance the ability of internal feature extraction. Second, a global information perception module is constructed to guide the network in generating long-distance information flow, alleviating the interferences of cross-scale variations on the algorithm's discrimination ability. Finally, the network's sensitivity to target regions is improved, and the interference of noise information is suppressed through the utilization of parallel spatial and serial channel attention modules. The interactions between each module fully establish the mapping relationship between feature representation and information decision-making and improve the accuracy of lesion segmentation. Extensive experiments on three datasets of COVID-19 CXR images, and the results demonstrate that the proposed method outperforms other state-of-the-art segmentation methods of CXR images.
Collapse
Affiliation(s)
- Gongtao Yue
- School of Computer Science, Xijing University, Xi’an, China
| | - Chen Yang
- School of Computer Science, Xijing University, Xi’an, China
| | - Zhengyang Zhao
- School of Information and Navigation, Air Force Engineering University, Xi’an, China
| | - Ziheng An
- School of Integrated Circuits, Anhui University, Hefei, China
| | - Yongsheng Yang
- School of Computer Science, Xijing University, Xi’an, China
| |
Collapse
|
48
|
Sun J, Shi W, Giuste FO, Vaghani YS, Tang L, Wang MD. Improving explainable AI with patch perturbation-based evaluation pipeline: a COVID-19 X-ray image analysis case study. Sci Rep 2023; 13:19488. [PMID: 37945586 PMCID: PMC10636093 DOI: 10.1038/s41598-023-46493-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 11/01/2023] [Indexed: 11/12/2023] Open
Abstract
Recent advances in artificial intelligence (AI) have sparked interest in developing explainable AI (XAI) methods for clinical decision support systems, especially in translational research. Although using XAI methods may enhance trust in black-box models, evaluating their effectiveness has been challenging, primarily due to the absence of human (expert) intervention, additional annotations, and automated strategies. In order to conduct a thorough assessment, we propose a patch perturbation-based approach to automatically evaluate the quality of explanations in medical imaging analysis. To eliminate the need for human efforts in conventional evaluation methods, our approach executes poisoning attacks during model retraining by generating both static and dynamic triggers. We then propose a comprehensive set of evaluation metrics during the model inference stage to facilitate the evaluation from multiple perspectives, covering a wide range of correctness, completeness, consistency, and complexity. In addition, we include an extensive case study to showcase the proposed evaluation strategy by applying widely-used XAI methods on COVID-19 X-ray imaging classification tasks, as well as a thorough review of existing XAI methods in medical imaging analysis with evaluation availability. The proposed patch perturbation-based workflow offers model developers an automated and generalizable evaluation strategy to identify potential pitfalls and optimize their proposed explainable solutions, while also aiding end-users in comparing and selecting appropriate XAI methods that meet specific clinical needs in real-world clinical research and practice.
Collapse
Affiliation(s)
- Jimin Sun
- School of Computer Science and Engineering, Georgia Institute of Technology, Atlanta, 30322, USA
| | - Wenqi Shi
- School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, 30322, USA
| | - Felipe O Giuste
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, 30322, USA
| | - Yog S Vaghani
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, 30322, USA
| | - Lingzi Tang
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, 30322, USA
| | - May D Wang
- The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, 30322, USA.
| |
Collapse
|
49
|
Celniak W, Wodziński M, Jurgas A, Burti S, Zotti A, Atzori M, Müller H, Banzato T. Improving the classification of veterinary thoracic radiographs through inter-species and inter-pathology self-supervised pre-training of deep learning models. Sci Rep 2023; 13:19518. [PMID: 37945653 PMCID: PMC10636209 DOI: 10.1038/s41598-023-46345-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Accepted: 10/31/2023] [Indexed: 11/12/2023] Open
Abstract
The analysis of veterinary radiographic imaging data is an essential step in the diagnosis of many thoracic lesions. Given the limited time that physicians can devote to a single patient, it would be valuable to implement an automated system to help clinicians make faster but still accurate diagnoses. Currently, most of such systems are based on supervised deep learning approaches. However, the problem with these solutions is that they need a large database of labeled data. Access to such data is often limited, as it requires a great investment of both time and money. Therefore, in this work we present a solution that allows higher classification scores to be obtained using knowledge transfer from inter-species and inter-pathology self-supervised learning methods. Before training the network for classification, pretraining of the model was performed using self-supervised learning approaches on publicly available unlabeled radiographic data of human and dog images, which allowed substantially increasing the number of images for this phase. The self-supervised learning approaches included the Beta Variational Autoencoder, the Soft-Introspective Variational Autoencoder, and a Simple Framework for Contrastive Learning of Visual Representations. After the initial pretraining, fine-tuning was performed for the collected veterinary dataset using 20% of the available data. Next, a latent space exploration was performed for each model after which the encoding part of the model was fine-tuned again, this time in a supervised manner for classification. Simple Framework for Contrastive Learning of Visual Representations proved to be the most beneficial pretraining method. Therefore, it was for this method that experiments with various fine-tuning methods were carried out. We achieved a mean ROC AUC score of 0.77 and 0.66, respectively, for the laterolateral and dorsoventral projection datasets. The results show significant improvement compared to using the model without any pretraining approach.
Collapse
Affiliation(s)
- Weronika Celniak
- University of Applied Sciences Western Switzerland (HES-SO), 3960, Sierre, Switzerland.
- Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Krakow, 30059, Kraków, Poland.
| | - Marek Wodziński
- University of Applied Sciences Western Switzerland (HES-SO), 3960, Sierre, Switzerland
- Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Krakow, 30059, Kraków, Poland
| | - Artur Jurgas
- University of Applied Sciences Western Switzerland (HES-SO), 3960, Sierre, Switzerland
- Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering, AGH University of Krakow, 30059, Kraków, Poland
| | - Silvia Burti
- Department of Animal Medicine, Productions, and Health, Legnaro (PD), University of Padua, 35020, Padua, Italy
| | - Alessandro Zotti
- Department of Animal Medicine, Productions, and Health, Legnaro (PD), University of Padua, 35020, Padua, Italy
| | - Manfredo Atzori
- University of Applied Sciences Western Switzerland (HES-SO), 3960, Sierre, Switzerland
- Department of Neuroscience, University of Padua, 35121, Padua, IT, Italy
- Padova Neuroscience Center, University of Padova, Via Orus 2/B, 35129, Padova, Italy
| | - Henning Müller
- University of Applied Sciences Western Switzerland (HES-SO), 3960, Sierre, Switzerland
- Medical Faculty, University of Geneva, 1206, Geneva, Switzerland
- The Sense Research and Innovation Insitute, 1950, Sion, Switzerland
| | - Tommaso Banzato
- Department of Animal Medicine, Productions, and Health, Legnaro (PD), University of Padua, 35020, Padua, Italy
| |
Collapse
|
50
|
Saju B, Tressa N, Dhanaraj RK, Tharewal S, Mathew JC, Pelusi D. Effective multi-class lungdisease classification using the hybridfeature engineering mechanism. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:20245-20273. [PMID: 38052644 DOI: 10.3934/mbe.2023896] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
The utilization of computational models in the field of medical image classification is an ongoing and unstoppable trend, driven by the pursuit of aiding medical professionals in achieving swift and precise diagnoses. Post COVID-19, many researchers are studying better classification and diagnosis of lung diseases particularly, as it was reported that one of the very few diseases greatly affecting human beings was related to lungs. This research study, as presented in the paper, introduces an advanced computer-assisted model that is specifically tailored for the classification of 13 lung diseases using deep learning techniques, with a focus on analyzing chest radiograph images. The work flows from data collection, image quality enhancement, feature extraction to a comparative classification performance analysis. For data collection, an open-source data set consisting of 112,000 chest X-Ray images was used. Since, the quality of the pictures was significant for the work, enhanced image quality is achieved through preprocessing techniques such as Otsu-based binary conversion, contrast limited adaptive histogram equalization-driven noise reduction, and Canny edge detection. Feature extraction incorporates connected regions, histogram of oriented gradients, gray-level co-occurrence matrix and Haar wavelet transformation, complemented by feature selection via regularized neighbourhood component analysis. The paper proposes an optimized hybrid model, improved Aquila optimization convolutional neural networks (CNN), which is a combination of optimized CNN and DENSENET121 with applied batch equalization, which provides novelty for the model compared with other similar works. The comparative evaluation of classification performance among CNN, DENSENET121 and the proposed hybrid model is also done to find the results. The findings highlight the proposed hybrid model's supremacy, boasting 97.00% accuracy, 94.00% precision, 96.00% sensitivity, 96.00% specificity and 95.00% F1-score. In the future, potential avenues encompass exploring explainable machine learning for discerning model decisions and optimizing performance through strategic model restructuring.
Collapse
Affiliation(s)
- Binju Saju
- Department of Master of Computer Applications, New Horizon College of Engineering, Bengaluru, India
| | - Neethu Tressa
- Department of Master of Computer Applications, New Horizon College of Engineering, Bengaluru, India
| | - Rajesh Kumar Dhanaraj
- Symbiosis Institute of Computer Studies and Research (SICSR), Symbiosis International University, Pune, India
| | - Sumegh Tharewal
- Symbiosis Institute of Computer Studies and Research (SICSR), Symbiosis International University, Pune, India
| | | | - Danilo Pelusi
- Department of Communication Sciences, University of Teramo, Teramo, Italy
| |
Collapse
|