1
|
Abdulsalam Hamwi W, Almustafa MM. Development and integration of VGG and dense transfer-learning systems supported with diverse lung images for discovery of the Coronavirus identity. Inform Med Unlocked 2022; 32:101004. [PMID: 35822170 PMCID: PMC9263684 DOI: 10.1016/j.imu.2022.101004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 06/24/2022] [Accepted: 06/25/2022] [Indexed: 12/18/2022] Open
Abstract
The contagious SARS-CoV-2 has had a tremendous impact on the life and health of many communities. It was first rampant in early 2019 and so far, 539 million cases of COVID-19 have been reported worldwide. This is reminiscent of the 1918 influenza pandemic. However, we can detect the infected cases of COVID-19 by analysing either X-rays or CT, which are presumably considered the least expensive methods. In the existence of state-of-the-art convolutional neural networks (CNNs), which integrate image pre-processing techniques with fully connected layers, we can develop a sophisticated AI system contingent on various pre-trained models. Each pre-trained model we involved in our study assumed its role in extracting some specific features from different chest image datasets in many verified sources, such as (Mendeley, Kaggle, and GitHub). First, for CXR datasets associated with the CNN trained model from the beginning, whereby is comprised of four layers beginning with the Conv2D layer, which comprises 32 filters, followed by the MaxPooling and afterwards, we reiterated similarly. We used two techniques to avoid overgeneralization, the early stopping and the Dropout techniques. After all, the output was one neuron to classify both cases of 0 or 1, followed by a sigmoid function; in addition, we used the Adam optimizer owing to the more improved outcomes than what other optimizers conducted; ultimately, we referred to our findings by using a confusion matrix, classification report (Recall & Precision), sensitivity and specificity; in this approach, we achieved a classification accuracy of 96%. Our three integrated pre-trained models (VGG16, DenseNet201, and DenseNet121) yielded a remarkable test accuracy of 98.81%. Besides, our merged models (VGG16, DenseNet201) trained on CT images with the utmost effort; this model held an accurate test of 99.73% for binary classification with the (Normal/Covid-19) scenario. Comparing our results with related studies shows that our proposed models were superior to the previous CNN machine learning models in terms of various performance metrics. Our pre-trained model associated with the CT dataset achieved 100% of the F1score and the loss value was approximately 0.00268.
Collapse
Key Words
- AI, Artificial Intelligence
- ANNs, Artificilal Neural Networks
- Artificial intelligence
- CNNs, Convolutional Neural Networks
- CT, Computed Tomography
- CXR&CT chest COVID-19 images integration of three pre-trained CNN models Fine-tuning
- Conv2D, 2D Convolutional Layer
- Covid-19, Coronavirus disease of 2019
- DL, Deep Learning
- Image processing
- ML, Machine Learning
- Performance evaluation
- RT-PCR, Reverse Transcription Polymerase Chain Reaction
- ReLU, Rectified Linear Unit
- SARS_COV_2, Severe acute respiratory syndrome coronavirus
- X-ray,CXR, energic high frequency electromagnetic radiation
Collapse
|
2
|
Ayoobi N, Sharifrazi D, Alizadehsani R, Shoeibi A, Gorriz JM, Moosaei H, Khosravi A, Nahavandi S, Gholamzadeh Chofreh A, Goni FA, Klemeš JJ, Mosavi A. Time series forecasting of new cases and new deaths rate for COVID-19 using deep learning methods. Results Phys 2021; 27:104495. [PMID: 34221854 PMCID: PMC8233414 DOI: 10.1016/j.rinp.2021.104495] [Citation(s) in RCA: 45] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 06/19/2021] [Accepted: 06/22/2021] [Indexed: 05/17/2023]
Abstract
The first known case of Coronavirus disease 2019 (COVID-19) was identified in December 2019. It has spread worldwide, leading to an ongoing pandemic, imposed restrictions and costs to many countries. Predicting the number of new cases and deaths during this period can be a useful step in predicting the costs and facilities required in the future. The purpose of this study is to predict new cases and deaths rate one, three and seven-day ahead during the next 100 days. The motivation for predicting every n days (instead of just every day) is the investigation of the possibility of computational cost reduction and still achieving reasonable performance. Such a scenario may be encountered in real-time forecasting of time series. Six different deep learning methods are examined on the data adopted from the WHO website. Three methods are LSTM, Convolutional LSTM, and GRU. The bidirectional extension is then considered for each method to forecast the rate of new cases and new deaths in Australia and Iran countries. This study is novel as it carries out a comprehensive evaluation of the aforementioned three deep learning methods and their bidirectional extensions to perform prediction on COVID-19 new cases and new death rate time series. To the best of our knowledge, this is the first time that Bi-GRU and Bi-Conv-LSTM models are used for prediction on COVID-19 new cases and new deaths time series. The evaluation of the methods is presented in the form of graphs and Friedman statistical test. The results show that the bidirectional models have lower errors than other models. A several error evaluation metrics are presented to compare all models, and finally, the superiority of bidirectional methods is determined. This research could be useful for organisations working against COVID-19 and determining their long-term plans.
Collapse
Key Words
- ANFIS, Adaptive Network-based Fuzzy Inference System
- ANN, Artificial Neural Network
- AU, Australia
- Bi-Conv-LSTM, Bidirectional Convolutional Long Short Term Memory
- Bi-GRU, Bidirectional Gated Recurrent Unit
- Bi-LSTM, Bidirectional Long Short-Term Memory
- Bidirectional
- COVID-19 Prediction
- COVID-19, Coronavirus Disease 2019
- Conv-LSTM, Convolutional Long Short Term Memory
- Convolutional Long Short Term Memory (Conv-LSTM)
- DL, Deep Learning
- DLSTM, Delayed Long Short-Term Memory
- Deep learning
- EMRO, Eastern Mediterranean Regional Office
- ES, Exponential Smoothing
- EV, Explained Variance
- GRU, Gated Recurrent Unit
- Gated Recurrent Unit (GRU)
- IR, Iran
- LR, Linear Regression
- LSTM, Long Short-Term Memory
- Lasso, Least Absolute Shrinkage and Selection Operator
- Long Short Term Memory (LSTM)
- MAE, Mean Absolute Error
- MAPE, Mean Absolute Percentage Error
- MERS, Middle East Respiratory Syndrome
- ML, Machine Learning
- MLP-ICA, Multi-layered Perceptron-Imperialist Competitive Calculation
- MSE, Mean Square Error
- MSLE, Mean Squared Log Error
- Machine learning
- New Cases of COVID-19
- New Deaths of COVID-19
- PRISMA, Preferred Reporting Items for Precise Surveys and Meta-Analyses
- RMSE, Root Mean Square Error
- RMSLE, Root Mean Squared Log Error
- RNN, Repetitive Neural Network
- ReLU, Rectified Linear Unit
- SARS, Serious Intense Respiratory Disorder
- SARS-COV, SARS coronavirus
- SARS-COV-2, Serious Intense Respiratory Disorder Coronavirus 2
- SVM, Support Vector Machine
- VAE, Variational Auto Encoder
- WHO, World Health Organization
- WPRO, Western Pacific Regional Office
Collapse
Affiliation(s)
- Nooshin Ayoobi
- Department of Mathematics, Savitribai Phule Pune University, Pune 411007, India
| | - Danial Sharifrazi
- Department of Computer Engineering, School of Technical and Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia
| | - Afshin Shoeibi
- Computer Engineering Department, Ferdowsi University of Mashhad, Mashhad, Iran
- Faculty of Electrical and Computer Engineering, Biomedical Data Acquisition Lab, K. N. Toosi University of Technology, Tehran, Iran
| | - Juan M Gorriz
- Department of Signal Theory, Networking and Communications, Universidad de Granada, Spain
| | - Hossein Moosaei
- Department of Mathematics, Faculty of Science, University of Bojnord, Iran
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia
| | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Waurn Ponds, VIC 3217, Australia
| | - Abdoulmohammad Gholamzadeh Chofreh
- Sustainable Process Integration Laboratory - SPIL, NETME Centre, Faculty of Mechanical Engineering, Brno University of Technology - VUT Brno, Technická 2896/2, 616 69 Brno, Czech Republic
| | - Feybi Ariani Goni
- Department of Management, Faculty of Business and Management, Brno University of Technology - VUT Brno, Kolejní 2906/4, 612 00 Brno, Czech Republic
| | - Jiří Jaromír Klemeš
- Sustainable Process Integration Laboratory - SPIL, NETME Centre, Faculty of Mechanical Engineering, Brno University of Technology - VUT Brno, Technická 2896/2, 616 69 Brno, Czech Republic
| | - Amir Mosavi
- John von Neumann Faculty of Informatics, Obuda University, 1034 Budapest, Hungary
- School of Economics and Business, Norwegian University of Life Sciences, 1430 Ås, Norway
| |
Collapse
|
3
|
Montalbo FJP. Diagnosing Covid-19 chest x-rays with a lightweight truncated DenseNet with partial layer freezing and feature fusion. Biomed Signal Process Control 2021; 68:102583. [PMID: 33828610 PMCID: PMC8015405 DOI: 10.1016/j.bspc.2021.102583] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2021] [Revised: 03/23/2021] [Accepted: 03/26/2021] [Indexed: 12/26/2022]
Abstract
Due to the unforeseen turn of events, our world has undergone another global pandemic from a highly contagious novel coronavirus named COVID-19. The novel virus inflames the lungs similarly to Pneumonia, making it challenging to diagnose. Currently, the common standard to diagnose the virus's presence from an individual is using a molecular real-time Reverse-Transcription Polymerase Chain Reaction (rRT-PCR) test from fluids acquired through nasal swabs. Such a test is difficult to acquire in most underdeveloped countries with a few experts that can perform the test. As a substitute, the widely available Chest X-Ray (CXR) became an alternative to rule out the virus. However, such a method does not come easy as the virus still possesses unknown characteristics that even experienced radiologists and other medical experts find difficult to diagnose through CXRs. Several studies have recently used computer-aided methods to automate and improve such diagnosis of CXRs through Artificial Intelligence (AI) based on computer vision and Deep Convolutional Neural Networks (DCNN), which some require heavy processing costs and other tedious methods to produce. Therefore, this work proposed the Fused-DenseNet-Tiny, a lightweight DCNN model based on a densely connected neural network (DenseNet) truncated and concatenated. The model trained to learn CXR features based on transfer learning, partial layer freezing, and feature fusion. Upon evaluation, the proposed model achieved a remarkable 97.99 % accuracy, with only 1.2 million parameters and a shorter end-to-end structure. It has also shown better performance than some existing studies and other massive state-of-the-art models that diagnosed COVID-19 from CXRs.
Collapse
Key Words
- AP, Average Pooling
- AUC, Area Under the Curve
- BN, Batch Normalization
- BS, Batch Size
- CAD, Computer-Aided Diagnosis
- CCE, Categorical Cross-Entropy
- CNN, Convolutional Neural Networks
- CT, Computer Tomography
- CV, Computer Vision
- CXR, Chest X-Rays
- Chest x-rays
- Computer-aided diagnosis
- Covid-19
- DCNN, Deep Convolutional Neural Networks
- DL, Deep Learning
- DR, Dropout Rate
- Deep learning
- Densely connected neural networks
- GAP, Global Average Pooling
- GRAD-CAM, Gradient-Weighted Class Activation Maps
- JPG, Joint Photographic Group
- LR, Learning Rate
- MP, Max-Pooling
- P-R, Precision-Recall
- PEPX, Projection-Expansion-Projection-Extension
- ROC, Receiver Operating Characteristic
- ReLU, Rectified Linear Unit
- SGD, Stochastic Gradient Descent
- WHO, World Health Organization
- rRT-PCR, real-time Reverse-Transcription Polymerase Chain Reaction
Collapse
|
4
|
Pomyen Y, Wanichthanarak K, Poungsombat P, Fahrmann J, Grapov D, Khoomrung S. Deep metabolome: Applications of deep learning in metabolomics. Comput Struct Biotechnol J 2020; 18:2818-2825. [PMID: 33133423 PMCID: PMC7575644 DOI: 10.1016/j.csbj.2020.09.033] [Citation(s) in RCA: 62] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2020] [Revised: 09/21/2020] [Accepted: 09/21/2020] [Indexed: 01/11/2023] Open
Abstract
In the past few years, deep learning has been successfully applied to various omics data. However, the applications of deep learning in metabolomics are still relatively low compared to others omics. Currently, data pre-processing using convolutional neural network architecture appears to benefit the most from deep learning. Compound/structure identification and quantification using artificial neural network/deep learning performed relatively better than traditional machine learning techniques, whereas only marginally better results are observed in biological interpretations. Before deep learning can be effectively applied to metabolomics, several challenges should be addressed, including metabolome-specific deep learning architectures, dimensionality problems, and model evaluation regimes.
Collapse
Key Words
- AI, Artificial Intelligence
- ANN, Artificial Neural Network
- AUC, Area Under the receiver-operating characteristic Curve
- Artificial neural network
- CCS value, Collision Cross Section value
- CFM-EI, Competitive Fragmentation Modeling-Electron Ionization
- CNN, Convolutional Neural Network
- DL, Deep Learning
- DNN, Deep Neural Network
- Deep learning
- ECFP, Extended Circular Fingerprint
- ER, Estrogen Receptor
- FID, Free Induction Decay
- FP score, Fingerprint correlation score
- FTIR, Fourier Transform Infrared
- GC–MS, Gas Chromatography-Mass Spectrometry
- HDLSS data, High Dimensional Low Sample Size data
- IST, Iterative Soft Thresholding
- LC-MS, Liquid Chromatography-Mass Spectrometry
- LSTM, Long Short-Term Memory
- ML, Machine Learning
- MLP, Multi-layered Perceptron
- MS, Mass Spectrometry
- Mass spectrometry
- Metabolomics
- NEIMS, Neural Electron-Ionization Mass Spectrometry
- NMR
- NMR, Nuclear Magnetic Resonance
- NUS, Non-Uniformly Sampling
- PARAFAC2, Parallel Factor Analysis 2
- RF, Random Forest
- RNN, Recurrent Neural Network
- ReLU, Rectified Linear Unit
- SMARTS, SMILES arbitrary target specification
- SMILE, Sparse Multidimensional Iterative Lineshape-enhanced
- SMILES, Simplified Molecular-Input Line-Entry System
- SRA, Sequence Read Archive
- VAE, Variational Autoencoder
- istHMS, Implementation of IST at Harvard Medical School
- m/z, mass/charge ratio
Collapse
Affiliation(s)
- Yotsawat Pomyen
- Translational Research Unit, Chulabhorn Research Institute, Bangkok, Thailand
| | - Kwanjeera Wanichthanarak
- Metabolomics and Systems Biology, Department of Biochemistry, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
- Siriraj Metabolomics and Phenomics Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
| | - Patcha Poungsombat
- Metabolomics and Systems Biology, Department of Biochemistry, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
- Siriraj Metabolomics and Phenomics Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
- Center for Innovation in Chemistry (PERCH-CIC), Faculty of Science, Mahidol University, Rama 6 Road, Bangkok 10400, Thailand
| | - Johannes Fahrmann
- Department of Clinical Cancer Prevention, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, TX 77030, USA
| | - Dmitry Grapov
- CDS- Creative Data Solutions LLC, https://creative-data.solutions, USA
| | - Sakda Khoomrung
- Metabolomics and Systems Biology, Department of Biochemistry, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
- Siriraj Metabolomics and Phenomics Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
- Center for Innovation in Chemistry (PERCH-CIC), Faculty of Science, Mahidol University, Rama 6 Road, Bangkok 10400, Thailand
| |
Collapse
|