1
|
Lella KK, Jagadeesh MS, Alphonse PJA. Artificial intelligence-based framework to identify the abnormalities in the COVID-19 disease and other common respiratory diseases from digital stethoscope data using deep CNN. Health Inf Sci Syst 2024; 12:22. [PMID: 38469455 PMCID: PMC10924857 DOI: 10.1007/s13755-024-00283-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 02/21/2024] [Indexed: 03/13/2024] Open
Abstract
The utilization of lung sounds to diagnose lung diseases using respiratory sound features has significantly increased in the past few years. The Digital Stethoscope data has been examined extensively by medical researchers and technical scientists to diagnose the symptoms of respiratory diseases. Artificial intelligence-based approaches are applied in the real universe to distinguish respiratory disease signs from human pulmonary auscultation sounds. The Deep CNN model is implemented with combined multi-feature channels (Modified MFCC, Log Mel, and Soft Mel) to obtain the sound parameters from lung-based Digital Stethoscope data. The model analysis is observed with max-pooling and without max-pool operations using multi-feature channels on respiratory digital stethoscope data. In addition, COVID-19 sound data and enriched data, which are recently acquired data to enhance model performance using a combination of L2 regularization to overcome the risk of overfitting because of less respiratory sound data, are included in the work. The suggested DCNN with Max-Pooling on the improved dataset demonstrates cutting-edge performance employing a multi-feature channels spectrogram. The model has been developed with different convolutional filter sizes (1 × 12 , 1 × 24 , 1 × 36 , 1 × 48 , and 1 × 60 ) that helped to test the proposed neural network. According to the experimental findings, the suggested DCNN architecture with a max-pooling function performs better to identify respiratory disease symptoms than DCNN without max-pooling. In order to demonstrate the model's effectiveness in categorization, it is trained and tested with the DCNN model that extract several modalities of respiratory sound data.
Collapse
Affiliation(s)
- Kranthi Kumar Lella
- School of Computer Science and Engineering, VIT-AP University, Vijayawada, Guntur, Andhra Pradesh 522237 India
| | - M. S. Jagadeesh
- School of Computer Science and Engineering, VIT-AP University, Vijayawada, Guntur, Andhra Pradesh 522237 India
| | - P. J. A. Alphonse
- Department of Computer Applications, NIT Tiruchirappalli, Tiruchirappalli, Guntur, Tamil Nadu 620015 India
| |
Collapse
|
2
|
Mousavi M, Hosseini S. A deep convolutional neural network approach using medical image classification. BMC Med Inform Decis Mak 2024; 24:239. [PMID: 39210320 PMCID: PMC11360845 DOI: 10.1186/s12911-024-02646-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 08/22/2024] [Indexed: 09/04/2024] Open
Abstract
The epidemic diseases such as COVID-19 are rapidly spreading all around the world. The diagnosis of epidemic at initial stage is of high importance to provide medical care to and recovery of infected people as well as protecting the uninfected population. In this paper, an automatic COVID-19 detection model using respiratory sound and medical image based on internet of health things (IoHT) is proposed. In this model, primarily to screen those people having suspected Coronavirus disease, the sound of coughing used to detect healthy people and those suffering from COVID-19, which finally obtained an accuracy of 94.999%. This approach not only expedites diagnosis and enhances accuracy but also facilitates swift screening in public places using simple equipment. Then, in the second step, in order to help radiologists to interpret medical images as best as possible, we use three pre-trained convolutional neural network models InceptionResNetV2, InceptionV3 and EfficientNetB4 and two data sets of chest radiology medical images, and CT Scan in a three-class classification. Utilizing transfer learning and pre-existing knowledge in these models leads to notable improvements in disease diagnosis and identification compared to traditional techniques. Finally, the best result obtained for CT-Scan images belonging to InceptionResNetV2 architecture with 99.414% accuracy and for radiology images related to InceptionV3 and EfficientNetB4 architectures with the accuracy is 96.943%. Therefore, the proposed model can help radiology specialists to confirm the initial assessments of the COVID-19 disease.
Collapse
Affiliation(s)
- Mohammad Mousavi
- Department of Computer Science, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman, Kerman, Iran
| | - Soodeh Hosseini
- Department of Computer Science, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman, Kerman, Iran.
| |
Collapse
|
3
|
Budd J, Baker K, Karoune E, Coppock H, Patel S, Payne R, Tendero Cañadas A, Titcomb A, Hurley D, Egglestone S, Butler L, Mellor J, Nicholson G, Kiskin I, Koutra V, Jersakova R, McKendry RA, Diggle P, Richardson S, Schuller BW, Gilmour S, Pigoli D, Roberts S, Packham J, Thornley T, Holmes C. A large-scale and PCR-referenced vocal audio dataset for COVID-19. Sci Data 2024; 11:700. [PMID: 38937483 PMCID: PMC11211414 DOI: 10.1038/s41597-024-03492-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Accepted: 06/10/2024] [Indexed: 06/29/2024] Open
Abstract
The UK COVID-19 Vocal Audio Dataset is designed for the training and evaluation of machine learning models that classify SARS-CoV-2 infection status or associated respiratory symptoms using vocal audio. The UK Health Security Agency recruited voluntary participants through the national Test and Trace programme and the REACT-1 survey in England from March 2021 to March 2022, during dominant transmission of the Alpha and Delta SARS-CoV-2 variants and some Omicron variant sublineages. Audio recordings of volitional coughs, exhalations, and speech were collected in the 'Speak up and help beat coronavirus' digital survey alongside demographic, symptom and self-reported respiratory condition data. Digital survey submissions were linked to SARS-CoV-2 test results. The UK COVID-19 Vocal Audio Dataset represents the largest collection of SARS-CoV-2 PCR-referenced audio recordings to date. PCR results were linked to 70,565 of 72,999 participants and 24,105 of 25,706 positive cases. Respiratory symptoms were reported by 45.6% of participants. This dataset has additional potential uses for bioacoustics research, with 11.3% participants self-reporting asthma, and 27.2% with linked influenza PCR test results.
Collapse
Affiliation(s)
- Jobie Budd
- London Centre for Nanotechnology, University College London, London, UK
- Division of Medicine, University College London, London, UK
| | - Kieran Baker
- King's College London, London, UK
- The Alan Turing Institute, London, UK
| | | | - Harry Coppock
- The Alan Turing Institute, London, UK
- Imperial College London, London, UK
| | - Selina Patel
- UK Health Security Agency, London, UK
- Institute of Health Informatics, University College London, London, UK
| | | | - Ana Tendero Cañadas
- UK Health Security Agency, London, UK
- Centre for Stress and Age-Related Disease, School of Applied Sciences, University of Brighton, Brighton, UK
| | | | | | | | | | | | - George Nicholson
- The Alan Turing Institute, London, UK
- University of Oxford, Oxford, UK
| | - Ivan Kiskin
- University of Surrey, Guildford, UK
- The Surrey Institute for People-Centred AI, Centre for Vision, Speech and Signal Processing, Guildford, UK
| | - Vasiliki Koutra
- King's College London, London, UK
- The Alan Turing Institute, London, UK
| | | | - Rachel A McKendry
- London Centre for Nanotechnology, University College London, London, UK
- Division of Medicine, University College London, London, UK
| | | | | | - Björn W Schuller
- The Alan Turing Institute, London, UK
- Imperial College London, London, UK
- CHI, MRI, Technical University of Munich, Munich, Germany
| | - Steven Gilmour
- King's College London, London, UK
- The Alan Turing Institute, London, UK
| | - Davide Pigoli
- King's College London, London, UK
- The Alan Turing Institute, London, UK
| | - Stephen Roberts
- The Alan Turing Institute, London, UK
- University of Oxford, Oxford, UK
| | | | - Tracey Thornley
- UK Health Security Agency, London, UK
- University of Nottingham, Nottingham, UK
| | - Chris Holmes
- The Alan Turing Institute, London, UK
- University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Kapetanidis P, Kalioras F, Tsakonas C, Tzamalis P, Kontogiannis G, Karamanidou T, Stavropoulos TG, Nikoletseas S. Respiratory Diseases Diagnosis Using Audio Analysis and Artificial Intelligence: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2024; 24:1173. [PMID: 38400330 PMCID: PMC10893010 DOI: 10.3390/s24041173] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 02/03/2024] [Accepted: 02/04/2024] [Indexed: 02/25/2024]
Abstract
Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases' symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems.
Collapse
Affiliation(s)
- Panagiotis Kapetanidis
- Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece (C.T.); (G.K.); (S.N.)
| | - Fotios Kalioras
- Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece (C.T.); (G.K.); (S.N.)
| | - Constantinos Tsakonas
- Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece (C.T.); (G.K.); (S.N.)
| | - Pantelis Tzamalis
- Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece (C.T.); (G.K.); (S.N.)
| | - George Kontogiannis
- Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece (C.T.); (G.K.); (S.N.)
| | - Theodora Karamanidou
- Pfizer Center for Digital Innovation, 55535 Thessaloniki, Greece; (T.K.); (T.G.S.)
| | | | - Sotiris Nikoletseas
- Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece (C.T.); (G.K.); (S.N.)
| |
Collapse
|
5
|
Kim J, Choi YS, Lee YJ, Yeo SG, Kim KW, Kim MS, Rahmati M, Yon DK, Lee J. Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study. J Med Internet Res 2024; 26:e51640. [PMID: 38319694 PMCID: PMC10879967 DOI: 10.2196/51640] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2023] [Revised: 11/10/2023] [Accepted: 01/02/2024] [Indexed: 02/07/2024] Open
Abstract
BACKGROUND The outbreak of SARS-CoV-2 in 2019 has necessitated the rapid and accurate detection of COVID-19 to manage patients effectively and implement public health measures. Artificial intelligence (AI) models analyzing cough sounds have emerged as promising tools for large-scale screening and early identification of potential cases. OBJECTIVE This study aimed to investigate the efficacy of using cough sounds as a diagnostic tool for COVID-19, considering the unique acoustic features that differentiate positive and negative cases. We investigated whether an AI model trained on cough sound recordings from specific periods, especially the early stages of the COVID-19 pandemic, were applicable to the ongoing situation with persistent variants. METHODS We used cough sound recordings from 3 data sets (Cambridge, Coswara, and Virufy) representing different stages of the pandemic and variants. Our AI model was trained using the Cambridge data set with subsequent evaluation against all data sets. The performance was analyzed based on the area under the receiver operating curve (AUC) across different data measurement periods and COVID-19 variants. RESULTS The AI model demonstrated a high AUC when tested with the Cambridge data set, indicative of its initial effectiveness. However, the performance varied significantly with other data sets, particularly in detecting later variants such as Delta and Omicron, with a marked decline in AUC observed for the latter. These results highlight the challenges in maintaining the efficacy of AI models against the backdrop of an evolving virus. CONCLUSIONS While AI models analyzing cough sounds offer a promising noninvasive and rapid screening method for COVID-19, their effectiveness is challenged by the emergence of new virus variants. Ongoing research and adaptations in AI methodologies are crucial to address these limitations. The adaptability of AI models to evolve with the virus underscores their potential as a foundational technology for not only the current pandemic but also future outbreaks, contributing to a more agile and resilient global health infrastructure.
Collapse
Affiliation(s)
- Jina Kim
- Department of Biomedical Engineering, Kyung Hee University, Seoul, Republic of Korea
| | - Yong Sung Choi
- Department of Biomedical Engineering, Kyung Hee University, Seoul, Republic of Korea
| | - Young Joo Lee
- Department of Biomedical Engineering, Kyung Hee University, Seoul, Republic of Korea
| | - Seung Geun Yeo
- Department of Biomedical Engineering, Kyung Hee University, Seoul, Republic of Korea
| | - Kyung Won Kim
- Department of Radiology and Research Institute of Radiology, Asan Image Metrics, Clinical Trial Center, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Min Seo Kim
- Medical and Population Genetics and Cardiovascular Disease Initiative, Broad Institute of MIT and Harvard, Cambridge, MA, United States
| | - Masoud Rahmati
- Department of Physical Education and Sport Sciences, Faculty of Literature and Human Sciences, Lorestan University, Khoramabad, Iran
- Department of Physical Education and Sport Sciences, Faculty of Literature and Humanities, Vali-E-Asr University of Rafsanjan, Rafsanjan, Iran
| | - Dong Keon Yon
- Center for Digital Health, Medical Science Research Institute, Kyung Hee University Medical Center, Kyung Hee University College of Medicine, Seoul, Republic of Korea
- Department of Pediatrics, Kyung Hee University Medical Center, Kyung Hee University College of Medicine, Seoul, Republic of Korea
| | - Jinseok Lee
- Department of Biomedical Engineering, Kyung Hee University, Seoul, Republic of Korea
- Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin, Republic of Korea
| |
Collapse
|
6
|
Saeed T, Ijaz A, Sadiq I, Qureshi HN, Rizwan A, Imran A. An AI-Enabled Bias-Free Respiratory Disease Diagnosis Model Using Cough Audio. Bioengineering (Basel) 2024; 11:55. [PMID: 38247932 PMCID: PMC10813025 DOI: 10.3390/bioengineering11010055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 12/25/2023] [Accepted: 01/03/2024] [Indexed: 01/23/2024] Open
Abstract
Cough-based diagnosis for respiratory diseases (RDs) using artificial intelligence (AI) has attracted considerable attention, yet many existing studies overlook confounding variables in their predictive models. These variables can distort the relationship between cough recordings (input data) and RD status (output variable), leading to biased associations and unrealistic model performance. To address this gap, we propose the Bias-Free Network (RBF-Net), an end-to-end solution that effectively mitigates the impact of confounders in the training data distribution. RBF-Net ensures accurate and unbiased RD diagnosis features, emphasizing its relevance by incorporating a COVID-19 dataset in this study. This approach aims to enhance the reliability of AI-based RD diagnosis models by navigating the challenges posed by confounding variables. A hybrid of a Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks is proposed for the feature encoder module of RBF-Net. An additional bias predictor is incorporated in the classification scheme to formulate a conditional Generative Adversarial Network (c-GAN) that helps in decorrelating the impact of confounding variables from RD prediction. The merit of RBF-Net is demonstrated by comparing classification performance with a State-of-The-Art (SoTA) Deep Learning (DL) model (CNN-LSTM) after training on different unbalanced COVID-19 data sets, created by using a large-scale proprietary cough data set. RBF-Net proved its robustness against extremely biased training scenarios by achieving test set accuracies of 84.1%, 84.6%, and 80.5% for the following confounding variables-gender, age, and smoking status, respectively. RBF-Net outperforms the CNN-LSTM model test set accuracies by 5.5%, 7.7%, and 8.2%, respectively.
Collapse
Affiliation(s)
- Tabish Saeed
- AI4Networks Research Center, Department of Electrical & Computer Engineering, University of Oklahoma, Tulsa, OK 74135, USA; (H.N.Q.); (A.I.)
| | - Aneeqa Ijaz
- AI4Networks Research Center, Department of Electrical & Computer Engineering, University of Oklahoma, Tulsa, OK 74135, USA; (H.N.Q.); (A.I.)
| | - Ismail Sadiq
- James Watt School of Engineering, University of Glasgow, Glasgow G12 8QQ, UK;
| | - Haneya Naeem Qureshi
- AI4Networks Research Center, Department of Electrical & Computer Engineering, University of Oklahoma, Tulsa, OK 74135, USA; (H.N.Q.); (A.I.)
| | - Ali Rizwan
- AI4lyf, Bahria Town Lahore, Lahore 54000, Pakistan;
| | - Ali Imran
- AI4Networks Research Center, Department of Electrical & Computer Engineering, University of Oklahoma, Tulsa, OK 74135, USA; (H.N.Q.); (A.I.)
- James Watt School of Engineering, University of Glasgow, Glasgow G12 8QQ, UK;
| |
Collapse
|
7
|
Zarkogianni K, Dervakos E, Filandrianos G, Ganitidis T, Gkatzou V, Sakagianni A, Raghavendra R, Max Nikias CL, Stamou G, Nikita KS. The smarty4covid dataset and knowledge base as a framework for interpretable physiological audio data analysis. Sci Data 2023; 10:770. [PMID: 37932314 PMCID: PMC10628219 DOI: 10.1038/s41597-023-02646-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 10/10/2023] [Indexed: 11/08/2023] Open
Abstract
Harnessing the power of Artificial Intelligence (AI) and m-health towards detecting new bio-markers indicative of the onset and progress of respiratory abnormalities/conditions has greatly attracted the scientific and research interest especially during COVID-19 pandemic. The smarty4covid dataset contains audio signals of cough (4,676), regular breathing (4,665), deep breathing (4,695) and voice (4,291) as recorded by means of mobile devices following a crowd-sourcing approach. Other self reported information is also included (e.g. COVID-19 virus tests), thus providing a comprehensive dataset for the development of COVID-19 risk detection models. The smarty4covid dataset is released in the form of a web-ontology language (OWL) knowledge base enabling data consolidation from other relevant datasets, complex queries and reasoning. It has been utilized towards the development of models able to: (i) extract clinically informative respiratory indicators from regular breathing records, and (ii) identify cough, breath and voice segments in crowd-sourced audio recordings. A new framework utilizing the smarty4covid OWL knowledge base towards generating counterfactual explanations in opaque AI-based COVID-19 risk detection models is proposed and validated.
Collapse
Affiliation(s)
- Konstantia Zarkogianni
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece.
- Maastricht University, Faculty of Science and Engineering, Department of Advanced Computing Sciences, Maastricht, 6200 MD, Netherlands.
| | - Edmund Dervakos
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece
| | - George Filandrianos
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece
| | - Theofanis Ganitidis
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece
| | - Vasiliki Gkatzou
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece
| | - Aikaterini Sakagianni
- Sismanoglion General Hospital, Department of Intensive Care Unit, Athens, 15126, Greece
| | - Raghu Raghavendra
- University of Southern California, Viterbi School of Engineering, Los Angeles, 90089, USA
| | - C L Max Nikias
- University of Southern California, Viterbi School of Engineering, Los Angeles, 90089, USA
| | - Giorgos Stamou
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece
| | - Konstantina S Nikita
- National Technical University of Athens, School of Electrical and Computer Engineering, Athens, 157 80, Greece
- University of Southern California, Viterbi School of Engineering, Los Angeles, 90089, USA
| |
Collapse
|
8
|
Orlandic L, Teijeiro T, Atienza D. A semi-supervised algorithm for improving the consistency of crowdsourced datasets: The COVID-19 case study on respiratory disorder classification. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 241:107743. [PMID: 37598473 DOI: 10.1016/j.cmpb.2023.107743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 07/12/2023] [Accepted: 08/02/2023] [Indexed: 08/22/2023]
Abstract
BACKGROUND AND OBJECTIVE Cough audio signal classification is a potentially useful tool in screening for respiratory disorders, such as COVID-19. Since it is dangerous to collect data from patients with contagious diseases, many research teams have turned to crowdsourcing to quickly gather cough sound data. The COUGHVID dataset enlisted expert physicians to annotate and diagnose the underlying diseases present in a limited number of recordings. However, this approach suffers from potential cough mislabeling, as well as disagreement between experts. METHODS In this work, we use a semi-supervised learning (SSL) approach - based on audio signal processing tools and interpretable machine learning models - to improve the labeling consistency of the COUGHVID dataset for 1) COVID-19 versus healthy cough sound classification 2) distinguishing wet from dry coughs, and 3) assessing cough severity. First, we leverage SSL expert knowledge aggregation techniques to overcome the labeling inconsistencies and label sparsity in the dataset. Next, our SSL approach is used to identify a subsample of re-labeled COUGHVID audio samples that can be used to train or augment future cough classifiers. RESULTS The consistency of the re-labeled COVID-19 and healthy data is demonstrated in that it exhibits a high degree of inter-class feature separability: 3x higher than that of the user-labeled data. Similarly, the SSL method increases this separability by 11.3x for cough type and 5.1x for severity classifications. Furthermore, the spectral differences in the user-labeled audio segments are amplified in the re-labeled data, resulting in significantly different power spectral densities between healthy and COVID-19 coughs in the 1-1.5 kHz range (p=1.2×10-64), which demonstrates both the increased consistency of the new dataset and its explainability from an acoustic perspective. Finally, we demonstrate how the re-labeled dataset can be used to train a COVID-19 classifier, achieving an AUC of 0.797. CONCLUSIONS We propose a SSL expert knowledge aggregation technique for the field of cough sound classification for the first time, and demonstrate how it can be used to combine the medical knowledge of multiple experts in an explainable fashion, thus providing abundant, consistent data for cough classification tasks.
Collapse
Affiliation(s)
- Lara Orlandic
- Embedded Systems Laboratory (ESL), EPFL, Lausanne, Switzerland.
| | - Tomas Teijeiro
- Embedded Systems Laboratory (ESL), EPFL, Lausanne, Switzerland; Department of Mathematics, University of the Basque Country (UPV/EHU), Spain
| | - David Atienza
- Embedded Systems Laboratory (ESL), EPFL, Lausanne, Switzerland
| |
Collapse
|
9
|
Celik G. CovidCoughNet: A new method based on convolutional neural networks and deep feature extraction using pitch-shifting data augmentation for covid-19 detection from cough, breath, and voice signals. Comput Biol Med 2023; 163:107153. [PMID: 37321101 PMCID: PMC10249348 DOI: 10.1016/j.compbiomed.2023.107153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 05/25/2023] [Accepted: 06/07/2023] [Indexed: 06/17/2023]
Abstract
This study proposes a new deep learning-based method that demonstrates high performance in detecting Covid-19 disease from cough, breath, and voice signals. This impressive method, named CovidCoughNet, consists of a deep feature extraction network (InceptionFireNet) and a prediction network (DeepConvNet). The InceptionFireNet architecture, based on Inception and Fire modules, was designed to extract important feature maps. The DeepConvNet architecture, which is made up of convolutional neural network blocks, was developed to predict the feature vectors obtained from the InceptionFireNet architecture. The COUGHVID dataset containing cough data and the Coswara dataset containing cough, breath, and voice signals were used as the data sets. The pitch-shifting technique was used to data augmentation the signal data, which significantly contributed to improving performance. Additionally, Chroma features (CF), Root mean square energy (RMSE), Spectral centroid (SC), Spectral bandwidth (SB), Spectral rolloff (SR), Zero crossing rate (ZCR), and Mel frequency cepstral coefficients (MFCC) feature extraction techniques were used to extract important features from voice signals. Experimental studies have shown that using the pitch-shifting technique improved performance by around 3% compared to raw signals. When the proposed model was used with the COUGHVID dataset (Healthy, Covid-19, and Symptomatic), a high performance of 99.19% accuracy, 0.99 precision, 0.98 recall, 0.98 F1-Score, 97.77% specificity, and 98.44% AUC was achieved. Similarly, when the voice data in the Coswara dataset was used, higher performance was achieved compared to the cough and breath studies, with 99.63% accuracy, 100% precision, 0.99 recall, 0.99 F1-Score, 99.24% specificity, and 99.24% AUC. Moreover, when compared with current studies in the literature, the proposed model was observed to exhibit highly successful performance. The codes and details of the experimental studies can be accessed from the relevant Github page: (https://github.com/GaffariCelik/CovidCoughNet).
Collapse
Affiliation(s)
- Gaffari Celik
- Agri Ibrahim Cecen University, Department of Computer Technology, Agri, Turkey.
| |
Collapse
|
10
|
Alvarado E, Grágeda N, Luzanto A, Mahu R, Wuth J, Mendoza L, Stern RM, Yoma NB. Automatic Detection of Dyspnea in Real Human-Robot Interaction Scenarios. SENSORS (BASEL, SWITZERLAND) 2023; 23:7590. [PMID: 37688044 PMCID: PMC10490721 DOI: 10.3390/s23177590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 08/20/2023] [Accepted: 08/25/2023] [Indexed: 09/10/2023]
Abstract
A respiratory distress estimation technique for telephony previously proposed by the authors is adapted and evaluated in real static and dynamic HRI scenarios. The system is evaluated with a telephone dataset re-recorded using the robotic platform designed and implemented for this study. In addition, the original telephone training data are modified using an environmental model that incorporates natural robot-generated and external noise sources and reverberant effects using room impulse responses (RIRs). The results indicate that the average accuracy and AUC are just 0.4% less than those obtained with matched training/testing conditions with simulated data. Quite surprisingly, there is not much difference in accuracy and AUC between static and dynamic HRI conditions. Moreover, the beamforming methods delay-and-sum and MVDR lead to average improvement in accuracy and AUC equal to 8% and 2%, respectively, when applied to training and testing data. Regarding the complementarity of time-dependent and time-independent features, the combination of both types of classifiers provides the best joint accuracy and AUC score.
Collapse
Affiliation(s)
- Eduardo Alvarado
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile; (E.A.); (N.G.); (A.L.); (R.M.); (J.W.)
| | - Nicolás Grágeda
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile; (E.A.); (N.G.); (A.L.); (R.M.); (J.W.)
| | - Alejandro Luzanto
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile; (E.A.); (N.G.); (A.L.); (R.M.); (J.W.)
| | - Rodrigo Mahu
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile; (E.A.); (N.G.); (A.L.); (R.M.); (J.W.)
| | - Jorge Wuth
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile; (E.A.); (N.G.); (A.L.); (R.M.); (J.W.)
| | - Laura Mendoza
- Hospital Clínico Universidad de Chile, Santiago 8380420, Chile;
- Clínica Alemana, Santiago 7630000, Chile
| | - Richard M. Stern
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA;
| | - Néstor Becerra Yoma
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile; (E.A.); (N.G.); (A.L.); (R.M.); (J.W.)
| |
Collapse
|
11
|
Orlandic L, Thevenot J, Teijeiro T, Atienza D. A Multimodal Dataset for Automatic Edge-AI Cough Detection. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-7. [PMID: 38082667 DOI: 10.1109/embc40787.2023.10340413] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Counting the number of times a patient coughs per day is an essential biomarker in determining treatment efficacy for novel antitussive therapies and personalizing patient care. Automatic cough counting tools must provide accurate information, while running on a lightweight, portable device that protects the patient's privacy. Several devices and algorithms have been developed for cough counting, but many use only error-prone audio signals, rely on offline processing that compromises data privacy, or utilize processing and memory-intensive neural networks that require more hardware resources than can fit on a wearable device. Therefore, there is a need for wearable devices that employ multimodal sensors to perform accurate, privacy-preserving, automatic cough counting algorithms directly on the device in an edge Artificial Intelligence (edge-AI) fashion. To advance this research field, we contribute the first publicly accessible cough counting dataset of multimodal biosignals. The database contains nearly 4 hours of biosignal data, with both acoustic and kinematic modalities, covering 4,300 annotated cough events from 15 subjects. Furthermore, a variety of non-cough sounds and motion scenarios mimicking daily life activities are also present, which the research community can use to accelerate machine learning (ML) algorithm development. A technical validation of the dataset reveals that it represents a wide variety of signal-to-noise ratios, which can be expected in a real-life use case, as well as consistency across experimental trials. Finally, to demonstrate the usability of the dataset, we train a simple cough vs non-cough signal classifier that obtains a 91% sensitivity, 92% specificity, and 80% precision on unseen test subject data. Such edge-friendly AI algorithms have the potential to provide continuous ambulatory monitoring of the numerous chronic cough patients.
Collapse
|
12
|
Barahona JA, Mills K, Hernandez M, Bozkurt A, Carpenter D, Lobaton EJ. Adolescent Asthma Monitoring: A Preliminary Study of Audio and Spirometry Modalities. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083189 DOI: 10.1109/embc40787.2023.10340643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Asthma patients' sleep quality is correlated with how well their asthma symptoms are controlled. In this paper, deep learning techniques are explored to improve forecasting of forced expiratory volume in one second (FEV1) by using audio data from participants and test whether auditory sleep disturbances are correlated with poorer asthma outcomes. These are applied to a representative data set of FEV1 collected from a commercially available sprirometer and audio spectrograms collected overnight using a smartphone. A model for detecting nonverbal vocalizations including coughs, sneezes, sighs, snoring, throat clearing, sniffs, and breathing sounds was trained and used to capture nightly sleep disturbances. Our preliminary analysis found significant improvement in FEV1 forecasting when using overnight nonverbal vocalization detections as an additional feature for regression using XGBoost over using only spirometry data.Clinical relevance- This preliminary study establishes up to 30% improvement of FEV1 forecasting using features generated by deep learning techniques over only spirometry-based features.
Collapse
|
13
|
Bhattacharya D, Sharma NK, Dutta D, Chetupalli SR, Mote P, Ganapathy S, Chandrakiran C, Nori S, Suhail KK, Gonuguntla S, Alagesan M. Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection. Sci Data 2023; 10:397. [PMID: 37349364 PMCID: PMC10287715 DOI: 10.1038/s41597-023-02266-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 05/25/2023] [Indexed: 06/24/2023] Open
Abstract
This paper presents the Coswara dataset, a dataset containing diverse set of respiratory sounds and rich meta-data, recorded between April-2020 and February-2022 from 2635 individuals (1819 SARS-CoV-2 negative, 674 positive, and 142 recovered subjects). The respiratory sounds contained nine sound categories associated with variants of breathing, cough and speech. The rich metadata contained demographic information associated with age, gender and geographic location, as well as the health information relating to the symptoms, pre-existing respiratory ailments, comorbidity and SARS-CoV-2 test status. Our study is the first of its kind to manually annotate the audio quality of the entire dataset (amounting to 65 hours) through manual listening. The paper summarizes the data collection procedure, demographic, symptoms and audio data information. A COVID-19 classifier based on bi-directional long short-term (BLSTM) architecture, is trained and evaluated on the different population sub-groups contained in the dataset to understand the bias/fairness of the model. This enabled the analysis of the impact of gender, geographic location, date of recording, and language proficiency on the COVID-19 detection performance.
Collapse
Affiliation(s)
| | - Neeraj Kumar Sharma
- Mehta Family School of Data Science and Artificial Intelligence, Indian Institute of Technology Guwahati, Guwahati, India
| | - Debottam Dutta
- Department of Electrical Engineering, Indian Institute of Science, Bangalore, India
| | | | - Pravin Mote
- Department of Electrical Engineering, Indian Institute of Science, Bangalore, India
| | - Sriram Ganapathy
- Department of Electrical Engineering, Indian Institute of Science, Bangalore, India.
| | | | - Sahiti Nori
- Ramaiah Medical College Hospital, Bangalore, India
| | - K K Suhail
- Ramaiah Medical College Hospital, Bangalore, India
| | | | - Murali Alagesan
- PSG Institute of Medical Sciences and Research, Coimbatore, India
| |
Collapse
|
14
|
Pavel I, Ciocoiu IB. COVID-19 Detection from Cough Recordings Using Bag-of-Words Classifiers. SENSORS (BASEL, SWITZERLAND) 2023; 23:4996. [PMID: 37299721 PMCID: PMC10255075 DOI: 10.3390/s23114996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 05/21/2023] [Accepted: 05/22/2023] [Indexed: 06/12/2023]
Abstract
Reliable detection of COVID-19 from cough recordings is evaluated using bag-of-words classifiers. The effect of using four distinct feature extraction procedures and four different encoding strategies is evaluated in terms of the Area Under Curve (AUC), accuracy, sensitivity, and F1-score. Additional studies include assessing the effect of both input and output fusion approaches and a comparative analysis against 2D solutions using Convolutional Neural Networks. Extensive experiments conducted on the COUGHVID and COVID-19 Sounds datasets indicate that sparse encoding yields the best performances, showing robustness against various combinations of feature type, encoding strategy, and codebook dimension parameters.
Collapse
Affiliation(s)
| | - Iulian B. Ciocoiu
- Faculty of Electronics, Telecommunications and Information Technology, “Gheorghe Asachi” Technical University of Iasi, Bd. Carol I 11A, 700050 Iasi, Romania;
| |
Collapse
|
15
|
Cough sound detection from raw waveform using SincNet and bidirectional GRU. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
|
16
|
Coppock H, Akman A, Bergler C, Gerczuk M, Brown C, Chauhan J, Grammenos A, Hasthanasombat A, Spathis D, Xia T, Cicuta P, Han J, Amiriparian S, Baird A, Stappen L, Ottl S, Tzirakis P, Batliner A, Mascolo C, Schuller BW. A summary of the ComParE COVID-19 challenges. Front Digit Health 2023; 5:1058163. [PMID: 36969956 PMCID: PMC10031089 DOI: 10.3389/fdgth.2023.1058163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 01/09/2023] [Indexed: 03/29/2023] Open
Abstract
The COVID-19 pandemic has caused massive humanitarian and economic damage. Teams of scientists from a broad range of disciplines have searched for methods to help governments and communities combat the disease. One avenue from the machine learning field which has been explored is the prospect of a digital mass test which can detect COVID-19 from infected individuals' respiratory sounds. We present a summary of the results from the INTERSPEECH 2021 Computational Paralinguistics Challenges: COVID-19 Cough, (CCS) and COVID-19 Speech, (CSS).
Collapse
Affiliation(s)
- Harry Coppock
- Department of Computing, Imperial College London, London, United Kingdom
| | - Alican Akman
- Department of Computing, Imperial College London, London, United Kingdom
| | - Christian Bergler
- Department of Computing, FAU Erlangen-Nürnberg, Erlangen-Nürnberg, Germany
| | - Maurice Gerczuk
- Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | - Chloë Brown
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Jagmohan Chauhan
- Department of Computing, University of Southampton, Southampton, United Kingdom
| | - Andreas Grammenos
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Apinan Hasthanasombat
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Dimitris Spathis
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Tong Xia
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Pietro Cicuta
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Jing Han
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | | | - Alice Baird
- Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | - Lukas Stappen
- Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | - Sandra Ottl
- Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | | | - Anton Batliner
- Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| | - Cecilia Mascolo
- Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom
| | - Björn W. Schuller
- Department of Computing, Imperial College London, London, United Kingdom
- Institute of Computer Science, Universität Augsburg, Augsburg, Germany
| |
Collapse
|
17
|
Manzella F, Pagliarini G, Sciavicco G, Stan IE. The voice of COVID-19: Breath and cough recording classification with temporal decision trees and random forests. Artif Intell Med 2023; 137:102486. [PMID: 36868683 PMCID: PMC9904537 DOI: 10.1016/j.artmed.2022.102486] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Revised: 12/27/2022] [Accepted: 12/28/2022] [Indexed: 02/05/2023]
Abstract
Symbolic learning is the logic-based approach to machine learning, and its mission is to provide algorithms and methodologies to extract logical information from data and express it in an interpretable way. Interval temporal logic has been recently proposed as a suitable tool for symbolic learning, specifically via the design of an interval temporal logic decision tree extraction algorithm. In order to improve their performances, interval temporal decision trees can be embedded into interval temporal random forests, mimicking the corresponding schema at the propositional level. In this article we consider a dataset of cough and breath sample recordings of volunteer subjects, labeled with their COVID-19 status, originally collected by the University of Cambridge. By interpreting such recordings as multivariate time series, we study the problem of their automated classification using interval temporal decision trees and forests. While this problem has been approached with the same dataset as well as with other datasets, in all cases, non-symbolic learning methods (usually, deep learning-based) have been applied to solve it; in this article we apply a symbolic approach, and show that it does not only outperform the state-of-the-art obtained with the same dataset, but its results are also superior to those of most non-symbolic techniques applied on other datasets. As an added bonus, thanks to the symbolic nature of our approach, we are also able to extract explicit knowledge to help physicians characterize typical COVID-positive cough and breath.
Collapse
Affiliation(s)
- F Manzella
- Department of Mathematics and Computer Science, University of Ferrara, Italy.
| | - G Pagliarini
- Department of Mathematics and Computer Science, University of Ferrara, Italy.
| | - G Sciavicco
- Department of Mathematics and Computer Science, University of Ferrara, Italy.
| | - I E Stan
- Department of Mathematics and Computer Science, University of Ferrara, Italy.
| |
Collapse
|
18
|
Nguyen LH, Pham NT, Do VH, Nguyen LT, Nguyen TT, Nguyen H, Nguyen ND, Nguyen TT, Nguyen SD, Bhatti A, Lim CP. Fruit-CoV: An efficient vision-based framework for speedy detection and diagnosis of SARS-CoV-2 infections through recorded cough sounds. EXPERT SYSTEMS WITH APPLICATIONS 2023; 213:119212. [PMID: 36407848 PMCID: PMC9639421 DOI: 10.1016/j.eswa.2022.119212] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Revised: 10/20/2022] [Accepted: 11/01/2022] [Indexed: 06/16/2023]
Abstract
COVID-19 is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This deadly virus has spread worldwide, leading to a global pandemic since March 2020. A recent variant of SARS-CoV-2 named Delta is intractably contagious and responsible for more than four million deaths globally. Therefore, developing an efficient self-testing service for SARS-CoV-2 at home is vital. In this study, a two-stage vision-based framework, namely Fruit-CoV, is introduced for detecting SARS-CoV-2 infections through recorded cough sounds. Specifically, audio signals are converted into Log-Mel spectrograms, and the EfficientNet-V2 network is used to extract their visual features in the first stage. In the second stage, 14 convolutional layers extracted from the large-scale Pretrained Audio Neural Networks for audio pattern recognition (PANNs) and the Wavegram-Log-Mel-CNN are employed to aggregate feature representations of the Log-Mel spectrograms and the waveform. Finally, the combined features are used to train a binary classifier. In this study, a dataset provided by the AICovidVN 115M Challenge is employed for evaluation. It includes 7,371 recorded cough sounds collected throughout Vietnam, India, and Switzerland. Experimental results indicate that the proposed model achieves an Area Under the Receiver Operating Characteristic Curve (AUC) score of 92.8% and ranks first on the final leaderboard of the AICovidVN 115M Challenge. Our code is publicly available.
Collapse
Affiliation(s)
- Long H Nguyen
- Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam
| | - Nhat Truong Pham
- Division of Computational Mechatronics, Institute for Computational Science, Ton Duc Thang University, Ho Chi Minh City, Vietnam
- Faculty of Electrical and Electronics Engineering, Ton Duc Thang University, Ho Chi Minh City, Vietnam
| | | | - Liu Tai Nguyen
- Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam
| | - Thanh Tin Nguyen
- Human Computer Interaction Lab, Sejong University, Seoul, South Korea
| | - Hai Nguyen
- Khoury College of Computer Sciences, Northeastern University, Boston, USA
| | - Ngoc Duy Nguyen
- Institute for Intelligent Systems Research and Innovation, Deakin University, Victoria, Australia
| | - Thanh Thi Nguyen
- School of Information Technology, Deakin University, Victoria, Australia
| | - Sy Dzung Nguyen
- Laboratory for Computational Mechatronics, Institute for Computational Science and Artificial Intelligence, Van Lang University, Ho Chi Minh City, Vietnam
- Faculty of Mechanical-Electrical and Computer Engineering, Van Lang University, Ho Chi Minh City, Vietnam
| | - Asim Bhatti
- Institute for Intelligent Systems Research and Innovation, Deakin University, Victoria, Australia
| | - Chee Peng Lim
- Institute for Intelligent Systems Research and Innovation, Deakin University, Victoria, Australia
| |
Collapse
|
19
|
Ulukaya S, Sarıca AA, Erdem O, Karaali A. MSCCov19Net: multi-branch deep learning model for COVID-19 detection from cough sounds. Med Biol Eng Comput 2023:10.1007/s11517-023-02803-4. [PMID: 36828944 PMCID: PMC9955529 DOI: 10.1007/s11517-023-02803-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Accepted: 01/24/2023] [Indexed: 02/26/2023]
Abstract
Coronavirus has an impact on millions of lives and has been added to the important pandemics that continue to affect with its variants. Since it is transmitted through the respiratory tract, it has had significant effects on public health and social relations. Isolating people who are COVID positive can minimize the transmission, therefore several exams are proposed to detect the virus such as reverse transcription-polymerase chain reaction (RT-PCR), chest X-Ray, and computed tomography (CT). However, these methods suffer from either a low detection rate or high radiation dosage, along with being expensive. In this study, deep neural network-based model capable of detecting coronavirus from only coughing sound, which is fast, remotely operable and has no harmful side effects, has been proposed. The proposed multi-branch model takes M el Frequency Cepstral Coefficients (MFCC), S pectrogram, and C hromagram as inputs and is abbreviated as MSCCov19Net. The system is trained on publicly available crowdsourced datasets, and tested on two unseen (used only for testing) clinical and non-clinical datasets. Experimental outcomes represent that the proposed system outperforms the 6 popular deep learning architectures on four datasets by representing a better generalization ability. The proposed system has reached an accuracy of 61.5 % in Virufy and 90.4 % in NoCoCoDa for unseen test datasets.
Collapse
Affiliation(s)
- Sezer Ulukaya
- Department of Electrical and Electronics Engineering, Trakya University, Edirne, 22030 Turkey
| | - Ahmet Alp Sarıca
- Department of Electrical and Electronics Engineering, Trakya University, Edirne, 22030 Turkey
| | - Oğuzhan Erdem
- Department of Electrical and Electronics Engineering, Trakya University, Edirne, 22030 Turkey
| | - Ali Karaali
- The Irish Longitudinal Study on Ageing (TILDA), School of Medicine, Trinity College Dublin, Dublin, D02 R590 Ireland
| |
Collapse
|
20
|
Alvarado E, Grágeda N, Luzanto A, Mahu R, Wuth J, Mendoza L, Yoma NB. Dyspnea Severity Assessment Based on Vocalization Behavior with Deep Learning on the Telephone. SENSORS (BASEL, SWITZERLAND) 2023; 23:2441. [PMID: 36904646 PMCID: PMC10007248 DOI: 10.3390/s23052441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 02/08/2023] [Accepted: 02/17/2023] [Indexed: 06/18/2023]
Abstract
In this paper, a system to assess dyspnea with the mMRC scale, on the phone, via deep learning, is proposed. The method is based on modeling the spontaneous behavior of subjects while pronouncing controlled phonetization. These vocalizations were designed, or chosen, to deal with the stationary noise suppression of cellular handsets, to provoke different rates of exhaled air, and to stimulate different levels of fluency. Time-independent and time-dependent engineered features were proposed and selected, and a k-fold scheme with double validation was adopted to select the models with the greatest potential for generalization. Moreover, score fusion methods were also investigated to optimize the complementarity of the controlled phonetizations and features that were engineered and selected. The results reported here were obtained from 104 participants, where 34 corresponded to healthy individuals and 70 were patients with respiratory conditions. The subjects' vocalizations were recorded with a telephone call (i.e., with an IVR server). The system provided an accuracy of 59% (i.e., estimating the correct mMRC), a root mean square error equal to 0.98, false positive rate of 6%, false negative rate of 11%, and an area under the ROC curve equal to 0.97. Finally, a prototype was developed and implemented, with an ASR-based automatic segmentation scheme, to estimate dyspnea on line.
Collapse
Affiliation(s)
- Eduardo Alvarado
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile
| | - Nicolás Grágeda
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile
| | - Alejandro Luzanto
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile
| | - Rodrigo Mahu
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile
| | - Jorge Wuth
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile
| | - Laura Mendoza
- Clinical Hospital, University of Chile, Santiago 8380420, Chile
| | - Néstor Becerra Yoma
- Speech Processing and Transmission Laboratory, Electrical Engineering Department, University of Chile, Santiago 8370451, Chile
| |
Collapse
|
21
|
A Performance Study of CNN Architectures for the Autonomous Detection of COVID-19 Symptoms Using Cough and Breathing. COMPUTERS 2023. [DOI: 10.3390/computers12020044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/22/2023]
Abstract
Deep learning (DL) methods have the potential to be used for detecting COVID-19 symptoms. However, the rationale for which DL method to use and which symptoms to detect has not yet been explored. In this paper, we present the first performance study which compares various convolutional neural network (CNN) architectures for the autonomous preliminary COVID-19 detection of cough and/or breathing symptoms. We compare and analyze residual networks (ResNets), visual geometry Groups (VGGs), Alex neural networks (AlexNet), densely connected networks (DenseNet), squeeze neural networks (SqueezeNet), and COVID-19 identification ResNet (CIdeR) architectures to investigate their classification performance. We uniquely train and validate both unimodal and multimodal CNN architectures using the EPFL and Cambridge datasets. Performance comparison across all modes and datasets showed that the VGG19 and DenseNet-201 achieved the highest unimodal and multimodal classification performance. VGG19 and DensNet-201 had high F1 scores (0.94 and 0.92) for unimodal cough classification on the Cambridge dataset, compared to the next highest F1 score for ResNet (0.79), with comparable F1 scores to ResNet for the larger EPFL cough dataset. They also had consistently high accuracy, recall, and precision. For multimodal detection, VGG19 and DenseNet-201 had the highest F1 scores (0.91) compared to the other CNN structures (≤0.90), with VGG19 also having the highest accuracy and recall. Our investigation provides the foundation needed to select the appropriate deep CNN method to utilize for non-contact early COVID-19 detection.
Collapse
|
22
|
Campana MG, Delmastro F, Pagani E. Transfer learning for the efficient detection of COVID-19 from smartphone audio data. PERVASIVE AND MOBILE COMPUTING 2023; 89:101754. [PMID: 36741300 PMCID: PMC9884612 DOI: 10.1016/j.pmcj.2023.101754] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 01/18/2023] [Accepted: 01/25/2023] [Indexed: 06/12/2023]
Abstract
Disease detection from smartphone data represents an open research challenge in mobile health (m-health) systems. COVID-19 and its respiratory symptoms are an important case study in this area and their early detection is a potential real instrument to counteract the pandemic situation. The efficacy of this solution mainly depends on the performances of AI algorithms applied to the collected data and their possible implementation directly on the users' mobile devices. Considering these issues, and the limited amount of available data, in this paper we present the experimental evaluation of 3 different deep learning models, compared also with hand-crafted features, and of two main approaches of transfer learning in the considered scenario: both feature extraction and fine-tuning. Specifically, we considered VGGish, YAMNET, and L3-Net (including 12 different configurations) evaluated through user-independent experiments on 4 different datasets (13,447 samples in total). Results clearly show the advantages of L3-Net in all the experimental settings as it overcomes the other solutions by 12.3% in terms of Precision-Recall AUC as features extractor, and by 10% when the model is fine-tuned. Moreover, we note that to fine-tune only the fully-connected layers of the pre-trained models generally leads to worse performances, with an average drop of 6.6% with respect to feature extraction. Finally, we evaluate the memory footprints of the different models for their possible applications on commercial mobile devices.
Collapse
Affiliation(s)
- Mattia Giovanni Campana
- Institute for Informatics and Telematics of the National Research Council of Italy (IIT-CNR), Pisa, Italy
| | - Franca Delmastro
- Institute for Informatics and Telematics of the National Research Council of Italy (IIT-CNR), Pisa, Italy
| | - Elena Pagani
- Computer Science Department, University of Milano, Milan, Italy
| |
Collapse
|
23
|
Zhang J, Wu J, Qiu Y, Song A, Li W, Li X, Liu Y. Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review. Comput Biol Med 2023; 153:106517. [PMID: 36623438 PMCID: PMC9814440 DOI: 10.1016/j.compbiomed.2022.106517] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 12/23/2022] [Accepted: 12/31/2022] [Indexed: 01/07/2023]
Abstract
The growing and aging of the world population have driven the shortage of medical resources in recent years, especially during the COVID-19 pandemic. Fortunately, the rapid development of robotics and artificial intelligence technologies help to adapt to the challenges in the healthcare field. Among them, intelligent speech technology (IST) has served doctors and patients to improve the efficiency of medical behavior and alleviate the medical burden. However, problems like noise interference in complex medical scenarios and pronunciation differences between patients and healthy people hamper the broad application of IST in hospitals. In recent years, technologies such as machine learning have developed rapidly in intelligent speech recognition, which is expected to solve these problems. This paper first introduces IST's procedure and system architecture and analyzes its application in medical scenarios. Secondly, we review existing IST applications in smart hospitals in detail, including electronic medical documentation, disease diagnosis and evaluation, and human-medical equipment interaction. In addition, we elaborate on an application case of IST in the early recognition, diagnosis, rehabilitation training, evaluation, and daily care of stroke patients. Finally, we discuss IST's limitations, challenges, and future directions in the medical field. Furthermore, we propose a novel medical voice analysis system architecture that employs active hardware, active software, and human-computer interaction to realize intelligent and evolvable speech recognition. This comprehensive review and the proposed architecture offer directions for future studies on IST and its applications in smart hospitals.
Collapse
Affiliation(s)
- Jun Zhang
- The State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China,Corresponding author
| | - Jingyue Wu
- The State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China
| | - Yiyi Qiu
- The State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China
| | - Aiguo Song
- The State Key Laboratory of Bioelectronics, School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China
| | - Weifeng Li
- Department of Emergency Medicine, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, 510080, China
| | - Xin Li
- Department of Emergency Medicine, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, 510080, China
| | - Yecheng Liu
- Emergency Department, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, 100730, China
| |
Collapse
|
24
|
Najaran MHT. An evolutionary ensemble learning for diagnosing COVID-19 via cough signals. INTELLIGENT MEDICINE 2023; 3:S2667-1026(23)00002-5. [PMID: 36743333 PMCID: PMC9882956 DOI: 10.1016/j.imed.2023.01.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 01/10/2023] [Accepted: 01/11/2023] [Indexed: 01/30/2023]
Abstract
Objective The spread of the COVID-19 disease has caused great concern around the world and detecting the positive cases is crucial in curbing the pandemic. One of the symptoms of the disease is the dry cough it causes. It has previously been shown that cough signals can be used to identify a variety of diseases including tuberculosis, asthma, etc. In this paper, we proposed an algorithm to diagnose via cough signals the COVID-19 disease. Methods The proposed algorithm is an ensemble scheme that consists of a number of base learners, where each base learner uses a different feature extractor method, including statistical approaches and convolutional neural networks (CNN) for automatic feature extraction. Features are extracted from the raw signal and some transforms performed it, including Fourier, wavelet, Hilbert-Huang, and short-term Fourier transforms. The outputs of these base-learners are aggregated via a weighted voting scheme, with the weights optimised via an evolutionary paradigm. This paper also proposes a memetic algorithm for training the CNNs in the base-learners, which combines the speed of gradient descent (GD) algorithms and global search space coverage of the evolutionary algorithms. Results Experiments were performed on the proposed algorithm and different rival algorithms which included a number of CNN architectures in the literature and generic machine learning algorithms. The results suggested that the proposed algorithm achieves better performance compared to the existing algorithms in diagnosing COVID-19 via cough signals. Conclusion This research showed that COVID-19 could be diagnosed via cough signals and CNNs could be employed to process these signals and it may be further improved by the optimization of CNN architecture.
Collapse
|
25
|
Kuhn M, Nalbant E, Kohlbrenner D, Alge M, Kuett L, Arvaji A, Sievi NA, Russi EW, Clarenbach CF. Validation of a small cough detector. ERJ Open Res 2023; 9:00279-2022. [PMID: 36699651 PMCID: PMC9868968 DOI: 10.1183/23120541.00279-2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 09/23/2022] [Indexed: 01/28/2023] Open
Abstract
Research question The assessment of cough frequency in clinical practice relies predominantly on the patient's history. Currently, objective evaluation of cough is feasible with bulky equipment during a brief time (i.e. hours up to 1 day). Thus, monitoring of cough has been rarely performed outside clinical studies. We developed a small wearable cough detector (SIVA-P3) that uses deep neural networks for the automatic counting of coughs. This study examined the performance of the SIVA-P3 in an outpatient setting. Methods We recorded cough epochs with SIVA-P3 over eight consecutive days in patients suffering from chronic cough. During the first 24 h, the detector was validated against cough events counted by trained human listeners. The wearing comfort and the device usage were assessed using a questionnaire. Results In total, 27 participants (mean±sd age 50±14 years) with either chronic unexplained cough (n=12), COPD (n=4), asthma (n=5) or interstitial lung disease (n=6) were studied. During the daytime, the sensitivity of SIVA-P3 cough detection was 88.5±2.49% and the specificity was 99.97±0.01%. During the night-time, the sensitivity was 84.15±5.04% and the specificity was 99.97±0.02%. The wearing comfort and usage of the device was rated as very high by most participants. Conclusion SIVA-P3 enables automatic continuous cough monitoring in an outpatient setting for objective assessment of cough over days and weeks. It shows comparable sensitivity or higher sensitivity than other devices with fully automatic cough counting. Thanks to its wearing comfort and the high performance for cough detection, it has the potential for being used in routine clinical practice.
Collapse
Affiliation(s)
- Manuel Kuhn
- Faculty of Medicine, University of Zurich, Zurich, Switzerland,Department of Pulmonology, University Hospital Zurich, Zurich, Switzerland,Corresponding author: Manuel Kuhn ()
| | | | - Dario Kohlbrenner
- Department of Pulmonology, University Hospital Zurich, Zurich, Switzerland
| | | | | | - Alexandra Arvaji
- Department of Pulmonology, University Hospital Zurich, Zurich, Switzerland
| | - Noriane A. Sievi
- Department of Pulmonology, University Hospital Zurich, Zurich, Switzerland
| | - Erich W. Russi
- Faculty of Medicine, University of Zurich, Zurich, Switzerland
| | - Christian F. Clarenbach
- Faculty of Medicine, University of Zurich, Zurich, Switzerland,Department of Pulmonology, University Hospital Zurich, Zurich, Switzerland
| |
Collapse
|
26
|
Isgut M, Gloster L, Choi K, Venugopalan J, Wang MD. Systematic Review of Advanced AI Methods for Improving Healthcare Data Quality in Post COVID-19 Era. IEEE Rev Biomed Eng 2023; 16:53-69. [PMID: 36269930 DOI: 10.1109/rbme.2022.3216531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
At the beginning of the COVID-19 pandemic, there was significant hype about the potential impact of artificial intelligence (AI) tools in combatting COVID-19 on diagnosis, prognosis, or surveillance. However, AI tools have not yet been widely successful. One of the key reason is the COVID-19 pandemic has demanded faster real-time development of AI-driven clinical and health support tools, including rapid data collection, algorithm development, validation, and deployment. However, there was not enough time for proper data quality control. Learning from the hard lessons in COVID-19, we summarize the important health data quality challenges during COVID-19 pandemic such as lack of data standardization, missing data, tabulation errors, and noise and artifact. Then we conduct a systematic investigation of computational methods that address these issues, including emerging novel advanced AI data quality control methods that achieve better data quality outcomes and, in some cases, simplify or automate the data cleaning process. We hope this article can assist healthcare community to improve health data quality going forward with novel AI development.
Collapse
|
27
|
Reliability of crowdsourced data and patient-reported outcome measures in cough-based COVID-19 screening. Sci Rep 2022; 12:21990. [PMID: 36539519 PMCID: PMC9764298 DOI: 10.1038/s41598-022-26492-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2022] [Accepted: 12/15/2022] [Indexed: 12/25/2022] Open
Abstract
Mass community testing is a critical means for monitoring the spread of the COVID-19 pandemic. Polymerase chain reaction (PCR) is the gold standard for detecting the causative coronavirus 2 (SARS-CoV-2) but the test is invasive, test centers may not be readily available, and the wait for laboratory results can take several days. Various machine learning based alternatives to PCR screening for SARS-CoV-2 have been proposed, including cough sound analysis. Cough classification models appear to be a robust means to predict infective status, but collecting reliable PCR confirmed data for their development is challenging and recent work using unverified crowdsourced data is seen as a viable alternative. In this study, we report experiments that assess cough classification models trained (i) using data from PCR-confirmed COVID subjects and (ii) using data of individuals self-reporting their infective status. We compare performance using PCR-confirmed data. Models trained on PCR-confirmed data perform better than those trained on patient-reported data. Models using PCR-confirmed data also exploit more stable predictive features and converge faster. Crowd-sourced cough data is less reliable than PCR-confirmed data for developing predictive models for COVID-19, and raises concerns about the utility of patient reported outcome data in developing other clinical predictive models when better gold-standard data are available.
Collapse
|
28
|
Grant D, McLane I, Rennoll V, West J. Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System. SENSORS (BASEL, SWITZERLAND) 2022; 22:9530. [PMID: 36502232 PMCID: PMC9739601 DOI: 10.3390/s22239530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 11/30/2022] [Accepted: 12/03/2022] [Indexed: 06/17/2023]
Abstract
Coronavirus disease 2019 (COVID-19) has led to countless deaths and widespread global disruptions. Acoustic-based artificial intelligence (AI) tools could provide a simple, scalable, and prompt method to screen for COVID-19 using easily acquirable physiological sounds. These systems have been demonstrated previously and have shown promise but lack robust analysis of their deployment in real-world settings when faced with diverse recording equipment, noise environments, and test subjects. The primary aim of this work is to begin to understand the impacts of these real-world deployment challenges on the system performance. Using Mel-Frequency Cepstral Coefficients (MFCC) and RelAtive SpecTrAl-Perceptual Linear Prediction (RASTA-PLP) features extracted from cough, speech, and breathing sounds in a crowdsourced dataset, we present a baseline classification system that obtains an average receiver operating characteristic area under the curve (AUC-ROC) of 0.77 when discriminating between COVID-19 and non-COVID subjects. The classifier performance is then evaluated on four additional datasets, resulting in performance variations between 0.64 and 0.87 AUC-ROC, depending on the sound type. By analyzing subsets of the available recordings, it is noted that the system performance degrades with certain recording devices, noise contamination, and with symptom status. Furthermore, performance degrades when a uniform classification threshold from the training data is subsequently used across all datasets. However, the system performance is robust to confounding factors, such as gender, age group, and the presence of other respiratory conditions. Finally, when analyzing multiple speech recordings from the same subjects, the system achieves promising performance with an AUC-ROC of 0.78, though the classification does appear to be impacted by natural speech variations. Overall, the proposed system, and by extension other acoustic-based diagnostic aids in the literature, could provide comparable accuracy to rapid antigen testing but significant deployment challenges need to be understood and addressed prior to clinical use.
Collapse
|
29
|
Shen J, Ghatti S, Levkov NR, Shen H, Sen T, Rheuban K, Enfield K, Facteau NR, Engel G, Dowdell K. A survey of COVID-19 detection and prediction approaches using mobile devices, AI, and telemedicine. Front Artif Intell 2022; 5:1034732. [PMID: 36530356 PMCID: PMC9755752 DOI: 10.3389/frai.2022.1034732] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Accepted: 11/02/2022] [Indexed: 09/19/2023] Open
Abstract
Since 2019, the COVID-19 pandemic has had an extremely high impact on all facets of the society and will potentially have an everlasting impact for years to come. In response to this, over the past years, there have been a significant number of research efforts on exploring approaches to combat COVID-19. In this paper, we present a survey of the current research efforts on using mobile Internet of Thing (IoT) devices, Artificial Intelligence (AI), and telemedicine for COVID-19 detection and prediction. We first present the background and then present current research in this field. Specifically, we present the research on COVID-19 monitoring and detection, contact tracing, machine learning based approaches, telemedicine, and security. We finally discuss the challenges and the future work that lay ahead in this field before concluding this paper.
Collapse
Affiliation(s)
- John Shen
- Department of Computer Science, University of Virginia, Charlottesville, VA, United States
| | - Siddharth Ghatti
- Department of Computer Science, University of Virginia, Charlottesville, VA, United States
| | - Nate Ryan Levkov
- Department of Computer Science, University of Virginia, Charlottesville, VA, United States
| | - Haiying Shen
- Department of Computer Science, University of Virginia, Charlottesville, VA, United States
| | - Tanmoy Sen
- Department of Computer Science, University of Virginia, Charlottesville, VA, United States
| | - Karen Rheuban
- School of Medicine, University of Virginia, Charlottesville, VA, United States
| | - Kyle Enfield
- School of Medicine, University of Virginia, Charlottesville, VA, United States
| | - Nikki Reyer Facteau
- University of Virginia (UVA) Health System, University of Virginia, Charlottesville, VA, United States
| | - Gina Engel
- School of Medicine, University of Virginia, Charlottesville, VA, United States
| | - Kim Dowdell
- School of Medicine, University of Virginia, Charlottesville, VA, United States
| |
Collapse
|
30
|
Nguyen-Trong K, Nguyen-Hoang K. Multi-modal approach for COVID-19 detection using coughs and self-reported symptoms. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-222863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
COVID-19 (Coronavirus Disease of 2019) is one of the most challenging healthcare crises of the twenty-first century. The pandemic causes many negative impacts on all aspects of life and livelihoods. Although recent developments of relevant vaccines, such as Pfizer/BioNTech mRNA, AstraZeneca, or Moderna, the emergence of new virus mutations and their fast infection rate yet pose significant threats to public health. In this context, early detection of the disease is an important factor to reduce its effect and quickly control the spread of pandemic. Nevertheless, many countries still rely on methods that are either expensive and time-consuming (i.e., Reverse-transcription polymerase chain reaction) or uncomfortable and difficult for self-testing (i.e., Rapid Antigen Test Nasal). Recently, deep learning methods have been proposed as a potential solution for COVID-19 analysis. However, previous works usually focus on a single symptom, which can omit critical information for disease diagnosis. Therefore, in this study, we propose a multi-modal method to detect COVID-19 using cough sounds and self-reported symptoms. The proposed method consists of five neural networks to deal with different input features, including CNN-biLSTM for MFCC features, EfficientNetV2 for Mel spectrogram images, MLP for self-reported symptoms, C-YAMNet for cough detection, and RNNoise for noise-canceling. Experimental results demonstrated that our method outperformed the other state-of-the-art methods with a high AUC, accuracy, and F1-score of 98.6%, 96.9%, and 96.9% on the testing set.
Collapse
Affiliation(s)
- Khanh Nguyen-Trong
- Faculty of Information Technology, Posts and Telecommunications Institute of Technology, Hanoi, Viet Nam
| | - Khoi Nguyen-Hoang
- Faculty of Information Technology, Posts and Telecommunications Institute of Technology, Hanoi, Viet Nam
| |
Collapse
|
31
|
Elbéji A, Zhang L, Higa E, Fischer A, Despotovic V, Nazarov PV, Aguayo G, Fagherazzi G. Vocal biomarker predicts fatigue in people with COVID-19: results from the prospective Predi-COVID cohort study. BMJ Open 2022; 12:e062463. [PMID: 36414294 PMCID: PMC9684280 DOI: 10.1136/bmjopen-2022-062463] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
OBJECTIVE To develop a vocal biomarker for fatigue monitoring in people with COVID-19. DESIGN Prospective cohort study. SETTING Predi-COVID data between May 2020 and May 2021. PARTICIPANTS A total of 1772 voice recordings were used to train an AI-based algorithm to predict fatigue, stratified by gender and smartphone's operating system (Android/iOS). The recordings were collected from 296 participants tracked for 2 weeks following SARS-CoV-2 infection. PRIMARY AND SECONDARY OUTCOME MEASURES Four machine learning algorithms (logistic regression, k-nearest neighbours, support vector machine and soft voting classifier) were used to train and derive the fatigue vocal biomarker. The models were evaluated based on the following metrics: area under the curve (AUC), accuracy, F1-score, precision and recall. The Brier score was also used to evaluate the models' calibrations. RESULTS The final study population included 56% of women and had a mean (±SD) age of 40 (±13) years. Women were more likely to report fatigue (p<0.001). We developed four models for Android female, Android male, iOS female and iOS male users with a weighted AUC of 86%, 82%, 79%, 85% and a mean Brier Score of 0.15, 0.12, 0.17, 0.12, respectively. The vocal biomarker derived from the prediction models successfully discriminated COVID-19 participants with and without fatigue. CONCLUSIONS This study demonstrates the feasibility of identifying and remotely monitoring fatigue thanks to voice. Vocal biomarkers, digitally integrated into telemedicine technologies, are expected to improve the monitoring of people with COVID-19 or Long-COVID. TRIAL REGISTRATION NUMBER NCT04380987.
Collapse
Affiliation(s)
- Abir Elbéji
- Department of Precision Health, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Lu Zhang
- Department of Precision Health, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Eduardo Higa
- Department of Precision Health, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Aurélie Fischer
- Department of Precision Health, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Vladimir Despotovic
- Bioinformatics Platform, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Petr V Nazarov
- Bioinformatics Platform, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Gloria Aguayo
- Department of Precision Health, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Guy Fagherazzi
- Department of Precision Health, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| |
Collapse
|
32
|
Xia T, Han J, Mascolo C. Exploring machine learning for audio-based respiratory condition screening: A concise review of databases, methods, and open issues. Exp Biol Med (Maywood) 2022; 247:2053-2061. [PMID: 35974706 PMCID: PMC9791302 DOI: 10.1177/15353702221115428] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022] Open
Abstract
Auscultation plays an important role in the clinic, and the research community has been exploring machine learning (ML) to enable remote and automatic auscultation for respiratory condition screening via sounds. To give the big picture of what is going on in this field, in this narrative review, we describe publicly available audio databases that can be used for experiments, illustrate the developed ML methods proposed to date, and flag some under-considered issues which still need attention. Compared to existing surveys on the topic, we cover the latest literature, especially those audio-based COVID-19 detection studies which have gained extensive attention in the last two years. This work can help to facilitate the application of artificial intelligence in the respiratory auscultation field.
Collapse
|
33
|
Aleixandre JG, Elgendi M, Menon C. The Use of Audio Signals for Detecting COVID-19: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:8114. [PMID: 36365811 PMCID: PMC9653621 DOI: 10.3390/s22218114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 10/17/2022] [Accepted: 10/20/2022] [Indexed: 06/16/2023]
Abstract
A systematic review on the topic of automatic detection of COVID-19 using audio signals was performed. A total of 48 papers were obtained after screening 659 records identified in the PubMed, IEEE Xplore, Embase, and Google Scholar databases. The reviewed studies employ a mixture of open-access and self-collected datasets. Because COVID-19 has only recently been investigated, there is a limited amount of available data. Most of the data are crowdsourced, which motivated a detailed study of the various pre-processing techniques used by the reviewed studies. Although 13 of the 48 identified papers show promising results, several have been performed with small-scale datasets (<200). Among those papers, convolutional neural networks and support vector machine algorithms were the best-performing methods. The analysis of the extracted features showed that Mel-frequency cepstral coefficients and zero-crossing rate continue to be the most popular choices. Less common alternatives, such as non-linear features, have also been proven to be effective. The reported values for sensitivity range from 65.0% to 99.8% and those for accuracy from 59.0% to 99.8%.
Collapse
Affiliation(s)
- José Gómez Aleixandre
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Department of Physics, ETH Zurich, 8093 Zurich, Switzerland
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
| |
Collapse
|
34
|
Fagherazzi G, Zhang L, Elbéji A, Higa E, Despotovic V, Ollert M, Aguayo GA, Nazarov PV, Fischer A. A voice-based biomarker for monitoring symptom resolution in adults with COVID-19: Findings from the prospective Predi-COVID cohort study. PLOS DIGITAL HEALTH 2022; 1:e0000112. [PMID: 36812535 PMCID: PMC9931359 DOI: 10.1371/journal.pdig.0000112] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Accepted: 08/26/2022] [Indexed: 11/06/2022]
Abstract
People with COVID-19 can experience impairing symptoms that require enhanced surveillance. Our objective was to train an artificial intelligence-based model to predict the presence of COVID-19 symptoms and derive a digital vocal biomarker for easily and quantitatively monitoring symptom resolution. We used data from 272 participants in the prospective Predi-COVID cohort study recruited between May 2020 and May 2021. A total of 6473 voice features were derived from recordings of participants reading a standardized pre-specified text. Models were trained separately for Android devices and iOS devices. A binary outcome (symptomatic versus asymptomatic) was considered, based on a list of 14 frequent COVID-19 related symptoms. A total of 1775 audio recordings were analyzed (6.5 recordings per participant on average), including 1049 corresponding to symptomatic cases and 726 to asymptomatic ones. The best performances were obtained from Support Vector Machine models for both audio formats. We observed an elevated predictive capacity for both Android (AUC = 0.92, balanced accuracy = 0.83) and iOS (AUC = 0.85, balanced accuracy = 0.77) as well as low Brier scores (0.11 and 0.16 respectively for Android and iOS when assessing calibration. The vocal biomarker derived from the predictive models accurately discriminated asymptomatic from symptomatic individuals with COVID-19 (t-test P-values<0.001). In this prospective cohort study, we have demonstrated that using a simple, reproducible task of reading a standardized pre-specified text of 25 seconds enabled us to derive a vocal biomarker for monitoring the resolution of COVID-19 related symptoms with high accuracy and calibration.
Collapse
Affiliation(s)
- Guy Fagherazzi
- Deep Digital Phenotyping Research Unit. Department of Precision Health, Luxembourg Institute of Health, 1 A-B rue Thomas Edison, L-1445 Strassen, Luxembourg
- * E-mail:
| | - Lu Zhang
- Bioinformatics Platform, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Abir Elbéji
- Deep Digital Phenotyping Research Unit. Department of Precision Health, Luxembourg Institute of Health, 1 A-B rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Eduardo Higa
- Deep Digital Phenotyping Research Unit. Department of Precision Health, Luxembourg Institute of Health, 1 A-B rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Vladimir Despotovic
- Department of Computer Science, Faculty of Science, Technology and Medicine, University of Luxembourg, Avenue de la Fonte 6, L-4364 Esch-sur-Alzette, Luxembourg
| | - Markus Ollert
- Department of Infection and Immunity, Luxembourg Institute of Health, 29, Rue Henri Koch, L-4354 Esch-sur-Alzette, Luxembourg
- Department of Dermatology and Allergy Center, Odense Research Center for Anaphylaxis, University of Southern Denmark, 5000 Odense, Denmark
| | - Gloria A. Aguayo
- Deep Digital Phenotyping Research Unit. Department of Precision Health, Luxembourg Institute of Health, 1 A-B rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Petr V. Nazarov
- Bioinformatics Platform, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
- Multiomics Data Science, Luxembourg Institute of Health, 1A-B, rue Thomas Edison, L-1445 Strassen, Luxembourg
| | - Aurélie Fischer
- Deep Digital Phenotyping Research Unit. Department of Precision Health, Luxembourg Institute of Health, 1 A-B rue Thomas Edison, L-1445 Strassen, Luxembourg
| |
Collapse
|
35
|
Pah ND, Indrawati V, Kumar DK. Voice Features of Sustained Phoneme as COVID-19 Biomarker. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2022; 10:4901309. [PMID: 36304844 PMCID: PMC9592047 DOI: 10.1109/jtehm.2022.3208057] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Revised: 07/04/2022] [Accepted: 09/09/2022] [Indexed: 12/05/2022]
Abstract
BACKGROUND The COVID-19 pandemic has resulted in enormous costs to our society. Besides finding medicines to treat those infected by the virus, it is important to find effective and efficient strategies to prevent the spreading of the disease. One key factor to prevent transmission is to identify COVID-19 biomarkers that can be used to develop an efficient, accurate, noninvasive, and self-administered screening procedure. Several COVID-19 variants cause significant respiratory symptoms, and thus a voice signal may be a potential biomarker for COVID-19 infection. AIM This study investigated the effectiveness of different phonemes and a range of voice features in differentiating people infected by COVID-19 with respiratory tract symptoms. METHOD This cross-sectional, longitudinal study recorded six phonemes (i.e., /a/, /e/, /i/, /o/, /u/, and /m/) from 40 COVID-19 patients and 48 healthy subjects for 22 days. The signal features were obtained for the recordings, which were statistically analyzed and classified using Support Vector Machine (SVM). RESULTS The statistical analysis and SVM classification show that the voice features related to the vocal tract filtering (e.g., MFCC, VTL, and formants) and the stability of the respiratory muscles and lung volume (Intensity-SD) were the most sensitive to voice change due to COVID-19. The result also shows that the features extracted from the vowel /i/ during the first 3 days after admittance to the hospital were the most effective. The SVM classification accuracy with 18 ranked features extracted from /i/ was 93.5% (with F1 score of 94.3%). CONCLUSION A measurable difference exists between the voices of people with COVID-19 and healthy people, and the phoneme /i/ shows the most pronounced difference. This supports the potential for using computerized voice analysis to detect the disease and consider it a biomarker.
Collapse
Affiliation(s)
- Nemuel D. Pah
- Department of Electrical EngineeringUniversitas SurabayaSurabaya60293Indonesia
| | - Veronica Indrawati
- Department of Electrical EngineeringUniversitas SurabayaSurabaya60293Indonesia
| | - Dinesh K. Kumar
- School of EngineeringRMIT UniversityMelbourneVIC3000Australia
| |
Collapse
|
36
|
Ghrabli S, Elgendi M, Menon C. Challenges and Opportunities of Deep Learning for Cough-Based COVID-19 Diagnosis: A Scoping Review. Diagnostics (Basel) 2022; 12:2142. [PMID: 36140543 PMCID: PMC9498071 DOI: 10.3390/diagnostics12092142] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2022] [Revised: 08/26/2022] [Accepted: 08/31/2022] [Indexed: 11/16/2022] Open
Abstract
In the past two years, medical researchers and data scientists worldwide have focused their efforts on containing the pandemic of coronavirus disease 2019 (COVID-19). Deep learning models have been proven to be capable of efficient medical diagnosis and prognosis in cancer, common lung diseases, and COVID-19. On the other hand, artificial neural networks have demonstrated their potential in pattern recognition and classification in various domains, including healthcare. This literature review aims to report the state of research on developing neural network models to diagnose COVID-19 from cough sounds to create a cost-efficient and accessible testing tool in the fight against the pandemic. A total of 35 papers were included in this review following a screening of the 161 outputs of the literature search. We extracted information from articles on data resources, model structures, and evaluation metrics and then explored the scope of experimental studies and methodologies and analyzed their outcomes and limitations. We found that cough is a biomarker, and its associated information can determine an individual's health status. Convolutional neural networks were predominantly used, suggesting they are particularly suitable for feature extraction and classification. The reported accuracy values ranged from 73.1% to 98.5%. Moreover, the dataset sizes ranged from 16 to over 30,000 cough audio samples. Although deep learning is a promising prospect in identifying COVID-19, we identified a gap in the literature on research conducted over large and diversified data sets.
Collapse
Affiliation(s)
- Syrine Ghrabli
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
- Department of Physics, ETH Zurich, 8093 Zurich, Switzerland
| | - Mohamed Elgendi
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
| | - Carlo Menon
- Biomedical and Mobile Health Technology Lab, ETH Zurich, 8008 Zurich, Switzerland
| |
Collapse
|
37
|
Sunitha G, Arunachalam R, Abd‐Elnaby M, Eid MMA, Rashed ANZ. A comparative analysis of deep neural network architectures for the dynamic diagnosis of COVID-19 based on acoustic cough features. INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOLOGY 2022; 32:1433-1446. [PMID: 35941929 PMCID: PMC9348187 DOI: 10.1002/ima.22749] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 03/31/2022] [Accepted: 05/03/2022] [Indexed: 06/15/2023]
Abstract
The study aims to assess the detection performance of a rapid primary screening technique for COVID-19 that is purely based on the cough sound extracted from 2200 clinically validated samples using laboratory molecular testing (1100 COVID-19 negative and 1100 COVID-19 positive). Results and severity of samples based on quantitative RT-PCR (qRT-PCR), cycle threshold, and patient lymphocyte numbers were clinically labeled. Our suggested general methods consist of a tensor based on audio characteristics and deep-artificial neural network classification with deep cough convolutional layers, based on the dilated temporal convolution neural network (DTCN). DTCN has approximately 76% accuracy, 73.12% in TCN, and 72.11% in CNN-LSTM which have been trained at a learning rate of 0.2%, respectively. In our scenario, CNN-LSTM can no longer be employed for COVID-19 predictions, as they would generally offer questionable forecasts. In the previous stage, we discussed the exactness of the total cases of TCN, dilated TCN, and CNN-LSTM models which were truly predicted. Our proposed technique to identify COVID-19 can be considered as a robust and in-demand technique to rapidly detect the infection. We believe it can considerably hinder the COVID-19 pandemic worldwide.
Collapse
Affiliation(s)
- Gurram Sunitha
- Department of Computer Science EngineeringSree Vidyanikethan Engineering CollegeTirupatiAndhra PradeshIndia
| | - Rajesh Arunachalam
- Department of Electronics and Communication EngineeringCVR College of Engineering (Autonomous)HyderabadTelangana501510India
| | - Mohammed Abd‐Elnaby
- Department of Computer Engineering, College of Computers and Information TechnologyTaif UniversityP.O. Box 11099Taif 21944Saudi Arabia
| | - Mahmoud M. A. Eid
- Department of Electrical Engineering, College of EngineeringTaif UniversityP.O. Box 11099Taif 21944Saudi Arabia
| | - Ahmed Nabih Zaki Rashed
- Electronics and Electrical Communications Engineering DepartmentFaculty of Electronic Engineering, Menoufia UniversityMenoufEgypt
| |
Collapse
|
38
|
Kranthi Kumar L, Alphonse PJA. COVID-19: respiratory disease diagnosis with regularized deep convolutional neural network using human respiratory sounds. THE EUROPEAN PHYSICAL JOURNAL. SPECIAL TOPICS 2022; 231:3673-3696. [PMID: 35966369 PMCID: PMC9363874 DOI: 10.1140/epjs/s11734-022-00649-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Accepted: 07/26/2022] [Indexed: 06/15/2023]
Abstract
Human respiratory sound auscultation (HRSA) parameters have been the real choice for detecting human respiratory diseases in the last few years. It is a challenging task to extract the respiratory sound features from the breath, voice, and cough sounds. The existing methods failed to extract the sound features to diagnose respiratory diseases. We proposed and evaluated a new regularized deep convolutional neural network (RDCNN) architecture to accept COVID-19 sound data and essential sound features. The proposed architecture is trained with the COVID-19 sound data sets and gives a better learning curve than any other state-of-the-art model. We examine the performance of RDCNN with Max-Pooling (Model-1) and without Max-Pooling (Model-2) functions. In this work, we observed that RDCNN model performance with three sound feature extraction methods [Soft-Mel frequency channel, Log-Mel frequency spectrum, and Modified Mel-frequency Cepstral Coefficient (MMFCC) spectrum] for COVID-19 sound data sets (KDD-data, ComParE2021-CCS-CSS-Data, and NeurlPs2021-data). To amplify the models' performance, we applied the augmentation technique along with regularization. We have also carried out this work to estimate the mutation of SARS-CoV-2 in the five waves using prognostic models (fractal-based). The proposed model achieves state-of-the-art performance on the COVID-19 sound data set to identify COVID-19 disease symptoms. The model's learnable parameter gradients have vanished in the intermediate layers while optimizing the prediction error which is addressed with our proposed RDCNN model. Our experiments suggested that 3 × 3 kernel size for regularized deep CNN (without max-pooling) shows 2-3% better classification accuracy compared to RDCNN with max-pooling. The experimental results suggest that this new approach may achieve the finest results on respiratory diseases.
Collapse
Affiliation(s)
- Lella Kranthi Kumar
- Department of Computer Applications, NIT Tiruchirappalli, Tiruchirappalli, Tamil Nadu 620015 India
| | - P. J. A. Alphonse
- Department of Computer Applications, NIT Tiruchirappalli, Tiruchirappalli, Tamil Nadu 620015 India
| |
Collapse
|
39
|
Wang Z, Xiong H, Tang M, Boukhechba M, Flickinger TE, Barnes LE. Mobile Sensing in the COVID-19 Era: A Review. HEALTH DATA SCIENCE 2022; 2022:9830476. [PMID: 36408201 PMCID: PMC9629686 DOI: 10.34133/2022/9830476] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/12/2022] [Accepted: 07/18/2022] [Indexed: 12/03/2022]
Abstract
Background During the COVID-19 pandemic, mobile sensing and data analytics techniques have demonstrated their capabilities in monitoring the trajectories of the pandemic, by collecting behavioral, physiological, and mobility data on individual, neighborhood, city, and national scales. Notably, mobile sensing has become a promising way to detect individuals' infectious status, track the change in long-term health, trace the epidemics in communities, and monitor the evolution of viruses and subspecies. Methods We followed the PRISMA practice and reviewed 60 eligible papers on mobile sensing for monitoring COVID-19. We proposed a taxonomy system to summarize literature by the time duration and population scale under mobile sensing studies. Results We found that existing literature can be naturally grouped in four clusters, including remote detection, long-term tracking, contact tracing, and epidemiological study. We summarized each group and analyzed representative works with regard to the system design, health outcomes, and limitations on techniques and societal factors. We further discussed the implications and future directions of mobile sensing in communicable diseases from the perspectives of technology and applications. Conclusion Mobile sensing techniques are effective, efficient, and flexible to surveil COVID-19 in scales of time and populations. In the post-COVID era, technical and societal issues in mobile sensing are expected to be addressed to improve healthcare and social outcomes.
Collapse
Affiliation(s)
- Zhiyuan Wang
- School of Engineering and Applied Science, University of Virginia, Charlottesville, USA
| | - Haoyi Xiong
- Big Data Lab, Baidu Research, Baidu Inc., BeijingChina
| | - Mingyue Tang
- School of Engineering and Applied Science, University of Virginia, Charlottesville, USA
| | - Mehdi Boukhechba
- School of Engineering and Applied Science, University of Virginia, Charlottesville, USA
| | - Tabor E. Flickinger
- Department of Medicine, University of Virginia, Charlottesville, Virginia, USA
| | - Laura E. Barnes
- School of Engineering and Applied Science, University of Virginia, Charlottesville, USA
| |
Collapse
|
40
|
Wang G, Wang L, Meng Z, Su X, Jia C, Qiao X, Pan S, Chen Y, Cheng Y, Zhu M. Visual Detection of COVID-19 from Materials Aspect. ADVANCED FIBER MATERIALS 2022; 4:1304-1333. [PMID: 35966612 PMCID: PMC9358106 DOI: 10.1007/s42765-022-00179-y] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Accepted: 05/25/2022] [Indexed: 05/25/2023]
Abstract
ABSTRACT In the recent COVID-19 pandemic, World Health Organization emphasized that early detection is an effective strategy to reduce the spread of SARS-CoV-2 viruses. Several diagnostic methods, such as reverse transcription-polymerase chain reaction (RT-PCR) and lateral flow immunoassay (LFIA), have been applied based on the mechanism of specific recognition and binding of the probes to viruses or viral antigens. Although the remarkable progress, these methods still suffer from inadequate cellular materials or errors in the detection and sampling procedure of nasopharyngeal/oropharyngeal swab collection. Therefore, developing accurate, ultrafast, and visualized detection calls for more advanced materials and technology urgently to fight against the epidemic. In this review, we first summarize the current methodologies for SARS-CoV-2 diagnosis. Then, recent representative examples are introduced based on various output signals (e.g., colorimetric, fluorometric, electronic, acoustic). Finally, we discuss the limitations of the methods and provide our perspectives on priorities for future test development.
Collapse
Affiliation(s)
- Gang Wang
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Le Wang
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Zheyi Meng
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Xiaolong Su
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Chao Jia
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Xiaolan Qiao
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Shaowu Pan
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Yinjun Chen
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Yanhua Cheng
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| | - Meifang Zhu
- State Key Laboratory for Modification of Chemical Fibers and Polymer Materials, College of Materials Science and Engineering, Donghua University, Shanghai, 201620 China
| |
Collapse
|
41
|
Akman A, Coppock H, Gaskell A, Tzirakis P, Jones L, Schuller BW. Evaluating the COVID-19 Identification ResNet (CIdeR) on the INTERSPEECH COVID-19 From Audio Challenges. Front Digit Health 2022; 4:789980. [PMID: 35873349 PMCID: PMC9302571 DOI: 10.3389/fdgth.2022.789980] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Accepted: 05/31/2022] [Indexed: 11/20/2022] Open
Abstract
Several machine learning-based COVID-19 classifiers exploiting vocal biomarkers of COVID-19 has been proposed recently as digital mass testing methods. Although these classifiers have shown strong performances on the datasets on which they are trained, their methodological adaptation to new datasets with different modalities has not been explored. We report on cross-running the modified version of recent COVID-19 Identification ResNet (CIdeR) on the two Interspeech 2021 COVID-19 diagnosis from cough and speech audio challenges: ComParE and DiCOVA. CIdeR is an end-to-end deep learning neural network originally designed to classify whether an individual is COVID-19-positive or COVID-19-negative based on coughing and breathing audio recordings from a published crowdsourced dataset. In the current study, we demonstrate the potential of CIdeR at binary COVID-19 diagnosis from both the COVID-19 Cough and Speech Sub-Challenges of INTERSPEECH 2021, ComParE and DiCOVA. CIdeR achieves significant improvements over several baselines. We also present the results of the cross dataset experiments with CIdeR that show the limitations of using the current COVID-19 datasets jointly to build a collective COVID-19 classifier.
Collapse
Affiliation(s)
- Alican Akman
- GLAM–Group on Language, Audio, and Music, Imperial College London, London, United Kingdom
- *Correspondence: Alican Akman
| | - Harry Coppock
- GLAM–Group on Language, Audio, and Music, Imperial College London, London, United Kingdom
| | - Alexander Gaskell
- GLAM–Group on Language, Audio, and Music, Imperial College London, London, United Kingdom
| | - Panagiotis Tzirakis
- GLAM–Group on Language, Audio, and Music, Imperial College London, London, United Kingdom
| | - Lyn Jones
- Department of Radiology, North Bristol NHS Trust, Bristol, United Kingdom
| | - Björn W. Schuller
- GLAM–Group on Language, Audio, and Music, Imperial College London, London, United Kingdom
- Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany
| |
Collapse
|
42
|
Sharma G, Umapathy K, Krishnan S. Audio texture analysis of COVID-19 cough, breath, and speech sounds. Biomed Signal Process Control 2022; 76:103703. [PMID: 35464186 PMCID: PMC9013601 DOI: 10.1016/j.bspc.2022.103703] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Revised: 02/08/2022] [Accepted: 04/09/2022] [Indexed: 12/19/2022]
Abstract
The coronavirus disease (COVID-19) first appeared at the end of December 2019 and is still spreading in most countries. To diagnose COVID-19 using reverse transcription - Polymerase chain reaction (RT-PCR), one has to go to a dedicated center, which requires significant cost and human resources. Hence, there is a requirement for a remote monitoring tool that can perform the preliminary screening of COVID-19. In this paper, we propose that a detailed audio texture analysis of COVID-19 sounds may help in performing the initial screening of COVID-19. The texture analysis is done on three different signal modalities of COVID-19, i.e. cough, breath, and speech signals. In this work, we have used 1141 samples of cough signals, 392 samples of breath signals, and 893 samples of speech signals. To analyze the audio textural behavior of COVID-19 sounds, the local binary patterns LBP) and Haralick’s features were extracted from the spectrogram of the signals. The textural analysis on cough and breath sounds was done on the following 5 classes for the first time: COVID-19 positive with cough, COVID-19 positive without cough, healthy person with cough, healthy person without cough, and an asthmatic cough. For speech sounds there were only two classes: COVID-19 positive, and COVID-19 negative. During experiments, 71.7% of the cough samples and 72.2% of breath samples were classified into 5 classes. Also, 79.7% of speech samples are classified into 2 classes. The highest accuracy rate of 98.9% was obtained when binary classification between COVID-19 cough and non-COVID-19 cough was done.
Collapse
Affiliation(s)
- Garima Sharma
- Department of Electrical, Computer, & Biomedical Engineering, Ryerson University, Toronto, Canada
| | - Karthikeyan Umapathy
- Department of Electrical, Computer, & Biomedical Engineering, Ryerson University, Toronto, Canada
| | - Sri Krishnan
- Department of Electrical, Computer, & Biomedical Engineering, Ryerson University, Toronto, Canada
| |
Collapse
|
43
|
Sobahi N, Atila O, Deniz E, Sengur A, Acharya UR. Explainable COVID-19 detection using fractal dimension and vision transformer with Grad-CAM on cough sounds. Biocybern Biomed Eng 2022; 42:1066-1080. [PMID: 36092540 PMCID: PMC9444505 DOI: 10.1016/j.bbe.2022.08.005] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2022] [Revised: 08/28/2022] [Accepted: 08/29/2022] [Indexed: 11/27/2022]
Abstract
The polymerase chain reaction (PCR) test is not only time-intensive but also a contact method that puts healthcare personnel at risk. Thus, contactless and fast detection tests are more valuable. Cough sound is an important indicator of COVID-19, and in this paper, a novel explainable scheme is developed for cough sound-based COVID-19 detection. In the presented work, the cough sound is initially segmented into overlapping parts, and each segment is labeled as the input audio, which may contain other sounds. The deep Yet Another Mobile Network (YAMNet) model is considered in this work. After labeling, the segments labeled as cough are cropped and concatenated to reconstruct the pure cough sounds. Then, four fractal dimensions (FD) calculation methods are employed to acquire the FD coefficients on the cough sound with an overlapped sliding window that forms a matrix. The constructed matrixes are then used to form the fractal dimension images. Finally, a pretrained vision transformer (ViT) model is used to classify the constructed images into COVID-19, healthy and symptomatic classes. In this work, we demonstrate the performance of the ViT on cough sound-based COVID-19, and a visual explainability of the inner workings of the ViT model is shown. Three publically available cough sound datasets, namely COUGHVID, VIRUFY, and COSWARA, are used in this study. We have obtained 98.45%, 98.15%, and 97.59% accuracy for COUGHVID, VIRUFY, and COSWARA datasets, respectively. Our developed model obtained the highest performance compared to the state-of-the-art methods and is ready to be tested in real-world applications.
Collapse
Affiliation(s)
- Nebras Sobahi
- King Abdulaziz University, Department of Electrical and Computer Engineering, Jeddah, Saudi Arabia
| | - Orhan Atila
- Firat University, Technology Faculty, Electrical and Electronics Engineering Department, Elazig, Turkey
| | - Erkan Deniz
- Firat University, Technology Faculty, Electrical and Electronics Engineering Department, Elazig, Turkey
| | - Abdulkadir Sengur
- Firat University, Technology Faculty, Electrical and Electronics Engineering Department, Elazig, Turkey
| | - U Rajendra Acharya
- Ngee Ann Polytechnic, Department of Electronics and Computer Engineering, 599489, Singapore.,Biomedical Engineering, School of Science and Technology, SUSS University, Singapore.,Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan
| |
Collapse
|
44
|
Ren Z, Chang Y, Bartl-Pokorny KD, Pokorny FB, Schuller BW. The Acoustic Dissection of Cough: Diving Into Machine Listening-based COVID-19 Analysis and Detection. J Voice 2022:S0892-1997(22)00166-7. [PMID: 35835648 PMCID: PMC9197794 DOI: 10.1016/j.jvoice.2022.06.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Revised: 05/25/2022] [Accepted: 06/09/2022] [Indexed: 12/05/2022]
Abstract
OBJECTIVES The coronavirus disease 2019 (COVID-19) has caused a crisis worldwide. Amounts of efforts have been made to prevent and control COVID-19's transmission, from early screenings to vaccinations and treatments. Recently, due to the spring up of many automatic disease recognition applications based on machine listening techniques, it would be fast and cheap to detect COVID-19 from recordings of cough, a key symptom of COVID-19. To date, knowledge of the acoustic characteristics of COVID-19 cough sounds is limited but would be essential for structuring effective and robust machine learning models. The present study aims to explore acoustic features for distinguishing COVID-19 positive individuals from COVID-19 negative ones based on their cough sounds. METHODS By applying conventional inferential statistics, we analyze the acoustic correlates of COVID-19 cough sounds based on the ComParE feature set, i.e., a standardized set of 6,373 acoustic higher-level features. Furthermore, we train automatic COVID-19 detection models with machine learning methods and explore the latent features by evaluating the contribution of all features to the COVID-19 status predictions. RESULTS The experimental results demonstrate that a set of acoustic parameters of cough sounds, e.g., statistical functionals of the root mean square energy and Mel-frequency cepstral coefficients, bear essential acoustic information in terms of effect sizes for the differentiation between COVID-19 positive and COVID-19 negative cough samples. Our general automatic COVID-19 detection model performs significantly above chance level, i.e., at an unweighted average recall (UAR) of 0.632, on a data set consisting of 1,411 cough samples (COVID-19 positive/negative: 210/1,201). CONCLUSIONS Based on the acoustic correlates analysis on the ComParE feature set and the feature analysis in the effective COVID-19 detection approach, we find that several acoustic features that show higher effects in conventional group difference testing are also higher weighted in the machine learning models.
Collapse
Affiliation(s)
- Zhao Ren
- EIHW - Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany; L3S Research Center, Hannover, Germany.
| | - Yi Chang
- GLAM - Group on Language, Audio, & Music, Imperial College London, London, United Kingdom
| | - Katrin D Bartl-Pokorny
- EIHW - Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany; Division of Phoniatrics, Medical University of Graz, Graz, Austria; Division of Physiology, Medical University of Graz, Graz, Austria.
| | - Florian B Pokorny
- EIHW - Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany; Division of Phoniatrics, Medical University of Graz, Graz, Austria; Division of Physiology, Medical University of Graz, Graz, Austria
| | - Björn W Schuller
- EIHW - Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany; GLAM - Group on Language, Audio, & Music, Imperial College London, London, United Kingdom
| |
Collapse
|
45
|
Villa-Parra AC, Criollo I, Valadão C, Silva L, Coelho Y, Lampier L, Rangel L, Sharma G, Delisle-Rodríguez D, Calle-Siguencia J, Urgiles-Ortiz F, Díaz C, Caldeira E, Krishnan S, Bastos-Filho T. Towards Multimodal Equipment to Help in the Diagnosis of COVID-19 Using Machine Learning Algorithms. SENSORS 2022; 22:s22124341. [PMID: 35746121 PMCID: PMC9228002 DOI: 10.3390/s22124341] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 06/01/2022] [Accepted: 06/02/2022] [Indexed: 12/29/2022]
Abstract
COVID-19 occurs due to infection through respiratory droplets containing the SARS-CoV-2 virus, which are released when someone sneezes, coughs, or talks. The gold-standard exam to detect the virus is Real-Time Polymerase Chain Reaction (RT-PCR); however, this is an expensive test and may require up to 3 days after infection for a reliable result, and if there is high demand, the labs could be overwhelmed, which can cause significant delays in providing results. Biomedical data (oxygen saturation level—SpO2, body temperature, heart rate, and cough) are acquired from individuals and are used to help infer infection by COVID-19, using machine learning algorithms. The goal of this study is to introduce the Integrated Portable Medical Assistant (IPMA), which is a multimodal piece of equipment that can collect biomedical data, such as oxygen saturation level, body temperature, heart rate, and cough sound, and helps infer the diagnosis of COVID-19 through machine learning algorithms. The IPMA has the capacity to store the biomedical data for continuous studies and can be used to infer other respiratory diseases. Quadratic kernel-free non-linear Support Vector Machine (QSVM) and Decision Tree (DT) were applied on three datasets with data of cough, speech, body temperature, heart rate, and SpO2, obtaining an Accuracy rate (ACC) and Area Under the Curve (AUC) of approximately up to 88.0% and 0.85, respectively, as well as an ACC up to 99% and AUC = 0.94, respectively, for COVID-19 infection inference. When applied to the data acquired with the IMPA, these algorithms achieved 100% accuracy. Regarding the easiness of using the equipment, 36 volunteers reported that the IPMA has a high usability, according to results from two metrics used for evaluation: System Usability Scale (SUS) and Post Study System Usability Questionnaire (PSSUQ), with scores of 85.5 and 1.41, respectively. In light of the worldwide needs for smart equipment to help fight the COVID-19 pandemic, this new equipment may help with the screening of COVID-19 through data collected from biomedical signals and cough sounds, as well as the use of machine learning algorithms.
Collapse
Affiliation(s)
- Ana Cecilia Villa-Parra
- Biomedical Engineering Research Group—GIIB, Universidad Politécnica Salesiana (UPS), Cuenca 010105, Ecuador; (A.C.V.-P.); (I.C.); (J.C.-S.); (F.U.-O.)
| | - Ismael Criollo
- Biomedical Engineering Research Group—GIIB, Universidad Politécnica Salesiana (UPS), Cuenca 010105, Ecuador; (A.C.V.-P.); (I.C.); (J.C.-S.); (F.U.-O.)
| | - Carlos Valadão
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Leticia Silva
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Yves Coelho
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Lucas Lampier
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Luara Rangel
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Garima Sharma
- Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON M5B 2K3, Canada; (G.S.); (S.K.)
| | - Denis Delisle-Rodríguez
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - John Calle-Siguencia
- Biomedical Engineering Research Group—GIIB, Universidad Politécnica Salesiana (UPS), Cuenca 010105, Ecuador; (A.C.V.-P.); (I.C.); (J.C.-S.); (F.U.-O.)
| | - Fernando Urgiles-Ortiz
- Biomedical Engineering Research Group—GIIB, Universidad Politécnica Salesiana (UPS), Cuenca 010105, Ecuador; (A.C.V.-P.); (I.C.); (J.C.-S.); (F.U.-O.)
| | - Camilo Díaz
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Eliete Caldeira
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
| | - Sridhar Krishnan
- Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON M5B 2K3, Canada; (G.S.); (S.K.)
| | - Teodiano Bastos-Filho
- Department of Electrical Engineering, Universidade Federal do Espírito Santo (UFES), Vitoria 29075-910, Brazil; (C.V.); (L.S.); (Y.C.); (L.L.); (L.R.); (D.D.-R.); (C.D.); (E.C.)
- Correspondence: ; Tel.: +593-98-441-2586
| |
Collapse
|
46
|
Chowdhury NK, Kabir MA, Rahman MM, Islam SMS. Machine learning for detecting COVID-19 from cough sounds: An ensemble-based MCDM method. Comput Biol Med 2022; 145:105405. [PMID: 35318171 PMCID: PMC8926945 DOI: 10.1016/j.compbiomed.2022.105405] [Citation(s) in RCA: 25] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2021] [Revised: 03/10/2022] [Accepted: 03/11/2022] [Indexed: 12/16/2022]
Abstract
This research aims to analyze the performance of state-of-the-art machine learning techniques for classifying COVID-19 from cough sounds and to identify the model(s) that consistently perform well across different cough datasets. Different performance evaluation metrics (precision, sensitivity, specificity, AUC, accuracy, etc.) make selecting the best performance model difficult. To address this issue, in this paper, we propose an ensemble-based multi-criteria decision making (MCDM) method for selecting top performance machine learning technique(s) for COVID-19 cough classification. We use four cough datasets, namely Cambridge, Coswara, Virufy, and NoCoCoDa to verify the proposed method. At first, our proposed method uses the audio features of cough samples and then applies machine learning (ML) techniques to classify them as COVID-19 or non-COVID-19. Then, we consider a multi-criteria decision-making (MCDM) method that combines ensemble technologies (i.e., soft and hard) to select the best model. In MCDM, we use the technique for order preference by similarity to ideal solution (TOPSIS) for ranking purposes, while entropy is applied to calculate evaluation criteria weights. In addition, we apply the feature reduction process through recursive feature elimination with cross-validation under different estimators. The results of our empirical evaluations show that the proposed method outperforms the state-of-the-art models. We see that when the proposed method is used for analysis using the Extra-Trees classifier, it has achieved promising results (AUC: 0.95, Precision: 1, Recall: 0.97).
Collapse
Affiliation(s)
- Nihad Karim Chowdhury
- Department of Computer Science and Engineering, University of Chittagong, Bangladesh,Corresponding author
| | - Muhammad Ashad Kabir
- Data Science Research Unit, School of Computing, Mathematics and Engineering, Charles Sturt University, NSW, Australia
| | - Md. Muhtadir Rahman
- Department of Computer Science and Engineering, University of Chittagong, Bangladesh
| | | |
Collapse
|
47
|
Pay attention to the speech: COVID-19 diagnosis using machine learning and crowdsourced respiratory and speech recordings. ALEXANDRIA ENGINEERING JOURNAL 2022; 61:3487-3500. [PMCID: PMC8397542 DOI: 10.1016/j.aej.2021.08.070] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Revised: 08/07/2021] [Accepted: 08/23/2021] [Indexed: 06/02/2023]
Abstract
Since the outbreak of COVID-19, many efforts have been made to utilize the respiratory sounds and coughs collected by smartphones for training Machine Learning models to classify and distinguish COVID-19 sounds from healthy ones. Embedding those models into mobile applications or Internet of things devices can make effective COVID-19 pre-screening tools afforded by anyone anywhere. Most of the previous researchers trained their classifiers with respiratory sounds such as breathing or coughs, and they achieved promising results. We claim that using special voice patterns besides other respiratory sounds can achieve better performance. In this study, we used the Coswara dataset where each user has recorded 9 different types of sounds as cough, breathing, and speech labeled with COVID-19 status. A combination of models trained on different sounds can diagnose COVID-19 more accurately than a single model trained on cough or breathing only. Our results show that using simple binary classifiers can achieve an AUC of 96.4% and an accuracy of 96% by averaging the predictions of multiple models trained and evaluated separately on different sound types. Finally, this study aims to draw attention to the importance of the human voice alongside other respiratory sounds for the sound-based COVID-19 diagnosis.
Collapse
|
48
|
Ismail Z, W Idris WF, Abdullah AH. Graphene-based temperature, humidity, and strain sensor: A review on progress, characterization, and potential applications during Covid-19 pandemic. SENSORS INTERNATIONAL 2022; 3:100183. [PMID: 35633818 PMCID: PMC9126002 DOI: 10.1016/j.sintl.2022.100183] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 05/19/2022] [Accepted: 05/19/2022] [Indexed: 11/24/2022] Open
Abstract
Graphene's potential as material for wearable, highly sensitive and robust sensor in various fields of technology has been widely investigated until now in order to capitalize on its unique intrinsic physical and chemical properties. In the wake of Covid-19 pandemic, it has been noticed that there are various potentials roles that can be fulfilled by graphene-based temperature, humidity and strain sensor, whose roles has not been widely explored to date. This paper takes the liberty to mainly highlight the progress layout and characterization technique for graphene-based sensor while including a brief discussion on the possible strategy of sensing data analysis that can be employed to minimize and prevent the risk of Covid-19 infection within a living community. While majority of the reported sensor is still in the in-progress status, its highlighted role in this work may provide a brief idea on how the ongoing research in graphene-based sensor may lead to the future implementation of the device for routine healthcare check-up and diagnostic point-care during and post-pandemic era. On the other hand, the sensitivity and response time data against working temperature, humidity and strain range that are provided could serve as a reference for benchmarking purpose, which certainly would help enthusiast in the development of a graphene-based sensor with a better performance for the future.
Collapse
|
49
|
Hamdi S, Oussalah M, Moussaoui A, Saidi M. Attention-based hybrid CNN-LSTM and spectral data augmentation for COVID-19 diagnosis from cough sound. J Intell Inf Syst 2022; 59:367-389. [PMID: 35498369 PMCID: PMC9034264 DOI: 10.1007/s10844-022-00707-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Revised: 03/23/2022] [Accepted: 03/27/2022] [Indexed: 12/11/2022]
Abstract
COVID-19 pandemic has fueled the interest in artificial intelligence tools for quick diagnosis to limit virus spreading. Over 60% of people who are infected complain of a dry cough. Cough and other respiratory sounds were used to build diagnosis models in much recent research. We propose in this work, an augmentation pipeline which is applied on the pre-filtered data and uses i) pitch-shifting technique to augment the raw signal and, ii) spectral data augmentation technique SpecAugment to augment the computed mel-spectrograms. A deep learning based architecture that hybridizes convolution neural networks and long-short term memory with an attention mechanism is proposed for building the classification model. The feasibility of the proposed is demonstrated through a set of testing scenarios using the large-scale COUGHVID cough dataset and through a comparison with three baselines models. We have shown that our classification model achieved 91.13% of testing accuracy, 90.93% of sensitivity and an area under the curve of receiver operating characteristic of 91.13%.
Collapse
Affiliation(s)
- Skander Hamdi
- Department of Computer Science, University of Ferhat Abbes Setif, 19000 Setif, Algeria
| | - Mourad Oussalah
- Department of Computer Science and Engineering, University of Oulu, 90570 Oulu, Finland
| | - Abdelouahab Moussaoui
- Department of Computer Science, University of Ferhat Abbes Setif, 19000 Setif, Algeria
| | - Mohamed Saidi
- Department of Computer Science, University of Ferhat Abbes Setif, 19000 Setif, Algeria
| |
Collapse
|
50
|
Serrurier A, Neuschaefer-Rube C, Röhrig R. Past and Trends in Cough Sound Acquisition, Automatic Detection and Automatic Classification: A Comparative Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:2896. [PMID: 35458885 PMCID: PMC9027375 DOI: 10.3390/s22082896] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 04/07/2022] [Accepted: 04/08/2022] [Indexed: 11/16/2022]
Abstract
Cough is a very common symptom and the most frequent reason for seeking medical advice. Optimized care goes inevitably through an adapted recording of this symptom and automatic processing. This study provides an updated exhaustive quantitative review of the field of cough sound acquisition, automatic detection in longer audio sequences and automatic classification of the nature or disease. Related studies were analyzed and metrics extracted and processed to create a quantitative characterization of the state-of-the-art and trends. A list of objective criteria was established to select a subset of the most complete detection studies in the perspective of deployment in clinical practice. One hundred and forty-four studies were short-listed, and a picture of the state-of-the-art technology is drawn. The trend shows an increasing number of classification studies, an increase of the dataset size, in part from crowdsourcing, a rapid increase of COVID-19 studies, the prevalence of smartphones and wearable sensors for the acquisition, and a rapid expansion of deep learning. Finally, a subset of 12 detection studies is identified as the most complete ones. An unequaled quantitative overview is presented. The field shows a remarkable dynamic, boosted by the research on COVID-19 diagnosis, and a perfect adaptation to mobile health.
Collapse
Affiliation(s)
- Antoine Serrurier
- Institute of Medical Informatics, University Hospital of the RWTH Aachen, 52057 Aachen, Germany;
- Clinic for Phoniatrics, Pedaudiology & Communication Disorders, University Hospital of the RWTH Aachen, 52057 Aachen, Germany;
| | - Christiane Neuschaefer-Rube
- Clinic for Phoniatrics, Pedaudiology & Communication Disorders, University Hospital of the RWTH Aachen, 52057 Aachen, Germany;
| | - Rainer Röhrig
- Institute of Medical Informatics, University Hospital of the RWTH Aachen, 52057 Aachen, Germany;
| |
Collapse
|