1
|
Majzoobi F, Khodabakhshi MB, Jamasb S, Goudarzi S. ConvLSNet: A lightweight architecture based on ConvLSTM model for the classification of pulmonary conditions using multichannel lung sound recordings. Artif Intell Med 2024; 154:102922. [PMID: 38924864 DOI: 10.1016/j.artmed.2024.102922] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2023] [Revised: 06/19/2024] [Accepted: 06/21/2024] [Indexed: 06/28/2024]
Abstract
Characterization of lung sounds (LS) is indispensable for diagnosing respiratory pathology. Although conventional neural networks (NNs) have been widely employed for the automatic diagnosis of lung sounds, deep neural networks can potentially be more useful than conventional NNs by allowing accurate classification without requiring preprocessing and feature extraction. Utilizing the long short-term memory (LSTM) layers to reveal the sequence-based properties of the LS time series, a novel architecture consisting of a cascade of convolutional long short-term memory (ConvLSTM) and LSTM layers, namely ConvLSNet is developed, which permits highly accurate diagnosis of pulmonary disease states. By modeling the multichannel lung sounds through the ConvLSTM layer, the proposed ConvLSNet architecture can concurrently deal with the spatial and temporal properties of the six-channel LS recordings without heavy preprocessing or data transformation. Notably, the proposed model achieves a classification accuracy of 97.4 % based on LS data corresponding to three pulmonary conditions, namely asthma, COPD, and the healthy state. Compared with architectures consisting exclusively of CNN or LSTM layers, as well as those employing a cascade integration of 2DCNN and LSTM layers, the proposed ConvLSNet architecture exhibited the highest classification accuracy, while imposing the lowest computational cost as quantified by the number of parameters, training time, and learning rate.
Collapse
Affiliation(s)
- Faezeh Majzoobi
- Biomedical Engineering Department, Hamedan University of Technology, Hamedan, Iran
| | | | - Shahriar Jamasb
- Biomedical Engineering Department, Hamedan University of Technology, Hamedan, Iran
| | - Sobhan Goudarzi
- Physical Science Platform, Sunnybrook Research Institute, Toronto, ON M4N 3M5, Canada
| |
Collapse
|
2
|
Sim JH, Yoo J, Lee ML, Han SH, Han SK, Lee JY, Yi SW, Nam J, Kim DS, Yang YS. Deep Learning Model for Cosmetic Gel Classification Based on a Short-Time Fourier Transform and Spectrogram. ACS APPLIED MATERIALS & INTERFACES 2024; 16:25825-25835. [PMID: 38738662 DOI: 10.1021/acsami.4c03675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2024]
Abstract
Cosmetics and topical medications, such as gels, foams, creams, and lotions, are viscoelastic substances that are applied to the skin or mucous membranes. The human perception of these materials is complex and involves multiple sensory modalities. Traditional panel-based sensory evaluations have limitations due to individual differences in sensory receptors and factors such as age, race, and gender. Therefore, this study proposes a deep-learning-based method for systematically analyzing and effectively identifying the physical properties of cosmetic gels. Time-series friction signals generated by rubbing the gels were measured. These signals were preprocessed through short-time Fourier transform (STFT) and continuous wavelet transform (CWT), respectively, and the frequency factors that change over time were distinguished and analyzed. The deep learning model employed a ResNet-based convolution neural network (CNN) structure with optimization achieved through a learning rate scheduler. The optimized STFT-based 2D CNN model outperforms the CWT-based 2D and 1D CNN models. The optimized STFT-based 2D CNN model also demonstrated robustness and reliability through k-fold cross-validation. This study suggests the potential for an innovative approach to replace traditional expert panel evaluations and objectively assess the user experience of cosmetics.
Collapse
Affiliation(s)
- Jae Ho Sim
- Materials and Components Research Division, Superintelligence Creative research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Republic of Korea
- Department of Creative Convergence Engineering, Hanbat National University, Daejeon 34158, Republic of Korea
| | - Jengsu Yoo
- Materials and Components Research Division, Superintelligence Creative research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Republic of Korea
| | - Myung Lae Lee
- Materials and Components Research Division, Superintelligence Creative research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Republic of Korea
| | | | | | - Jeong Yu Lee
- Basic Research & Innovation Division, R&I Center, AmorePacific Corporation, Yongin-si, Gyeonggi-do 17074, Republic of Korea
| | - Sung Won Yi
- Basic Research & Innovation Division, R&I Center, AmorePacific Corporation, Yongin-si, Gyeonggi-do 17074, Republic of Korea
| | - Jin Nam
- Basic Research & Innovation Division, R&I Center, AmorePacific Corporation, Yongin-si, Gyeonggi-do 17074, Republic of Korea
| | - Dong Soo Kim
- Department of Creative Convergence Engineering, Hanbat National University, Daejeon 34158, Republic of Korea
| | - Yong Suk Yang
- Materials and Components Research Division, Superintelligence Creative research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Republic of Korea
| |
Collapse
|
3
|
Crisdayanti IAPA, Nam SW, Jung SK, Kim SE. Attention Feature Fusion Network via Knowledge Propagation for Automated Respiratory Sound Classification. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2024; 5:383-392. [PMID: 38899013 PMCID: PMC11186653 DOI: 10.1109/ojemb.2024.3402139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2024] [Revised: 04/24/2024] [Accepted: 05/13/2024] [Indexed: 06/21/2024] Open
Abstract
Goal: In light of the COVID-19 pandemic, the early diagnosis of respiratory diseases has become increasingly crucial. Traditional diagnostic methods such as computed tomography (CT) and magnetic resonance imaging (MRI), while accurate, often face accessibility challenges. Lung auscultation, a simpler alternative, is subjective and highly dependent on the clinician's expertise. The pandemic has further exacerbated these challenges by restricting face-to-face consultations. This study aims to overcome these limitations by developing an automated respiratory sound classification system using deep learning, facilitating remote and accurate diagnoses. Methods: We developed a deep convolutional neural network (CNN) model that utilizes spectrographic representations of respiratory sounds within an image classification framework. Our model is enhanced with attention feature fusion of low-to-high-level information based on a knowledge propagation mechanism to increase classification effectiveness. This novel approach was evaluated using the ICBHI benchmark dataset and a larger, self-collected Pediatric dataset comprising outpatient children aged 1 to 6 years. Results: The proposed CNN model with knowledge propagation demonstrated superior performance compared to existing state-of-the-art models. Specifically, our model showed higher sensitivity in detecting abnormalities in the Pediatric dataset, indicating its potential for improving the accuracy of respiratory disease diagnosis. Conclusions: The integration of a knowledge propagation mechanism into a CNN model marks a significant advancement in the field of automated diagnosis of respiratory disease. This study paves the way for more accessible and precise healthcare solutions, which is especially crucial in pandemic scenarios.
Collapse
Affiliation(s)
- Ida A. P. A. Crisdayanti
- Department of Applied Artificial IntelligenceSeoul National University of Science and TechnologySeoul01811South Korea
| | - Sung Woo Nam
- Woorisoa Children's HospitalSeoul08291South Korea
| | | | - Seong-Eun Kim
- Department of Applied Artificial IntelligenceSeoul National University of Science and TechnologySeoul01811South Korea
| |
Collapse
|
4
|
Lei D, Dong C, Guo H, Ma P, Liu H, Bao N, Kang H, Chen X, Wu Y. A fused multi-subfrequency bands and CBAM SSVEP-BCI classification method based on convolutional neural network. Sci Rep 2024; 14:8616. [PMID: 38616204 PMCID: PMC11016546 DOI: 10.1038/s41598-024-59348-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 04/09/2024] [Indexed: 04/16/2024] Open
Abstract
For the brain-computer interface (BCI) system based on steady-state visual evoked potential (SSVEP), it is difficult to obtain satisfactory classification performance for short-time window SSVEP signals by traditional methods. In this paper, a fused multi-subfrequency bands and convolutional block attention module (CBAM) classification method based on convolutional neural network (CBAM-CNN) is proposed for discerning SSVEP-BCI tasks. This method extracts multi-subfrequency bands SSVEP signals as the initial input of the network model, and then carries out feature fusion on all feature inputs. In addition, CBAM is embedded in both parts of the initial input and feature fusion for adaptive feature refinement. To verify the effectiveness of the proposed method, this study uses the datasets of Inner Mongolia University of Technology (IMUT) and Tsinghua University (THU) to evaluate the performance of the proposed method. The experimental results show that the highest accuracy of CBAM-CNN reaches 0.9813 percentage point (pp). Within 0.1-2 s time window, the accuracy of CBAM-CNN is 0.0201-0.5388 (pp) higher than that of CNN, CCA-CWT-SVM, CCA-SVM, CCA-GNB, FBCCA, and CCA. Especially in the short-time window range of 0.1-1 s, the performance advantage of CBAM-CNN is more significant. The maximum information transmission rate (ITR) of CBAM-CNN is 503.87 bit/min, which is 227.53 bit/min-503.41 bit/min higher than the above six EEG decoding methods. The study further results show that CBAM-CNN has potential application value in SSVEP decoding.
Collapse
Affiliation(s)
- Dongyang Lei
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China
| | - Chaoyi Dong
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China.
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China.
- Engineering Research Center of Large Energy Storage Technology, Ministry of Education, Hohhot, 010080, China.
- Inner Mongolia Academy of Science and Technology, Hohhot, 010010, China.
| | - Hongfei Guo
- Inner Mongolia Academy of Science and Technology, Hohhot, 010010, China.
| | - Pengfei Ma
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China
| | - Huanzi Liu
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China
| | - Naqin Bao
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China
| | - Hongzhuo Kang
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China
| | - Xiaoyan Chen
- College of Electric Power, Inner Mongolia University of Technology, Hohhot, 010080, China
- Intelligent Energy Technology and Equipment Engineering Research Centre of Colleges and Universities in Inner Mongolia Autonomous Region, Hohhot, 010051, China
- Engineering Research Center of Large Energy Storage Technology, Ministry of Education, Hohhot, 010080, China
- Inner Mongolia Academy of Science and Technology, Hohhot, 010010, China
| | - Yi Wu
- Inner Mongolia Academy of Science and Technology, Hohhot, 010010, China
| |
Collapse
|
5
|
Yook S, Kim D, Gupte C, Joo EY, Kim H. Deep learning of sleep apnea-hypopnea events for accurate classification of obstructive sleep apnea and determination of clinical severity. Sleep Med 2024; 114:211-219. [PMID: 38232604 PMCID: PMC10872216 DOI: 10.1016/j.sleep.2024.01.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 12/28/2023] [Accepted: 01/10/2024] [Indexed: 01/19/2024]
Abstract
BACKGROUND /Objective: Automatic apnea/hypopnea events classification, crucial for clinical applications, often faces challenges, particularly in hypopnea detection. This study aimed to evaluate the efficiency of a combined approach using nasal respiration flow (RF), peripheral oxygen saturation (SpO2), and ECG signals during polysomnography (PSG) for improved sleep apnea/hypopnea detection and obstructive sleep apnea (OSA) severity screening. METHODS An Xception network was trained using main features from RF, SpO2, and ECG signals obtained during PSG. In addition, we incorporated demographic data for enhanced performance. The detection of apnea/hypopnea events was based on RF and SpO2 feature sets, while the screening and severity categorization of OSA utilized predicted apnea/hypopnea events in conjunction with demographic data. RESULTS Using RF and SpO2 feature sets, our model achieved an accuracy of 94 % in detecting apnea/hypopnea events. For OSA screening, an exceptional accuracy of 99 % and an AUC of 0.99 were achieved. OSA severity categorization yielded an accuracy of 93 % and an AUC of 0.91, with no misclassification between normal and mild OSA versus moderate and severe OSA. However, classification errors predominantly arose in cases with hypopnea-prevalent participants. CONCLUSIONS The proposed method offers a robust automatic detection system for apnea/hypopnea events, requiring fewer sensors than traditional PSG, and demonstrates exceptional performance. Additionally, the classification algorithms for OSA screening and severity categorization exhibit significant discriminatory capacity.
Collapse
Affiliation(s)
- Soonhyun Yook
- USC Stevens Neuroimaging and Informatics Institute, Keck School of Medicine of USC, University of Southern California, Los Angeles, CA, 90033, USA
| | - Dongyeop Kim
- Department of Neurology, Seoul Hospital, College of Medicine, Ewha Womans University, Seoul, 07804, South Korea
| | - Chaitanya Gupte
- USC Stevens Neuroimaging and Informatics Institute, Keck School of Medicine of USC, University of Southern California, Los Angeles, CA, 90033, USA
| | - Eun Yeon Joo
- Department of Neurology, Neuroscience Center, Samsung Medical Center, Samsung Biomedical Research Institute, School of Medicine, Sungkyunkwan University, Seoul, 06351, South Korea.
| | - Hosung Kim
- USC Stevens Neuroimaging and Informatics Institute, Keck School of Medicine of USC, University of Southern California, Los Angeles, CA, 90033, USA.
| |
Collapse
|
6
|
Mang LD, González Martínez FD, Martinez Muñoz D, García Galán S, Cortina R. Classification of Adventitious Sounds Combining Cochleogram and Vision Transformers. SENSORS (BASEL, SWITZERLAND) 2024; 24:682. [PMID: 38276373 PMCID: PMC10818433 DOI: 10.3390/s24020682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 01/13/2024] [Accepted: 01/19/2024] [Indexed: 01/27/2024]
Abstract
Early identification of respiratory irregularities is critical for improving lung health and reducing global mortality rates. The analysis of respiratory sounds plays a significant role in characterizing the respiratory system's condition and identifying abnormalities. The main contribution of this study is to investigate the performance when the input data, represented by cochleogram, is used to feed the Vision Transformer (ViT) architecture, since this input-classifier combination is the first time it has been applied to adventitious sound classification to our knowledge. Although ViT has shown promising results in audio classification tasks by applying self-attention to spectrogram patches, we extend this approach by applying the cochleogram, which captures specific spectro-temporal features of adventitious sounds. The proposed methodology is evaluated on the ICBHI dataset. We compare the classification performance of ViT with other state-of-the-art CNN approaches using spectrogram, Mel frequency cepstral coefficients, constant-Q transform, and cochleogram as input data. Our results confirm the superior classification performance combining cochleogram and ViT, highlighting the potential of ViT for reliable respiratory sound classification. This study contributes to the ongoing efforts in developing automatic intelligent techniques with the aim to significantly augment the speed and effectiveness of respiratory disease detection, thereby addressing a critical need in the medical field.
Collapse
Affiliation(s)
- Loredana Daria Mang
- Department of Telecommunication Engineering, University of Jaen, 23700 Linares, Spain; (F.D.G.M.); (D.M.M.); (S.G.G.)
| | | | - Damian Martinez Muñoz
- Department of Telecommunication Engineering, University of Jaen, 23700 Linares, Spain; (F.D.G.M.); (D.M.M.); (S.G.G.)
| | - Sebastián García Galán
- Department of Telecommunication Engineering, University of Jaen, 23700 Linares, Spain; (F.D.G.M.); (D.M.M.); (S.G.G.)
| | - Raquel Cortina
- Department of Computer Science, University of Oviedo, 33003 Oviedo, Spain;
| |
Collapse
|
7
|
Sharma M, Nduba V, Njagi LN, Murithi W, Mwongera Z, Hawn TR, Patel SN, Horne DJ. TBscreen: A passive cough classifier for tuberculosis screening with a controlled dataset. SCIENCE ADVANCES 2024; 10:eadi0282. [PMID: 38170773 PMCID: PMC10776005 DOI: 10.1126/sciadv.adi0282] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 12/01/2023] [Indexed: 01/05/2024]
Abstract
Recent respiratory disease screening studies suggest promising performance of cough classifiers, but potential biases in model training and dataset quality preclude robust conclusions. To examine tuberculosis (TB) cough diagnostic features, we enrolled subjects with pulmonary TB (N = 149) and controls with other respiratory illnesses (N = 46) in Nairobi. We collected a dataset with 33,000 passive coughs and 1600 forced coughs in a controlled setting with similar demographics. We trained a ResNet18-based cough classifier using images of passive cough scalogram as input and obtained a fivefold cross-validation sensitivity of 0.70 (±0.11 SD). The smartphone-based model had better performance in subjects with higher bacterial load {receiver operating characteristic-area under the curve (ROC-AUC): 0.87 [95% confidence interval (CI): 0.87 to 0.88], P < 0.001} or lung cavities [ROC-AUC: 0.89 (95% CI: 0.88 to 0.89), P < 0.001]. Overall, our data suggest that passive cough features distinguish TB from non-TB subjects and are associated with bacterial burden and disease severity.
Collapse
Affiliation(s)
- Manuja Sharma
- Department of Electrical and Computer Engineering, University of Washington, 185 E Stevens Way NE, Seattle, WA 98195, USA
| | - Videlis Nduba
- Centre for Respiratory Diseases Research, Kenya Medical Research Institute, Mbagathi Rd, Nairobi 610101, Kenya
| | - Lilian N. Njagi
- Centre for Respiratory Diseases Research, Kenya Medical Research Institute, Mbagathi Rd, Nairobi 610101, Kenya
| | - Wilfred Murithi
- Centre for Respiratory Diseases Research, Kenya Medical Research Institute, Mbagathi Rd, Nairobi 610101, Kenya
| | - Zipporah Mwongera
- Centre for Respiratory Diseases Research, Kenya Medical Research Institute, Mbagathi Rd, Nairobi 610101, Kenya
| | - Thomas R. Hawn
- Department of Medicine, University of Washington, 1959 NE Pacific Street, Seattle, WA 98195, USA
| | - Shwetak N. Patel
- Department of Electrical and Computer Engineering, University of Washington, 185 E Stevens Way NE, Seattle, WA 98195, USA
- Paul G. Allen School of Computer Science and Engineering, University of Washington, 185 E Stevens Way NE, Seattle, WA 98195, USA
| | - David J. Horne
- Department of Medicine, University of Washington, 1959 NE Pacific Street, Seattle, WA 98195, USA
| |
Collapse
|
8
|
Razvadauskas H, Vaičiukynas E, Buškus K, Arlauskas L, Nowaczyk S, Sadauskas S, Naudžiūnas A. Exploring classical machine learning for identification of pathological lung auscultations. Comput Biol Med 2024; 168:107784. [PMID: 38042100 DOI: 10.1016/j.compbiomed.2023.107784] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 11/17/2023] [Accepted: 11/28/2023] [Indexed: 12/04/2023]
Abstract
The use of machine learning in biomedical research has surged in recent years thanks to advances in devices and artificial intelligence. Our aim is to expand this body of knowledge by applying machine learning to pulmonary auscultation signals. Despite improvements in digital stethoscopes and attempts to find synergy between them and artificial intelligence, solutions for their use in clinical settings remain scarce. Physicians continue to infer initial diagnoses with less sophisticated means, resulting in low accuracy, leading to suboptimal patient care. To arrive at a correct preliminary diagnosis, the auscultation diagnostics need to be of high accuracy. Due to the large number of auscultations performed, data availability opens up opportunities for more effective sound analysis. In this study, digital 6-channel auscultations of 45 patients were used in various machine learning scenarios, with the aim of distinguishing between normal and abnormal pulmonary sounds. Audio features (such as fundamental frequencies F0-4, loudness, HNR, DFA, as well as descriptive statistics of log energy, RMS and MFCC) were extracted using the Python library Surfboard. Windowing, feature aggregation, and concatenation strategies were used to prepare data for machine learning algorithms in unsupervised (fair-cut forest, outlier forest) and supervised (random forest, regularized logistic regression) settings. The evaluation was carried out using 9-fold stratified cross-validation repeated 30 times. Decision fusion by averaging the outputs for a subject was also tested and found to be helpful. Supervised models showed a consistent advantage over unsupervised ones, with random forest achieving a mean AUC ROC of 0.691 (accuracy 71.11%, Kappa 0.416, F1-score 0.675) in side-based detection and a mean AUC ROC of 0.721 (accuracy 68.89%, Kappa 0.371, F1-score 0.650) in patient-based detection.
Collapse
|
9
|
Khan SU, Jan SU, Koo I. Robust Epileptic Seizure Detection Using Long Short-Term Memory and Feature Fusion of Compressed Time-Frequency EEG Images. SENSORS (BASEL, SWITZERLAND) 2023; 23:9572. [PMID: 38067944 PMCID: PMC10708722 DOI: 10.3390/s23239572] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 11/27/2023] [Accepted: 11/28/2023] [Indexed: 12/18/2023]
Abstract
Epilepsy is a prevalent neurological disorder with considerable risks, including physical impairment and irreversible brain damage from seizures. Given these challenges, the urgency for prompt and accurate seizure detection cannot be overstated. Traditionally, experts have relied on manual EEG signal analyses for seizure detection, which is labor-intensive and prone to human error. Recognizing this limitation, the rise in deep learning methods has been heralded as a promising avenue, offering more refined diagnostic precision. On the other hand, the prevailing challenge in many models is their constrained emphasis on specific domains, potentially diminishing their robustness and precision in complex real-world environments. This paper presents a novel model that seamlessly integrates the salient features from the time-frequency domain along with pivotal statistical attributes derived from EEG signals. This fusion process involves the integration of essential statistics, including the mean, median, and variance, combined with the rich data from compressed time-frequency (CWT) images processed using autoencoders. This multidimensional feature set provides a robust foundation for subsequent analytic steps. A long short-term memory (LSTM) network, meticulously optimized for the renowned Bonn Epilepsy dataset, was used to enhance the capability of the proposed model. Preliminary evaluations underscore the prowess of the proposed model: a remarkable 100% accuracy in most of the binary classifications, exceeding 95% accuracy in three-class and four-class challenges, and a commendable rate, exceeding 93.5% for the five-class classification.
Collapse
Affiliation(s)
- Shafi Ullah Khan
- Department of Electrical Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea
| | - Sana Ullah Jan
- School of Computing, Edinburgh Napier University, Edinburgh EH10 5DT, UK;
| | - Insoo Koo
- Department of Electrical Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea
| |
Collapse
|
10
|
Im S, Kim T, Min C, Kang S, Roh Y, Kim C, Kim M, Kim SH, Shim K, Koh JS, Han S, Lee J, Kim D, Kang D, Seo S. Real-time counting of wheezing events from lung sounds using deep learning algorithms: Implications for disease prediction and early intervention. PLoS One 2023; 18:e0294447. [PMID: 37983213 PMCID: PMC10659186 DOI: 10.1371/journal.pone.0294447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 10/23/2023] [Indexed: 11/22/2023] Open
Abstract
This pioneering study aims to revolutionize self-symptom management and telemedicine-based remote monitoring through the development of a real-time wheeze counting algorithm. Leveraging a novel approach that includes the detailed labeling of one breathing cycle into three types: break, normal, and wheeze, this study not only identifies abnormal sounds within each breath but also captures comprehensive data on their location, duration, and relationships within entire respiratory cycles, including atypical patterns. This innovative strategy is based on a combination of a one-dimensional convolutional neural network (1D-CNN) and a long short-term memory (LSTM) network model, enabling real-time analysis of respiratory sounds. Notably, it stands out for its capacity to handle continuous data, distinguishing it from conventional lung sound classification algorithms. The study utilizes a substantial dataset consisting of 535 respiration cycles from diverse sources, including the Child Sim Lung Sound Simulator, the EMTprep Open-Source Database, Clinical Patient Records, and the ICBHI 2017 Challenge Database. Achieving a classification accuracy of 90%, the exceptional result metrics encompass the identification of each breath cycle and simultaneous detection of the abnormal sound, enabling the real-time wheeze counting of all respirations. This innovative wheeze counter holds the promise of revolutionizing research on predicting lung diseases based on long-term breathing patterns and offers applicability in clinical and non-clinical settings for on-the-go detection and remote intervention of exacerbated respiratory symptoms.
Collapse
Affiliation(s)
- Sunghoon Im
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - Taewi Kim
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | | | - Sanghun Kang
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - Yeonwook Roh
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - Changhwan Kim
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - Minho Kim
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - Seung Hyun Kim
- Department of Medical Humanities, Korea University College of Medicine, Seoul, Republic of Korea
| | - KyungMin Shim
- Industry-University Cooperation Foundation, Seogyeong University, Seoul, Republic of Korea
| | - Je-sung Koh
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - Seungyong Han
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - JaeWang Lee
- Department of Biomedical Laboratory Science, College of Health Science, Eulji University, Seongnam-si, Gyeonggi-do, Republic of Korea
| | - Dohyeong Kim
- University of Texas at Dallas, Richardson, TX, United States of America
| | - Daeshik Kang
- Department of Mechanical Engineering, Ajou University, Suwon-si, Gyeonggi-do, Republic of Korea
| | - SungChul Seo
- Department of Nano-Chemical, Biological and Environmental Engineering, Seogyeong University, Seoul, Republic of Korea
| |
Collapse
|
11
|
Garcia-Mendez JP, Lal A, Herasevich S, Tekin A, Pinevich Y, Lipatov K, Wang HY, Qamar S, Ayala IN, Khapov I, Gerberi DJ, Diedrich D, Pickering BW, Herasevich V. Machine Learning for Automated Classification of Abnormal Lung Sounds Obtained from Public Databases: A Systematic Review. Bioengineering (Basel) 2023; 10:1155. [PMID: 37892885 PMCID: PMC10604310 DOI: 10.3390/bioengineering10101155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 09/15/2023] [Accepted: 09/26/2023] [Indexed: 10/29/2023] Open
Abstract
Pulmonary auscultation is essential for detecting abnormal lung sounds during physical assessments, but its reliability depends on the operator. Machine learning (ML) models offer an alternative by automatically classifying lung sounds. ML models require substantial data, and public databases aim to address this limitation. This systematic review compares characteristics, diagnostic accuracy, concerns, and data sources of existing models in the literature. Papers published from five major databases between 1990 and 2022 were assessed. Quality assessment was accomplished with a modified QUADAS-2 tool. The review encompassed 62 studies utilizing ML models and public-access databases for lung sound classification. Artificial neural networks (ANN) and support vector machines (SVM) were frequently employed in the ML classifiers. The accuracy ranged from 49.43% to 100% for discriminating abnormal sound types and 69.40% to 99.62% for disease class classification. Seventeen public databases were identified, with the ICBHI 2017 database being the most used (66%). The majority of studies exhibited a high risk of bias and concerns related to patient selection and reference standards. Summarizing, ML models can effectively classify abnormal lung sounds using publicly available data sources. Nevertheless, inconsistent reporting and methodologies pose limitations to advancing the field, and therefore, public databases should adhere to standardized recording and labeling procedures.
Collapse
Affiliation(s)
- Juan P. Garcia-Mendez
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Amos Lal
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN 55905, USA
| | - Svetlana Herasevich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Aysun Tekin
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Yuliya Pinevich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
- Department of Cardiac Anesthesiology and Intensive Care, Republican Clinical Medical Center, 223052 Minsk, Belarus
| | - Kirill Lipatov
- Division of Pulmonary Medicine, Mayo Clinic Health Systems, Essentia Health, Duluth, MN 55805, USA
| | - Hsin-Yi Wang
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
- Department of Anesthesiology, Taipei Veterans General Hospital, National Yang Ming Chiao Tung University, Taipei 11217, Taiwan
- Department of Biomedical Sciences and Engineering, National Central University, Taoyuan 320317, Taiwan
| | - Shahraz Qamar
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Ivan N. Ayala
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Ivan Khapov
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | | | - Daniel Diedrich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Brian W. Pickering
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, MN 55905, USA (Y.P.); (H.-Y.W.); (I.K.); (V.H.)
| |
Collapse
|
12
|
Pessoa D, Rocha BM, Strodthoff C, Gomes M, Rodrigues G, Petmezas G, Cheimariotis GA, Kilintzis V, Kaimakamis E, Maglaveras N, Marques A, Frerichs I, Carvalho PD, Paiva RP. BRACETS: Bimodal repository of auscultation coupled with electrical impedance thoracic signals. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107720. [PMID: 37544061 DOI: 10.1016/j.cmpb.2023.107720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Revised: 06/27/2023] [Accepted: 07/10/2023] [Indexed: 08/08/2023]
Abstract
BACKGROUND AND OBJECTIVE Respiratory diseases are among the most significant causes of morbidity and mortality worldwide, causing substantial strain on society and health systems. Over the last few decades, there has been increasing interest in the automatic analysis of respiratory sounds and electrical impedance tomography (EIT). Nevertheless, no publicly available databases with both respiratory sound and EIT data are available. METHODS In this work, we have assembled the first open-access bimodal database focusing on the differential diagnosis of respiratory diseases (BRACETS: Bimodal Repository of Auscultation Coupled with Electrical Impedance Thoracic Signals). It includes simultaneous recordings of single and multi-channel respiratory sounds and EIT. Furthermore, we have proposed several machine learning-based baseline systems for automatically classifying respiratory diseases in six distinct evaluation tasks using respiratory sound and EIT (A1, A2, A3, B1, B2, B3). These tasks included classifying respiratory diseases at sample and subject levels. The performance of the classification models was evaluated using a 5-fold cross-validation scheme (with subject isolation between folds). RESULTS The resulting database consists of 1097 respiratory sounds and 795 EIT recordings acquired from 78 adult subjects in two countries (Portugal and Greece). In the task of automatically classifying respiratory diseases, the baseline classification models have achieved the following average balanced accuracy: Task A1 - 77.9±13.1%; Task A2 - 51.6±9.7%; Task A3 - 38.6±13.1%; Task B1 - 90.0±22.4%; Task B2 - 61.4±11.8%; Task B3 - 50.8±10.6%. CONCLUSION The creation of this database and its public release will aid the research community in developing automated methodologies to assess and monitor respiratory function, and it might serve as a benchmark in the field of digital medicine for managing respiratory diseases. Moreover, it could pave the way for creating multi-modal robust approaches for that same purpose.
Collapse
Affiliation(s)
- Diogo Pessoa
- University of Coimbra Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, 3030-290 Coimbra, Portugal.
| | - Bruno Machado Rocha
- University of Coimbra Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, 3030-290 Coimbra, Portugal
| | - Claas Strodthoff
- Department of Anesthesiology, and Intensive Care Medicine, University Medical Center Schleswig-Holstein Campus Kiel, Kiel 24105, Schleswig-Holstein, Germany
| | - Maria Gomes
- Lab3R - Respiratory Research and Rehabilitation Laboratory, School of Health Sciences (ESSUA), University of Aveiro, 3810-193 Aveiro, Portugal
| | - Guilherme Rodrigues
- Lab3R - Respiratory Research and Rehabilitation Laboratory, School of Health Sciences (ESSUA), University of Aveiro, 3810-193 Aveiro, Portugal
| | - Georgios Petmezas
- 2nd Department of Obstetrics and Gynaecology, The Medical School, 54124 Thessaloniki, Greece
| | | | - Vassilis Kilintzis
- 2nd Department of Obstetrics and Gynaecology, The Medical School, 54124 Thessaloniki, Greece
| | - Evangelos Kaimakamis
- 1st Intensive Care Unit, "G. Papanikolaou" General Hospital of Thessaloniki, 57010 Pilea Hortiatis, Greece
| | - Nicos Maglaveras
- 2nd Department of Obstetrics and Gynaecology, The Medical School, 54124 Thessaloniki, Greece
| | - Alda Marques
- Lab3R - Respiratory Research and Rehabilitation Laboratory, School of Health Sciences (ESSUA), University of Aveiro, 3810-193 Aveiro, Portugal; Institute of Biomedicine (iBiMED), University of Aveiro, 3810-193 Aveiro, Portugal
| | - Inéz Frerichs
- Department of Anesthesiology, and Intensive Care Medicine, University Medical Center Schleswig-Holstein Campus Kiel, Kiel 24105, Schleswig-Holstein, Germany
| | - Paulo de Carvalho
- University of Coimbra Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, 3030-290 Coimbra, Portugal
| | - Rui Pedro Paiva
- University of Coimbra Centre for Informatics and Systems of the University of Coimbra, Department of Informatics Engineering, 3030-290 Coimbra, Portugal
| |
Collapse
|
13
|
Huang DM, Huang J, Qiao K, Zhong NS, Lu HZ, Wang WJ. Deep learning-based lung sound analysis for intelligent stethoscope. Mil Med Res 2023; 10:44. [PMID: 37749643 PMCID: PMC10521503 DOI: 10.1186/s40779-023-00479-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 09/05/2023] [Indexed: 09/27/2023] Open
Abstract
Auscultation is crucial for the diagnosis of respiratory system diseases. However, traditional stethoscopes have inherent limitations, such as inter-listener variability and subjectivity, and they cannot record respiratory sounds for offline/retrospective diagnosis or remote prescriptions in telemedicine. The emergence of digital stethoscopes has overcome these limitations by allowing physicians to store and share respiratory sounds for consultation and education. On this basis, machine learning, particularly deep learning, enables the fully-automatic analysis of lung sounds that may pave the way for intelligent stethoscopes. This review thus aims to provide a comprehensive overview of deep learning algorithms used for lung sound analysis to emphasize the significance of artificial intelligence (AI) in this field. We focus on each component of deep learning-based lung sound analysis systems, including the task categories, public datasets, denoising methods, and, most importantly, existing deep learning methods, i.e., the state-of-the-art approaches to convert lung sounds into two-dimensional (2D) spectrograms and use convolutional neural networks for the end-to-end recognition of respiratory diseases or abnormal lung sounds. Additionally, this review highlights current challenges in this field, including the variety of devices, noise sensitivity, and poor interpretability of deep models. To address the poor reproducibility and variety of deep learning in this field, this review also provides a scalable and flexible open-source framework that aims to standardize the algorithmic workflow and provide a solid basis for replication and future extension: https://github.com/contactless-healthcare/Deep-Learning-for-Lung-Sound-Analysis .
Collapse
Affiliation(s)
- Dong-Min Huang
- Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, 518055, Guangdong, China
| | - Jia Huang
- The Third People's Hospital of Shenzhen, Shenzhen, 518112, Guangdong, China
| | - Kun Qiao
- The Third People's Hospital of Shenzhen, Shenzhen, 518112, Guangdong, China
| | - Nan-Shan Zhong
- Guangzhou Institute of Respiratory Health, China State Key Laboratory of Respiratory Disease, National Clinical Research Center for Respiratory Disease, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou, 510120, China.
| | - Hong-Zhou Lu
- The Third People's Hospital of Shenzhen, Shenzhen, 518112, Guangdong, China.
| | - Wen-Jin Wang
- Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, 518055, Guangdong, China.
| |
Collapse
|
14
|
Choi Y, Lee H. Interpretation of lung disease classification with light attention connected module. Biomed Signal Process Control 2023; 84:104695. [PMID: 36879856 PMCID: PMC9978539 DOI: 10.1016/j.bspc.2023.104695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Revised: 12/21/2022] [Accepted: 02/11/2023] [Indexed: 03/06/2023]
Abstract
Lung diseases lead to complications from obstructive diseases, and the COVID-19 pandemic has increased lung disease-related deaths. Medical practitioners use stethoscopes to diagnose lung disease. However, an artificial intelligence model capable of objective judgment is required since the experience and diagnosis of respiratory sounds differ. Therefore, in this study, we propose a lung disease classification model that uses an attention module and deep learning. Respiratory sounds were extracted using log-Mel spectrogram MFCC. Normal and five types of adventitious sounds were effectively classified by improving VGGish and adding a light attention connected module to which the efficient channel attention module (ECA-Net) was applied. The performance of the model was evaluated for accuracy, precision, sensitivity, specificity, f1-score, and balanced accuracy, which were 92.56%, 92.81%, 92.22%, 98.50%, 92.29%, and 95.4%, respectively. We confirmed high performance according to the attention effect. The classification causes of lung diseases were analyzed using gradient-weighted class activation mapping (Grad-CAM), and the performances of their models were compared using open lung sounds measured using a Littmann 3200 stethoscope. The experts' opinions were also included. Our results will contribute to the early diagnosis and interpretation of diseases in patients with lung disease by utilizing algorithms in smart medical stethoscopes.
Collapse
Affiliation(s)
- Youngjin Choi
- School of Industrial Management Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea
| | - Hongchul Lee
- School of Industrial Management Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Republic of Korea
| |
Collapse
|
15
|
Mang L, Canadas-Quesada F, Carabias-Orti J, Combarro E, Ranilla J. Cochleogram-based adventitious sounds classification using convolutional neural networks. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
|
16
|
Farhan AMQ, Yang S. Automatic lung disease classification from the chest X-ray images using hybrid deep learning algorithm. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-27. [PMID: 37362647 PMCID: PMC10030349 DOI: 10.1007/s11042-023-15047-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Revised: 08/30/2022] [Accepted: 02/27/2023] [Indexed: 06/28/2023]
Abstract
The chest X-ray images provide vital information about the congestion cost-effectively. We propose a novel Hybrid Deep Learning Algorithm (HDLA) framework for automatic lung disease classification from chest X-ray images. The model consists of steps including pre-processing of chest X-ray images, automatic feature extraction, and detection. In a pre-processing step, our goal is to improve the quality of raw chest X-ray images using the combination of optimal filtering without data loss. The robust Convolutional Neural Network (CNN) is proposed using the pre-trained model for automatic lung feature extraction. We employed the 2D CNN model for the optimum feature extraction in minimum time and space requirements. The proposed 2D CNN model ensures robust feature learning with highly efficient 1D feature estimation from the input pre-processed image. As the extracted 1D features have suffered from significant scale variations, we optimized them using min-max scaling. We classify the CNN features using the different machine learning classifiers such as AdaBoost, Support Vector Machine (SVM), Random Forest (RM), Backpropagation Neural Network (BNN), and Deep Neural Network (DNN). The experimental results claim that the proposed model improves the overall accuracy by 3.1% and reduces the computational complexity by 16.91% compared to state-of-the-art methods.
Collapse
Affiliation(s)
- Abobaker Mohammed Qasem Farhan
- School of information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Shangming Yang
- School of information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
17
|
Zhang M, Li M, Guo L, Liu J. A Low-Cost AI-Empowered Stethoscope and a Lightweight Model for Detecting Cardiac and Respiratory Diseases from Lung and Heart Auscultation Sounds. SENSORS (BASEL, SWITZERLAND) 2023; 23:2591. [PMID: 36904794 PMCID: PMC10007545 DOI: 10.3390/s23052591] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/26/2022] [Revised: 02/11/2023] [Accepted: 02/17/2023] [Indexed: 06/18/2023]
Abstract
Cardiac and respiratory diseases are the primary causes of health problems. If we can automate anomalous heart and lung sound diagnosis, we can improve the early detection of disease and enable the screening of a wider population than possible with manual screening. We propose a lightweight yet powerful model for simultaneous lung and heart sound diagnosis, which is deployable in an embedded low-cost device and is valuable in remote areas or developing countries where Internet access may not be available. We trained and tested the proposed model with the ICBHI and the Yaseen datasets. The experimental results showed that our 11-class prediction model could achieve 99.94% accuracy, 99.84% precision, 99.89% specificity, 99.66% sensitivity, and 99.72% F1 score. We designed a digital stethoscope (around USD 5) and connected it to a low-cost, single-board-computer Raspberry Pi Zero 2W (around USD 20), on which our pretrained model can be smoothly run. This AI-empowered digital stethoscope is beneficial for anyone in the medical field, as it can automatically provide diagnostic results and produce digital audio records for further analysis.
Collapse
Affiliation(s)
- Miao Zhang
- School of Mathematics, Shandong University, Jinan 250100, China
- School of Mathematics and Statistics, Shandong University, Weihai 264200, China
| | - Min Li
- School of Mathematics and Statistics, Shandong University, Weihai 264200, China
| | - Liang Guo
- School of Mathematics and Statistics, Shandong University, Weihai 264200, China
- Data Science Institute, Shandong University, Jinan 250100, China
| | - Jianya Liu
- School of Mathematics, Shandong University, Jinan 250100, China
- Data Science Institute, Shandong University, Jinan 250100, China
| |
Collapse
|
18
|
Erfianto B, Rizal A, Hadiyoso S. Empirical Mode Decomposition and Hilbert Spectrum for Abnormality Detection in Normal and Abnormal Walking Transitions. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:3879. [PMID: 36900889 PMCID: PMC10002180 DOI: 10.3390/ijerph20053879] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Revised: 02/14/2023] [Accepted: 02/20/2023] [Indexed: 06/18/2023]
Abstract
Sensor-based human activity recognition (HAR) is a method for observing a person's activity in an environment. With this method, it is possible to monitor remotely. HAR can analyze a person's gait, whether normal or abnormal. Some of its applications may use several sensors mounted on the body, but this method tends to be complex and inconvenient. One alternative to wearable sensors is using video. One of the most commonly used HAR platforms is PoseNET. PoseNET is a sophisticated platform that can detect the skeleton and joints of the body, which are then known as joints. However, a method is still needed to process the raw data from PoseNET to detect subject activity. Therefore, this research proposes a way to detect abnormalities in gait using empirical mode decomposition and the Hilbert spectrum and transforming keys-joints, and skeletons from vision-based pose detection into the angular displacement of walking gait patterns (signals). Joint change information is extracted using the Hilbert Huang Transform to study how the subject behaves in the turning position. Furthermore, it is determined whether the transition goes from normal to abnormal subjects by calculating the energy in the time-frequency domain signal. The test results show that during the transition period, the energy of the gait signal tends to be higher than during the walking period.
Collapse
Affiliation(s)
- Bayu Erfianto
- School of Computing, Telkom University, Bandung 40257, Indonesia
| | - Achmad Rizal
- School of Electrical Engineering, Telkom University, Bandung 40257, Indonesia
| | - Sugondo Hadiyoso
- School of Applied Science, Telkom University, Bandung 40257, Indonesia
| |
Collapse
|
19
|
Song W, Han J. Patch-level contrastive embedding learning for respiratory sound classification. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
20
|
Wang W, Wang S, Qin D, Fang Y, Zheng Y. Heart-lung sound separation by nonnegative matrix factorization and deep learning. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104180] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
21
|
Sputum deposition classification for mechanically ventilated patients using LSTM method based on airflow signals. Heliyon 2022; 8:e11929. [DOI: 10.1016/j.heliyon.2022.e11929] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2022] [Revised: 06/15/2022] [Accepted: 11/11/2022] [Indexed: 12/03/2022] Open
|
22
|
Alqudah AM, Qazan S, Obeidat YM. Deep learning models for detecting respiratory pathologies from raw lung auscultation sounds. Soft comput 2022; 26:13405-13429. [PMID: 36186666 PMCID: PMC9510581 DOI: 10.1007/s00500-022-07499-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/09/2022] [Indexed: 11/23/2022]
Abstract
In recent years deep learning models improve the diagnosis performance of many diseases especially respiratory diseases. This paper will propose an evaluation for the performance of different deep learning models associated with the raw lung auscultation sounds in detecting respiratory pathologies to help in providing diagnostic of respiratory pathologies in digital recorded respiratory sounds. Also, we will find out the best deep learning model for this task. In this paper, three different deep learning models have been evaluated on non-augmented and augmented datasets, where two different datasets have been utilized to generate four different sub-datasets. The results show that all the proposed deep learning methods were successful and achieved high performance in classifying the raw lung sounds, the methods were applied on different datasets and used either augmentation or non-augmentation. Among all proposed deep learning models, the CNN–LSTM model was the best model in all datasets for both augmentation and non-augmentation cases. The accuracy of CNN–LSTM model using non-augmentation was 99.6%, 99.8%, 82.4%, and 99.4% for datasets 1, 2, 3, and 4, respectively, and using augmentation was 100%, 99.8%, 98.0%, and 99.5% for datasets 1, 2, 3, and 4, respectively. While the augmentation process successfully helps the deep learning models in enhancing their performance on the testing datasets with a notable value. Moreover, the hybrid model that combines both CNN and LSTM techniques performed better than models that are based only on one of these techniques, this mainly refers to the use of CNN for automatic deep features extraction from lung sound while LSTM is used for classification.
Collapse
Affiliation(s)
- Ali Mohammad Alqudah
- Department of Biomedical Systems and Informatics Engineering, Hijjawi Faculty for Engineering Technology, Yarmouk University, Irbid, Jordan
| | - Shoroq Qazan
- Department of Computer Engineering, Hijjawi Faculty for Engineering Technology, Yarmouk University, Irbid, Jordan
| | - Yusra M Obeidat
- Department of Electronic Engineering, Hijjawi Faculty for Engineering Technology, Yarmouk University, Irbid, Jordan
| |
Collapse
|
23
|
A lightweight hybrid deep learning system for cardiac valvular disease classification. Sci Rep 2022; 12:14297. [PMID: 35995814 PMCID: PMC9395359 DOI: 10.1038/s41598-022-18293-7] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Accepted: 08/09/2022] [Indexed: 12/21/2022] Open
Abstract
Cardiovascular diseases (CVDs) are a prominent cause of death globally. The introduction of medical big data and Artificial Intelligence (AI) technology encouraged the effort to develop and deploy deep learning models for distinguishing heart sound abnormalities. These systems employ phonocardiogram (PCG) signals because of their lack of sophistication and cost-effectiveness. Automated and early diagnosis of cardiovascular diseases (CVDs) helps alleviate deadly complications. In this research, a cardiac diagnostic system that combined CNN and LSTM components was developed, it uses phonocardiogram (PCG) signals, and utilizes either augmented or non-augmented datasets. The proposed model discriminates five heart valvular conditions, namely normal, Aortic Stenosis (AS), Mitral Regurgitation (MR), Mitral Stenosis (MS), and Mitral Valve Prolapse (MVP). The findings demonstrate that the suggested end-to-end architecture yields outstanding performance concerning all important evaluation metrics. For the five classes problem using the open heart sound dataset, accuracy was 98.5%, F1-score was 98.501%, and Area Under the Curve (AUC) was 0.9978 for the non-augmented dataset and accuracy was 99.87%, F1-score was 99.87%, and AUC was 0.9985 for the augmented dataset. Model performance was further evaluated using the PhysioNet/Computing in Cardiology 2016 challenge dataset, for the two classes problem, accuracy was 93.76%, F1-score was 85.59%, and AUC was 0.9505. The achieved results show that the proposed system outperforms all previous works that use the same audio signal databases. In the future, the findings will help build a multimodal structure that uses both PCG and ECG signals.
Collapse
|
24
|
Neili Z, Sundaraj K. A comparative study of the spectrogram, scalogram, melspectrogram and gammatonegram time-frequency representations for the classification of lung sounds using the ICBHI database based on CNNs. BIOMED ENG-BIOMED TE 2022; 67:367-390. [PMID: 35926850 DOI: 10.1515/bmt-2022-0180] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Accepted: 06/21/2022] [Indexed: 11/15/2022]
Abstract
In lung sound classification using deep learning, many studies have considered the use of short-time Fourier transform (STFT) as the most commonly used 2D representation of the input data. Consequently, STFT has been widely used as an analytical tool, but other versions of the representation have also been developed. This study aims to evaluate and compare the performance of the spectrogram, scalogram, melspectrogram and gammatonegram representations, and provide comparative information to users regarding the suitability of these time-frequency (TF) techniques in lung sound classification. Lung sound signals used in this study were obtained from the ICBHI 2017 respiratory sound database. These lung sound recordings were converted into images of spectrogram, scalogram, melspectrogram and gammatonegram TF representations respectively. The four types of images were fed separately into the VGG16, ResNet-50 and AlexNet deep-learning architectures. Network performances were analyzed and compared based on accuracy, precision, recall and F1-score. The results of the analysis on the performance of the four representations using these three commonly used CNN deep-learning networks indicate that the generated gammatonegram and scalogram TF images coupled with ResNet-50 achieved maximum classification accuracies.
Collapse
Affiliation(s)
- Zakaria Neili
- Electronics Department, University of Badji Mokhtar Annaba, Annaba, Algeria
| | - Kenneth Sundaraj
- Faculty of Electronics and Computer Engineering, Universiti Teknikal Malaysia Melaka, Melaka, Malaysia
| |
Collapse
|
25
|
Kala A, McCollum ED, Elhilali M. Implications of clinical variability on computer-aided lung auscultation classification. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:4421-4425. [PMID: 36086501 DOI: 10.1109/embc48229.2022.9871393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Thanks to recent advances in digital stethoscopes and rapid adoption of deep learning techniques, there has been tremendous progress in the field of Computerized Auscultation Analysis (CAA). Despite these promising leaps, the deploy-ment of these technologies in real-world applications remains limited due to inherent challenges with properly interpreting clinical data, particularly auscultations. One of the limiting factors is the inherent ambiguity that comes with variability in clinical opinion, even from highly trained experts. The lack of unanimity in expert opinions is often ignored in developing machine learning techniques to automatically screen normal from abnormal lung signals, with most algorithms being developed and tested on highly curated datasets. To better understand the potential pitfalls this selective analysis could cause in deployment, the current work explores the impact of clinical opinion variability on algorithms to detect adventitious patterns in lung sounds when trained on gold-standard data. The study shows that uncertainty in clinical opinion introduces far more variability and performance drop than dissidence in expert judgments. The study also explores the feasibility of automatically flagging auscultation signals based on their estimated uncertainty, thereby recommending further reassessment as well as improving computer-aided analysis.
Collapse
|
26
|
Pham Thi Viet H, Nguyen Thi Ngoc H, Tran Anh V, Hoang Quang H. Classification of lung sounds using scalogram representation of sound segments and convolutional neural network. J Med Eng Technol 2022; 46:270-279. [PMID: 35212591 DOI: 10.1080/03091902.2022.2040624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
Lung auscultation is one of the most common methods for screening of lung diseases. The increasingly high rate of respiratory diseases leads to the need for robust methods to detect the abnormalities in patients' breathing sounds. Lung sounds analysis stands out as a promising approach to automatic screening of lung diseases, serving as a second opinion for doctors as a stand-alone device for preliminary screening of lung diseases in remote areas. In previous research on lung classification using ICBHI Database on Kaggle, lung audios are converted to spectral images and fed into deep neural networks for training. There are a few studies which uses the scalogram, however they focussed on classification among different lung diseases. The use of scalograms in categorising the sound types are rarely used. In this paper, we combined scalograms and neural networks for classification of lung sound types. Padding methods and augmentation are also considered to evaluate the impacts on classification score. An ensemble learning is incorporated to increase classification accuracy by utilising voting of many models. The model trained and evaluated has shown prominent improvement of this method on classification on the benchmark ICBHI database.
Collapse
Affiliation(s)
| | - Huyen Nguyen Thi Ngoc
- School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam
| | - Vu Tran Anh
- School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam
| | - Huy Hoang Quang
- School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam
| |
Collapse
|
27
|
Ahmed S, Sultana S, Khan AM, Islam MS, Habib GMM, McLane IM, McCollum ED, Baqui AH, Cunningham S, Nair H. Digital auscultation as a diagnostic aid to detect childhood pneumonia: A systematic review. J Glob Health 2022; 12:04033. [PMID: 35493777 PMCID: PMC9024283 DOI: 10.7189/jogh.12.04033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
Background Frontline health care workers use World Health Organization Integrated Management of Childhood Illnesses (IMCI) guidelines for child pneumonia care in low-resource settings. IMCI guideline pneumonia diagnostic criterion performs with low specificity, resulting in antibiotic overtreatment. Digital auscultation with automated lung sound analysis may improve the diagnostic performance of IMCI pneumonia guidelines. This systematic review aims to summarize the evidence on detecting adventitious lung sounds by digital auscultation with automated analysis compared to reference physician acoustic analysis for child pneumonia diagnosis. Methods In this review, articles were searched from MEDLINE, Embase, CINAHL Plus, Web of Science, Global Health, IEEExplore database, Scopus, and the ClinicalTrial.gov databases from the inception of each database to October 27, 2021, and reference lists of selected studies and relevant review articles were searched manually. Studies reporting diagnostic performance of digital auscultation and/or computerized lung sound analysis compared against physicians’ acoustic analysis for pneumonia diagnosis in children under the age of 5 were eligible for this systematic review. Retrieved citations were screened and eligible studies were included for extraction. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. All these steps were independently performed by two authors and disagreements between the reviewers were resolved through discussion with an arbiter. Narrative data synthesis was performed. Results A total of 3801 citations were screened and 46 full-text articles were assessed. 10 studies met the inclusion criteria. Half of the studies used a publicly available respiratory sound database to evaluate their proposed work. Reported methodologies/approaches and performance metrics for classifying adventitious lung sounds varied widely across the included studies. All included studies except one reported overall diagnostic performance of the digital auscultation/computerised sound analysis to distinguish adventitious lung sounds, irrespective of the disease condition or age of the participants. The reported accuracies for classifying adventitious lung sounds in the included studies varied from 66.3% to 100%. However, it remained unclear to what extent these results would be applicable for classifying adventitious lung sounds in children with pneumonia. Conclusions This systematic review found very limited evidence on the diagnostic performance of digital auscultation to diagnose pneumonia in children. Well-designed studies and robust reporting are required to evaluate the accuracy of digital auscultation in the paediatric population.
Collapse
Affiliation(s)
- Salahuddin Ahmed
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Projahnmo Research Foundation, Dhaka, Bangladesh
| | | | - Ahad M Khan
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Projahnmo Research Foundation, Dhaka, Bangladesh
| | - Mohammad S Islam
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Child Health Research Foundation, Dhaka, Bangladesh
| | - GM Monsur Habib
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Bangladesh Primary Care Respiratory Society, Khulna, Bangladesh
| | | | - Eric D McCollum
- Global Program for Pediatric Respiratory Sciences, Eudowood Division of Paediatric Respiratory Sciences, School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
- Department of International Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, USA
| | - Abdullah H Baqui
- Department of International Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, USA
| | - Steven Cunningham
- Department of Child Life and Health, Centre for Inflammation Research, University of Edinburgh, Edinburgh, UK
| | - Harish Nair
- Usher Institute, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
28
|
Nguyen T, Pernkopf F. Lung Sound Classification Using Co-tuning and Stochastic Normalization. IEEE Trans Biomed Eng 2022; 69:2872-2882. [PMID: 35254969 DOI: 10.1109/tbme.2022.3156293] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Computational methods for lung sound analysis are beneficial for computer-aided diagnosis support, storage and monitoring in critical care. In this paper, we use pre-trained ResNet models as backbone architectures for classification of adventitious lung sounds and respiratory diseases. The learned representation of the pre-trained model is transferred by using vanilla fine-tuning, co-tuning, stochastic normalization and the combination of the co-tuning and stochastic normalization techniques. Furthermore, data augmentation in both time domain and time-frequency domain is used to account for the class imbalance of the ICBHI and our multi-channel lung sound dataset. Additionally, we introduce spectrum correction to account for the variations of the recording device properties on the ICBHI dataset. Empirically, our proposed systems mostly outperform all state-of-the-art lung sound classification systems for the adventitious lung sounds and respiratory diseases of both datasets.
Collapse
|
29
|
LW-FIRE: A Lightweight Wildfire Image Classification with a Deep Convolutional Neural Network. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12052646] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Analysis of reports published by the leading national centers for monitoring wildfires and other emergencies revealed that the devastation caused by wildfires has increased by 2.96-fold when compared to a decade earlier. The reports show that the total number of wildfires is declining; however, their impact on the wildlife appears to be more devastating. In recent years, deep neural network models have demonstrated state-of-the-art accuracy on many computer vision tasks. In this paper, we describe the design and implementation of a lightweight wildfire image classification model (LW-FIRE) based on convolutional neural networks. We explore different ways of using the existing dataset to efficiently train a deep convolutional neural network. We also propose a new method for dataset transformation to increase the number of samples in the dataset and improve the accuracy and generalization of the deep learning model. Experimental results show that the proposed model outperforms the state-of-the-art methods, and is suitable for real-time classification of wildfire images.
Collapse
|
30
|
Tariq Z, Shah SK, Lee Y. Feature-Based Fusion Using CNN for Lung and Heart Sound Classification. SENSORS 2022; 22:s22041521. [PMID: 35214424 PMCID: PMC8875944 DOI: 10.3390/s22041521] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2021] [Revised: 02/10/2022] [Accepted: 02/14/2022] [Indexed: 02/04/2023]
Abstract
Lung or heart sound classification is challenging due to the complex nature of audio data, its dynamic properties of time, and frequency domains. It is also very difficult to detect lung or heart conditions with small amounts of data or unbalanced and high noise in data. Furthermore, the quality of data is a considerable pitfall for improving the performance of deep learning. In this paper, we propose a novel feature-based fusion network called FDC-FS for classifying heart and lung sounds. The FDC-FS framework aims to effectively transfer learning from three different deep neural network models built from audio datasets. The innovation of the proposed transfer learning relies on the transformation from audio data to image vectors and from three specific models to one fused model that would be more suitable for deep learning. We used two publicly available datasets for this study, i.e., lung sound data from ICHBI 2017 challenge and heart challenge data. We applied data augmentation techniques, such as noise distortion, pitch shift, and time stretching, dealing with some data issues in these datasets. Importantly, we extracted three unique features from the audio samples, i.e., Spectrogram, MFCC, and Chromagram. Finally, we built a fusion of three optimal convolutional neural network models by feeding the image feature vectors transformed from audio features. We confirmed the superiority of the proposed fusion model compared to the state-of-the-art works. The highest accuracy we achieved with FDC-FS is 99.1% with Spectrogram-based lung sound classification while 97% for Spectrogram and Chromagram based heart sound classification.
Collapse
|
31
|
Alkhodari M, Khandoker AH. Detection of COVID-19 in smartphone-based breathing recordings: A pre-screening deep learning tool. PLoS One 2022; 17:e0262448. [PMID: 35025945 PMCID: PMC8758005 DOI: 10.1371/journal.pone.0262448] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Accepted: 12/24/2021] [Indexed: 12/14/2022] Open
Abstract
This study was sought to investigate the feasibility of using smartphone-based breathing sounds within a deep learning framework to discriminate between COVID-19, including asymptomatic, and healthy subjects. A total of 480 breathing sounds (240 shallow and 240 deep) were obtained from a publicly available database named Coswara. These sounds were recorded by 120 COVID-19 and 120 healthy subjects via a smartphone microphone through a website application. A deep learning framework was proposed herein that relies on hand-crafted features extracted from the original recordings and from the mel-frequency cepstral coefficients (MFCC) as well as deep-activated features learned by a combination of convolutional neural network and bi-directional long short-term memory units (CNN-BiLSTM). The statistical analysis of patient profiles has shown a significant difference (p-value: 0.041) for ischemic heart disease between COVID-19 and healthy subjects. The Analysis of the normal distribution of the combined MFCC values showed that COVID-19 subjects tended to have a distribution that is skewed more towards the right side of the zero mean (shallow: 0.59±1.74, deep: 0.65±4.35, p-value: <0.001). In addition, the proposed deep learning approach had an overall discrimination accuracy of 94.58% and 92.08% using shallow and deep recordings, respectively. Furthermore, it detected COVID-19 subjects successfully with a maximum sensitivity of 94.21%, specificity of 94.96%, and area under the receiver operating characteristic (AUROC) curves of 0.90. Among the 120 COVID-19 participants, asymptomatic subjects (18 subjects) were successfully detected with 100.00% accuracy using shallow recordings and 88.89% using deep recordings. This study paves the way towards utilizing smartphone-based breathing sounds for the purpose of COVID-19 detection. The observations found in this study were promising to suggest deep learning and smartphone-based breathing sounds as an effective pre-screening tool for COVID-19 alongside the current reverse-transcription polymerase chain reaction (RT-PCR) assay. It can be considered as an early, rapid, easily distributed, time-efficient, and almost no-cost diagnosis technique complying with social distancing restrictions during COVID-19 pandemic.
Collapse
Affiliation(s)
- Mohanad Alkhodari
- Healthcare Engineering Innovation Center (HEIC), Department of Biomedical Engineering, Khalifa University, Abu Dhabi, UAE
| | - Ahsan H. Khandoker
- Healthcare Engineering Innovation Center (HEIC), Department of Biomedical Engineering, Khalifa University, Abu Dhabi, UAE
| |
Collapse
|
32
|
Fernando T, Sridharan S, Denman S, Ghaemmaghami H, Fookes C. Robust and Interpretable Temporal Convolution Network for Event Detection in Lung Sound Recordings. IEEE J Biomed Health Inform 2022; 26:2898-2908. [DOI: 10.1109/jbhi.2022.3144314] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
33
|
Roslidar R, Syaryadhi M, Saddami K, Pradhan B, Arnia F, Syukri M, Munadi K. BreaCNet: A high-accuracy breast thermogram classifier based on mobile convolutional neural network. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2022; 19:1304-1331. [PMID: 35135205 DOI: 10.3934/mbe.2022060] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The presence of a well-trained, mobile CNN model with a high accuracy rate is imperative to build a mobile-based early breast cancer detector. In this study, we propose a mobile neural network model breast cancer mobile network (BreaCNet) and its implementation framework. BreaCNet consists of an effective segmentation algorithm for breast thermograms and a classifier based on the mobile CNN model. The segmentation algorithm employing edge detection and second-order polynomial curve fitting techniques can effectively capture the thermograms' region of interest (ROI), thereby facilitating efficient feature extraction. The classifier was developed based on ShuffleNet by adding one block consisting of a convolutional layer with 1028 filters. The modified Shufflenet demonstrated a good fit learning with 6.1 million parameters and 22 MB size. Simulation results showed that modified ShuffleNet alone resulted in a 72% accuracy rate, but the performance excelled to a 100% accuracy rate when integrated with the proposed segmentation algorithm. In terms of diagnostic accuracy of the normal and abnormal test, BreaCNet significantly improves the sensitivity rate from 43% to 100% and specificity of 100%. We confirmed that feeding only the ROI of the input dataset to the network can improve the classifier's performance. On the implementation aspect of BreaCNet, the on-device inference is recommended to ensure users' data privacy and handle an unreliable network connection.
Collapse
Affiliation(s)
- Roslidar Roslidar
- Doctoral Program, School of Engineering, Universitas Syiah Kuala, Banda Aceh, Indonesia
- Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Indonesia
- Telematics Research Center, Universitas Syiah Kuala, Banda Aceh, Indonesia
| | - Mohd Syaryadhi
- Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Indonesia
| | - Khairun Saddami
- Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Indonesia
| | - Biswajeet Pradhan
- Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), Faculty of Engineering and IT, University of Technology Sydney, Ultimo, Australia
- Center of Excellence for Climate Change Research, King Abdulaziz University, Jeddah, Saudi Arabia
- Earth Observation Center, Institute of Climate Change, Universiti Kebangsaan Malaysia, Bangi, Malaysia
| | - Fitri Arnia
- Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Indonesia
- Telematics Research Center, Universitas Syiah Kuala, Banda Aceh, Indonesia
| | - Maimun Syukri
- Medical Faculty, Universitas Syiah Kuala, Banda Aceh, Indonesia
| | - Khairul Munadi
- Department of Electrical and Computer Engineering, Universitas Syiah Kuala, Indonesia
- Telematics Research Center, Universitas Syiah Kuala, Banda Aceh, Indonesia
- Tsunami and Disaster Mitigation Research Center, Universitas Syiah Kuala, Banda Aceh, Indonesia
| |
Collapse
|
34
|
Zhu H, Lai J, Liu B, Wen Z, Xiong Y, Li H, Zhou Y, Fu Q, Yu G, Yan X, Yang X, Zhang J, Wang C, Zeng H. Automatic pulmonary auscultation grading diagnosis of Coronavirus Disease 2019 in China with artificial intelligence algorithms: A cohort study. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 213:106500. [PMID: 34768234 PMCID: PMC8550891 DOI: 10.1016/j.cmpb.2021.106500] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/05/2021] [Accepted: 10/20/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND AND OBJECTIVE Research on automatic auscultation diagnosis of COVID-19 has not yet been developed. We therefore aimed to engineer a deep learning approach for the automated grading diagnosis of COVID-19 by pulmonary auscultation analysis. METHODS 172 confirmed cases of COVID-19 in Tongji Hospital were divided into moderate, severe and critical group. Pulmonary auscultation were recorded in 6-10 sites per patient through 3M littmann stethoscope and the data were transferred to computer to construct the dataset. Convolutional neural network (CNN) were designed to generate classifications of the auscultation. F1 score, the area under the curve (AUC) of the receiver operating characteristic curve, sensitivity and specificity were quantified. Another 45 normal patients were served as control group. RESULTS There are about 56.52%, 59.46% and 78.85% abnormal auscultation in the moderate, severe and critical groups respectively. The model showed promising performance with an averaged F1 scores (0.9938 95% CI 0.9923-0.9952), AUC ROC score (0.9999 95% CI 0.9998-1.0000), sensitivity (0.9938 95% CI 0.9910-0.9965) and specificity (0.9979 95% CI 0.9970-0.9988) in identifying the COVID-19 patients among normal, moderate, severe and critical group. It is capable in identifying crackles, wheezes, phlegm sounds with an averaged F1 scores (0.9475 95% CI 0.9440-0.9508), AUC ROC score (0.9762 95% CI 0.9848-0.9865), sensitivity (0.9482 95% CI 0.9393-0.9578) and specificity (0.9835 95% CI 0.9806-0.9863). CONCLUSIONS Our model is accurate and efficient in automatically diagnosing COVID-19 according to different categories, laying a promising foundation for AI-enabled auscultation diagnosing systems for lung diseases in clinical applications.
Collapse
Affiliation(s)
- Hongling Zhu
- Division of Cardiology, Department of Internal Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430030, China
| | - Jinsheng Lai
- Division of Cardiology, Department of Internal Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430030, China
| | - Bingqiang Liu
- School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China; Wuhan National Laboratory of Optoelectronics, Wuhan, Hubei 430074, China
| | - Ziyuan Wen
- School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China; Wuhan National Laboratory of Optoelectronics, Wuhan, Hubei 430074, China
| | - Yulong Xiong
- Division of Cardiology, Department of Internal Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430030, China
| | - Honglin Li
- Division of Cardiology, Department of Internal Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430030, China
| | - Yuhua Zhou
- Department of Cardiology, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, 197 Rui Jin Road II, Shanghai, 200025, China
| | - Qiuyun Fu
- School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China; Wuhan National Laboratory of Optoelectronics, Wuhan, Hubei 430074, China
| | - Guoyi Yu
- School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China; Wuhan National Laboratory of Optoelectronics, Wuhan, Hubei 430074, China
| | - Xiaoxiang Yan
- Department of Cardiology, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, 197 Rui Jin Road II, Shanghai, 200025, China
| | - Xiaoyun Yang
- Division of Cardiology, Department of Internal Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430030, China
| | - Jianmin Zhang
- School of Artificial Intelligence, Jianghan University, Wuhan, Hubei 430056, China
| | - Chao Wang
- School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China; Wuhan National Laboratory of Optoelectronics, Wuhan, Hubei 430074, China.
| | - Hesong Zeng
- Division of Cardiology, Department of Internal Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei 430030, China.
| |
Collapse
|
35
|
McLane I, Lauwers E, Stas T, Busch-Vishniac I, Ides K, Verhulst S, Steckel J. Comprehensive Analysis System for Automated Respiratory Cycle Segmentation and Crackle Peak Detection. IEEE J Biomed Health Inform 2021; 26:1847-1860. [PMID: 34705660 DOI: 10.1109/jbhi.2021.3123353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Digital auscultation is a well-known method for assessing lung sounds, but remains a subjective process in typical practice, relying on the human interpretation. Several methods have been presented for detecting or analyzing crackles but are limited in their real-world application because few have been integrated into comprehensive systems or validated on non-ideal data. This work details a complete signal analysis methodology for analyzing crackles in challenging recordings. The procedure comprises five sequential processing blocks: (1) motion artifact detection, (2) deep learning denoising network, (3) respiratory cycle segmentation, (4) separation of discontinuous adventitious sounds from vesicular sounds, and (5) crackle peak detection. This system uses a collection of new methods and robustness-focused improvements on previous methods to analyze respiratory cycles and crackles therein. To validate the accuracy, the system is tested on a database of 1000 simulated lung sounds with varying levels of motion artifacts, ambient noise, cycle lengths and crackle intensities, in which ground truths are exactly known. The system performs with average F-score of 91.07% for detecting motion artifacts and 94.43% for respiratory cycle extraction, and an overall F-score of 94.08% for detecting the locations of individual crackles. The process also successfully detects healthy recordings. Preliminary validation is also presented on a small set of 20 patient recordings, for which the system performs comparably. These methods provide quantifiable analysis of respiratory sounds to enable clinicians to distinguish between types of crackles, their timing within the respiratory cycle, and the level of occurrence. Crackles are one of the most common abnormal lung sounds, presenting in multiple cardiorespiratory diseases. These features will contribute to a better understanding of disease severity and progression in an objective, simple and non-invasive way.
Collapse
|
36
|
Lin JD, Lin HH, Dy J, Chen JC, Tanveer M, Razzak I, Hua KL. Lightweight Face Anti-Spoofing Network for Telehealth Applications. IEEE J Biomed Health Inform 2021; 26:1987-1996. [PMID: 34432642 DOI: 10.1109/jbhi.2021.3107735] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Online healthcare applications have grown more popular over the years. For instance,telehealth is an online healthcare application that allows patients and doctors to schedule consultations,prescribe medication,share medical documents,and monitor health conditions conveniently. Apart from this,telehealth can also be used to store a patients personal and medical information. Given the amount of sensitive data it stores,security measures are necessary. With its rise in usage due to COVID-19,its usefulness may be undermined if security issues are not addressed. A simple way of making these applications more secure is through user authentication. One of the most common and often used authentications is face recognition. It is convenient and easy to use. However,face recognition systems are not foolproof. They are prone to malicious attacks like printed photos,paper cutouts,re-played videos,and 3D masks. In order to counter this,multiple face anti-spoofing methods have been proposed. The goal of face anti-spoofing is to differentiate real users (live) from attackers (spoof). Although effective in terms of performance,existing methods use a significant amount of parameters,making them resource-heavy and unsuitable for handheld devices. Apart from this,they fail to generalize well to new environments like changes in lighting or background. This paper proposes a lightweight face anti-spoofing framework that does not compromise on performance. A lightweight model is critical for applications like telehealth that run on handheld devices. Our proposed method achieves good performance with the help of an ArcFace Classifier (AC). The AC encourages differentiation between spoof and live samples by making clear boundaries between them. With clear boundaries,classification becomes more accurate. We further demonstrate our models capabilities by comparing the number of parameters,FLOPS,and performance with other state-of-the-art methods.
Collapse
|