Detecting Respiratory Pathologies Using Convolutional Neural Networks and Variational Autoencoders for Unbalancing Data

Auscultation-Based Pulmonary Disease Detection through Parallel Transformation and Deep Learning

Respiratory diseases are among the leading causes of death, with many individuals in a population frequently affected by various types of pulmonary disorders. Early diagnosis and patient monitoring (traditionally involving lung auscultation) are essential for the effective management of respiratory diseases. However, the interpretation of lung sounds is a subjective and labor-intensive process that demands considerable medical expertise, and there is a good chance of misclassification. To address this problem, we propose a hybrid deep learning technique that incorporates signal processing techniques. Parallel transformation is applied to adventitious respiratory sounds, transforming lung sound signals into two distinct time-frequency scalograms: the continuous wavelet transform and the mel spectrogram. Furthermore, parallel convolutional autoencoders are employed to extract features from scalograms, and the resulting latent space features are fused into a hybrid feature pool. Finally, leveraging a long short-term memory model, a feature from the latent space is used as input for classifying various types of respiratory diseases. Our work is evaluated using the ICBHI-2017 lung sound dataset. The experimental findings indicate that our proposed method achieves promising predictive performance, with average values for accuracy, sensitivity, specificity, and F1-score of 94.16%, 89.56%, 99.10%, and 89.56%, respectively, for eight-class respiratory diseases; 79.61%, 78.55%, 92.49%, and 78.67%, respectively, for four-class diseases; and 85.61%, 83.44%, 83.44%, and 84.21%, respectively, for binary-class (normal vs. abnormal) lung sounds.

Weighted aggregation through probability based ranking: An optimized federated learning architecture to classify respiratory diseases

Background and Objective Respiratory Diseases are one of the leading chronic illnesses in the world according to the reports by World Health Organization. Diagnosing these respiratory diseases is done through auscultation where a medical professional listens to sounds of air in the lungs for anomalies through a stethoscope. This method necessitates extensive experience and can also be misinterpreted by the medical professional. To address this issue, we introduce an AI-based solution that listens to the lung sounds and classifies the respiratory disease detected. Since the research work deals with medical data that is tightly under wraps due to privacy concerns in the medical field, we introduce a Deep learning solution to classify the diseases and a custom Federated learning (FL) approach to further improve the accuracy of the deep learning model and simultaneously maintain data privacy. Federated Learning architecture maintains data privacy and facilitates a distributed learning system for medical infrastructures. Methods The approach utilizes Generative Adversarial Networks (GAN) based Federated learning approach to ensure data privacy. Generative Adversarial Networks generate new data by synthesizing new lung sounds. This new synthesized data is then converted to spectrograms and trained on a neural network to classify four lung diseases, Heart Attack and Normal breathing patterns. Furthermore, to address performance loss during FL, we also propose a new "Weighted Aggregation through Probability-based Ranking (FedWAPR)" algorithm for optimizing the FL aggregation process. The FedWAPR aggregation takes inspiration from exponential distribution function and ranks better performing clients according to it. Results and Conclusion A test accuracy of about 92% was achieved by the trained model while classifying various respiratory diseases and heart failure. Additionally, we developed a novel FedWAPR approach that significantly outperformed the FedAVG approach for the FL aggregate function. A patient can be checked for respiratory diseases using this improved learning approach without the need for extensive sensitive data recording or for making sure the data sample obtained is secure. In a decentralized training runtime, the trained model successfully classifies various respiratory diseases and heart failure using lung sounds with a test accuracy on par with a centralized model.

Real-time counting of wheezing events from lung sounds using deep learning algorithms: Implications for disease prediction and early intervention

This pioneering study aims to revolutionize self-symptom management and telemedicine-based remote monitoring through the development of a real-time wheeze counting algorithm. Leveraging a novel approach that includes the detailed labeling of one breathing cycle into three types: break, normal, and wheeze, this study not only identifies abnormal sounds within each breath but also captures comprehensive data on their location, duration, and relationships within entire respiratory cycles, including atypical patterns. This innovative strategy is based on a combination of a one-dimensional convolutional neural network (1D-CNN) and a long short-term memory (LSTM) network model, enabling real-time analysis of respiratory sounds. Notably, it stands out for its capacity to handle continuous data, distinguishing it from conventional lung sound classification algorithms. The study utilizes a substantial dataset consisting of 535 respiration cycles from diverse sources, including the Child Sim Lung Sound Simulator, the EMTprep Open-Source Database, Clinical Patient Records, and the ICBHI 2017 Challenge Database. Achieving a classification accuracy of 90%, the exceptional result metrics encompass the identification of each breath cycle and simultaneous detection of the abnormal sound, enabling the real-time wheeze counting of all respirations. This innovative wheeze counter holds the promise of revolutionizing research on predicting lung diseases based on long-term breathing patterns and offers applicability in clinical and non-clinical settings for on-the-go detection and remote intervention of exacerbated respiratory symptoms.

Machine Learning for Automated Classification of Abnormal Lung Sounds Obtained from Public Databases: A Systematic Review

Pulmonary auscultation is essential for detecting abnormal lung sounds during physical assessments, but its reliability depends on the operator. Machine learning (ML) models offer an alternative by automatically classifying lung sounds. ML models require substantial data, and public databases aim to address this limitation. This systematic review compares characteristics, diagnostic accuracy, concerns, and data sources of existing models in the literature. Papers published from five major databases between 1990 and 2022 were assessed. Quality assessment was accomplished with a modified QUADAS-2 tool. The review encompassed 62 studies utilizing ML models and public-access databases for lung sound classification. Artificial neural networks (ANN) and support vector machines (SVM) were frequently employed in the ML classifiers. The accuracy ranged from 49.43% to 100% for discriminating abnormal sound types and 69.40% to 99.62% for disease class classification. Seventeen public databases were identified, with the ICBHI 2017 database being the most used (66%). The majority of studies exhibited a high risk of bias and concerns related to patient selection and reference standards. Summarizing, ML models can effectively classify abnormal lung sounds using publicly available data sources. Nevertheless, inconsistent reporting and methodologies pose limitations to advancing the field, and therefore, public databases should adhere to standardized recording and labeling procedures.

BRACETS: Bimodal repository of auscultation coupled with electrical impedance thoracic signals

BACKGROUND AND OBJECTIVE

Respiratory diseases are among the most significant causes of morbidity and mortality worldwide, causing substantial strain on society and health systems. Over the last few decades, there has been increasing interest in the automatic analysis of respiratory sounds and electrical impedance tomography (EIT). Nevertheless, no publicly available databases with both respiratory sound and EIT data are available.

METHODS

In this work, we have assembled the first open-access bimodal database focusing on the differential diagnosis of respiratory diseases (BRACETS: Bimodal Repository of Auscultation Coupled with Electrical Impedance Thoracic Signals). It includes simultaneous recordings of single and multi-channel respiratory sounds and EIT. Furthermore, we have proposed several machine learning-based baseline systems for automatically classifying respiratory diseases in six distinct evaluation tasks using respiratory sound and EIT (A1, A2, A3, B1, B2, B3). These tasks included classifying respiratory diseases at sample and subject levels. The performance of the classification models was evaluated using a 5-fold cross-validation scheme (with subject isolation between folds).

RESULTS

The resulting database consists of 1097 respiratory sounds and 795 EIT recordings acquired from 78 adult subjects in two countries (Portugal and Greece). In the task of automatically classifying respiratory diseases, the baseline classification models have achieved the following average balanced accuracy: Task A1 - 77.9±13.1%; Task A2 - 51.6±9.7%; Task A3 - 38.6±13.1%; Task B1 - 90.0±22.4%; Task B2 - 61.4±11.8%; Task B3 - 50.8±10.6%.

CONCLUSION

The creation of this database and its public release will aid the research community in developing automated methodologies to assess and monitor respiratory function, and it might serve as a benchmark in the field of digital medicine for managing respiratory diseases. Moreover, it could pave the way for creating multi-modal robust approaches for that same purpose.

Deep learning-based lung sound analysis for intelligent stethoscope

Auscultation is crucial for the diagnosis of respiratory system diseases. However, traditional stethoscopes have inherent limitations, such as inter-listener variability and subjectivity, and they cannot record respiratory sounds for offline/retrospective diagnosis or remote prescriptions in telemedicine. The emergence of digital stethoscopes has overcome these limitations by allowing physicians to store and share respiratory sounds for consultation and education. On this basis, machine learning, particularly deep learning, enables the fully-automatic analysis of lung sounds that may pave the way for intelligent stethoscopes. This review thus aims to provide a comprehensive overview of deep learning algorithms used for lung sound analysis to emphasize the significance of artificial intelligence (AI) in this field. We focus on each component of deep learning-based lung sound analysis systems, including the task categories, public datasets, denoising methods, and, most importantly, existing deep learning methods, i.e., the state-of-the-art approaches to convert lung sounds into two-dimensional (2D) spectrograms and use convolutional neural networks for the end-to-end recognition of respiratory diseases or abnormal lung sounds. Additionally, this review highlights current challenges in this field, including the variety of devices, noise sensitivity, and poor interpretability of deep models. To address the poor reproducibility and variety of deep learning in this field, this review also provides a scalable and flexible open-source framework that aims to standardize the algorithmic workflow and provide a solid basis for replication and future extension: https://github.com/contactless-healthcare/Deep-Learning-for-Lung-Sound-Analysis .

Lung anomaly detection from respiratory sound database (sound signals)

Chest or upper body auscultation has long been considered a useful part of the physical examination going back to the time of Hippocrates. However, it did not become a prevalent practice until the invention of the stethoscope by Rene Laennec in 1816, which made the practice suitable and hygienic. Pulmonary disease is a kind of sickness that affects the lungs and various parts of the respiratory system. Lung diseases are the third largest cause of death in the world. According to the World Health Organization (WHO), the five major respiratory diseases, namely chronic obstructive pulmonary disease (COPD), tuberculosis, acute lower respiratory tract infection (LRTI), asthma, and lung cancer, cause the death of more than 3 million people each year worldwide. Respiratory sounds disclose significant information regarding the lungs of patients. Numerous methods are developed for analyzing the lung sounds. However, clinical approaches require qualified pulmonologists to diagnose such kind of signals appropriately and are also time consuming. Hence, an efficient Fractional Water Cycle Swarm Optimizer-based Deep Residual Network (Fr-WCSO-based DRN) is developed in this research for detecting the pulmonary abnormalities using respiratory sounds signals. The proposed Fr-WCSO is newly designed by the incorporation of Fractional Calculus (FC) and Water Cycle Swarm Optimizer WCSO. Meanwhile, WCSO is the combination of Water Cycle Algorithm (WCA) with Competitive Swarm Optimizer (CSO). The respiratory input sound signals are pre-processed and the important features needed for the further processing are effectively extracted. With the extracted features, data augmentation is carried out for minimizing the over fitting issues for improving the overall detection performance. Once data augmentation is done, feature selection is performed using proposed Fr-WCSO algorithm. Finally, pulmonary abnormality detection is performed using DRN where the training procedure of DRN is performed using the developed Fr-WCSO algorithm. The developed method achieved superior performance by considering the evaluation measures, namely True Positive Rate (TPR), True Negative Rate (TNR) and testing accuracy with the values of 0.963(96.3%), 0.932,(93.2%) and 0.948(94.8%), respectively.

An Optimized Federated Learning Approach with the Data-Sharing Function to the Analysis of Cardiothoracic Time-Series Signals

Machine/deep learning has been widely used for big data analysis in the field of healthcare, but it is still a question to ensure both computation efficiency and data security/confidentiality for the protection of private information. Referring to the data-sharing function of the federated learning (FedL) model, we propose an optimized data-sharing FedL (DSFedL) framework via a data-sharing hub by evaluating an accuracy-privacy loss function. When applied to the derived non-identically and independently distributed (nonIID) datasets simulated from three open-source cardiothoracic databases (i.e., ICBHI, Coswara COVID-19, MIT-BIH Arrhythmia), our optimized DSFedL works efficiently and the results show an optimal outcome of both the accuracy/efficiency and data security/confidentiality management.Clinical Relevance-This provides a proof of concept for using DSFedL in clinical applications, particularly in those settings that require data confidentiality control.

A Low-Cost AI-Empowered Stethoscope and a Lightweight Model for Detecting Cardiac and Respiratory Diseases from Lung and Heart Auscultation Sounds

Cardiac and respiratory diseases are the primary causes of health problems. If we can automate anomalous heart and lung sound diagnosis, we can improve the early detection of disease and enable the screening of a wider population than possible with manual screening. We propose a lightweight yet powerful model for simultaneous lung and heart sound diagnosis, which is deployable in an embedded low-cost device and is valuable in remote areas or developing countries where Internet access may not be available. We trained and tested the proposed model with the ICBHI and the Yaseen datasets. The experimental results showed that our 11-class prediction model could achieve 99.94% accuracy, 99.84% precision, 99.89% specificity, 99.66% sensitivity, and 99.72% F1 score. We designed a digital stethoscope (around USD 5) and connected it to a low-cost, single-board-computer Raspberry Pi Zero 2W (around USD 20), on which our pretrained model can be smoothly run. This AI-empowered digital stethoscope is beneficial for anyone in the medical field, as it can automatically provide diagnostic results and produce digital audio records for further analysis.

An accurate deep learning model for wheezing in children using real world data

Auscultation is an important diagnostic method for lung diseases. However, it is a subjective modality and requires a high degree of expertise. To overcome this constraint, artificial intelligence models are being developed. However, these models require performance improvements and do not reflect the actual clinical situation. We aimed to develop an improved deep-learning model learning to detect wheezing in children, based on data from real clinical practice. In this prospective study, pediatric pulmonologists recorded and verified respiratory sounds in 76 pediatric patients who visited a university hospital in South Korea. In addition, structured data, such as sex, age, and auscultation location, were collected. Using our dataset, we implemented an optimal model by transforming it based on the convolutional neural network model. Finally, we proposed a model using a 34-layer residual network with the convolutional block attention module for audio data and multilayer perceptron layers for tabular data. The proposed model had an accuracy of 91.2%, area under the curve of 89.1%, precision of 94.4%, recall of 81%, and F1-score of 87.2%. The deep-learning model proposed had a high accuracy for detecting wheeze sounds. This high-performance model will be helpful for the accurate diagnosis of respiratory diseases in actual clinical practice.

SPRSound: Open-Source SJTU Paediatric Respiratory Sound Database

It has proved that the auscultation of respiratory sound has advantage in early respiratory diagnosis. Various methods have been raised to perform automatic respiratory sound analysis to reduce subjective diagnosis and physicians' workload. However, these methods highly rely on the quality of respiratory sound database. In this work, we have developed the first open-access paediatric respiratory sound database, SPRSound. The database consists of 2,683 records and 9,089 respiratory sound events from 292 participants. Accurate label is important to achieve a good prediction for adventitious respiratory sound classification problem. A custom-made sound label annotation software (SoundAnn) has been developed to perform sound editing, sound annotation, and quality assurance evaluation. A team of 11 experienced paediatric physicians is involved in the entire process to establish golden standard reference for the dataset. To verify the robustness and accuracy of the classification model, we have investigated the effects of different feature extraction methods and machine learning classifiers on the classification performance of our dataset. As such, we have achieved a score of 75.22%, 61.57%, 56.71%, and 37.84% for the four different classification challenges at the event level and record level.

Deep learning models for detecting respiratory pathologies from raw lung auscultation sounds

In recent years deep learning models improve the diagnosis performance of many diseases especially respiratory diseases. This paper will propose an evaluation for the performance of different deep learning models associated with the raw lung auscultation sounds in detecting respiratory pathologies to help in providing diagnostic of respiratory pathologies in digital recorded respiratory sounds. Also, we will find out the best deep learning model for this task. In this paper, three different deep learning models have been evaluated on non-augmented and augmented datasets, where two different datasets have been utilized to generate four different sub-datasets. The results show that all the proposed deep learning methods were successful and achieved high performance in classifying the raw lung sounds, the methods were applied on different datasets and used either augmentation or non-augmentation. Among all proposed deep learning models, the CNN–LSTM model was the best model in all datasets for both augmentation and non-augmentation cases. The accuracy of CNN–LSTM model using non-augmentation was 99.6%, 99.8%, 82.4%, and 99.4% for datasets 1, 2, 3, and 4, respectively, and using augmentation was 100%, 99.8%, 98.0%, and 99.5% for datasets 1, 2, 3, and 4, respectively. While the augmentation process successfully helps the deep learning models in enhancing their performance on the testing datasets with a notable value. Moreover, the hybrid model that combines both CNN and LSTM techniques performed better than models that are based only on one of these techniques, this mainly refers to the use of CNN for automatic deep features extraction from lung sound while LSTM is used for classification.

A lightweight hybrid deep learning system for cardiac valvular disease classification

Cardiovascular diseases (CVDs) are a prominent cause of death globally. The introduction of medical big data and Artificial Intelligence (AI) technology encouraged the effort to develop and deploy deep learning models for distinguishing heart sound abnormalities. These systems employ phonocardiogram (PCG) signals because of their lack of sophistication and cost-effectiveness. Automated and early diagnosis of cardiovascular diseases (CVDs) helps alleviate deadly complications. In this research, a cardiac diagnostic system that combined CNN and LSTM components was developed, it uses phonocardiogram (PCG) signals, and utilizes either augmented or non-augmented datasets. The proposed model discriminates five heart valvular conditions, namely normal, Aortic Stenosis (AS), Mitral Regurgitation (MR), Mitral Stenosis (MS), and Mitral Valve Prolapse (MVP). The findings demonstrate that the suggested end-to-end architecture yields outstanding performance concerning all important evaluation metrics. For the five classes problem using the open heart sound dataset, accuracy was 98.5%, F1-score was 98.501%, and Area Under the Curve (AUC) was 0.9978 for the non-augmented dataset and accuracy was 99.87%, F1-score was 99.87%, and AUC was 0.9985 for the augmented dataset. Model performance was further evaluated using the PhysioNet/Computing in Cardiology 2016 challenge dataset, for the two classes problem, accuracy was 93.76%, F1-score was 85.59%, and AUC was 0.9505. The achieved results show that the proposed system outperforms all previous works that use the same audio signal databases. In the future, the findings will help build a multimodal structure that uses both PCG and ECG signals.

Data augmentation using Variational Autoencoders for improvement of respiratory disease classification

Computerized auscultation of lung sounds is gaining importance today with the availability of lung sounds and its potential in overcoming the limitations of traditional diagnosis methods for respiratory diseases. The publicly available ICBHI respiratory sounds database is severely imbalanced, making it difficult for a deep learning model to generalize and provide reliable results. This work aims to synthesize respiratory sounds of various categories using variants of Variational Autoencoders like Multilayer Perceptron VAE (MLP-VAE), Convolutional VAE (CVAE) Conditional VAE and compare the influence of augmenting the imbalanced dataset on the performance of various lung sound classification models. We evaluated the quality of the synthetic respiratory sounds’ quality using metrics such as Fréchet Audio Distance (FAD), Cross-Correlation and Mel Cepstral Distortion. Our results showed that MLP-VAE achieved an average FAD of 12.42 over all classes, whereas Convolutional VAE and Conditional CVAE achieved an average FAD of 11.58 and 11.64 for all classes, respectively. A significant improvement in the classification performance metrics was observed upon augmenting the imbalanced dataset for certain minority classes and marginal improvement for the other classes. Hence, our work shows that deep learning-based lung sound classification models are not only a promising solution over traditional methods but can also achieve a significant performance boost upon augmenting an imbalanced training set.

Particle Swarm Optimized Fuzzy CNN With Quantitative Feature Fusion for Ultrasound Image Quality Identification

Inherently ultrasound images are susceptible to noise which leads to several image quality issues. Hence, rating of an image's quality is crucial since diagnosing diseases requires accurate and high-quality ultrasound images. This research presents an intelligent architecture to rate the quality of ultrasound images. The formulated image quality recognition approach fuses feature from a Fuzzy convolutional neural network (fuzzy CNN) and a handcrafted feature extraction method. We implement the fuzzy layer in between the last max pooling and the fully connected layer of the multiple state-of-the-art CNN models to handle the uncertainty of information. Moreover, the fuzzy CNN uses Particle swarm optimization (PSO) as an optimizer. In addition, a novel Quantitative feature extraction machine (QFEM) extracts hand-crafted features from ultrasound images. Next, the proposed method uses different classifiers to predict the image quality. The classifiers categories ultrasound images into four types (normal, noisy, blurry, and distorted) instead of binary classification into good or poor-quality images. The results of the proposed method exhibit a significant performance in accuracy (99.62%), precision (99.62%), recall (99.61%), and f1-score (99.61%). This method will assist a physician in automatically rating informative ultrasound images with steadfast operation in real-time medical diagnosis.

A comparative study of the spectrogram, scalogram, melspectrogram and gammatonegram time-frequency representations for the classification of lung sounds using the ICBHI database based on CNNs

In lung sound classification using deep learning, many studies have considered the use of short-time Fourier transform (STFT) as the most commonly used 2D representation of the input data. Consequently, STFT has been widely used as an analytical tool, but other versions of the representation have also been developed. This study aims to evaluate and compare the performance of the spectrogram, scalogram, melspectrogram and gammatonegram representations, and provide comparative information to users regarding the suitability of these time-frequency (TF) techniques in lung sound classification. Lung sound signals used in this study were obtained from the ICBHI 2017 respiratory sound database. These lung sound recordings were converted into images of spectrogram, scalogram, melspectrogram and gammatonegram TF representations respectively. The four types of images were fed separately into the VGG16, ResNet-50 and AlexNet deep-learning architectures. Network performances were analyzed and compared based on accuracy, precision, recall and F1-score. The results of the analysis on the performance of the four representations using these three commonly used CNN deep-learning networks indicate that the generated gammatonegram and scalogram TF images coupled with ResNet-50 achieved maximum classification accuracies.

Contribution of Synthetic Data Generation towards an Improved Patient Stratification in Palliative Care

AI model development for synthetic data generation to improve Machine Learning (ML) methodologies is an integral part of research in Computer Science and is currently being transferred to related medical fields, such as Systems Medicine and Medical Informatics. In general, the idea of personalized decision-making support based on patient data has driven the motivation of researchers in the medical domain for more than a decade, but the overall sparsity and scarcity of data are still major limitations. This is in contrast to currently applied technology that allows us to generate and analyze patient data in diverse forms, such as tabular data on health records, medical images, genomics data, or even audio and video. One solution arising to overcome these data limitations in relation to medical records is the synthetic generation of tabular data based on real world data. Consequently, ML-assisted decision-support can be interpreted more conveniently, using more relevant patient data at hand. At a methodological level, several state-of-the-art ML algorithms generate and derive decisions from such data. However, there remain key issues that hinder a broad practical implementation in real-life clinical settings. In this review, we will give for the first time insights towards current perspectives and potential impacts of using synthetic data generation in palliative care screening because it is a challenging prime example of highly individualized, sparsely available patient information. Taken together, the reader will obtain initial starting points and suitable solutions relevant for generating and using synthetic data for ML-based screenings in palliative care and beyond.

A Progressively Expanded Database for Automated Lung Sound Analysis: An Update

We previously established an open-access lung sound database, HF_Lung_V1, and developed deep learning models for inhalation, exhalation, continuous adventitious sound (CAS), and discontinuous adventitious sound (DAS) detection. The amount of data used for training contributes to model accuracy. In this study, we collected larger quantities of data to further improve model performance and explored issues of noisy labels and overlapping sounds. HF_Lung_V1 was expanded to HF_Lung_V2 with a 1.43× increase in the number of audio files. Convolutional neural network–bidirectional gated recurrent unit network models were trained separately using the HF_Lung_V1 (V1_Train) and HF_Lung_V2 (V2_Train) training sets. These were tested using the HF_Lung_V1 (V1_Test) and HF_Lung_V2 (V2_Test) test sets, respectively. Segment and event detection performance was evaluated. Label quality was assessed. Overlap ratios were computed between inhalation, exhalation, CAS, and DAS labels. The model trained using V2_Train exhibited improved performance in inhalation, exhalation, CAS, and DAS detection on both V1_Test and V2_Test. Poor CAS detection was attributed to the quality of CAS labels. DAS detection was strongly influenced by the overlapping of DAS with inhalation and exhalation. In conclusion, collecting greater quantities of lung sound data is vital for developing more accurate lung sound analysis models.

A Deep Ensemble Neural Network with Attention Mechanisms for Lung Abnormality Classification Using Audio Inputs

Medical audio classification for lung abnormality diagnosis is a challenging problem owing to comparatively unstructured audio signals present in the respiratory sound clips. To tackle such challenges, we propose an ensemble model by incorporating diverse deep neural networks with attention mechanisms for undertaking lung abnormality and COVID-19 diagnosis using respiratory, speech, and coughing audio inputs. Specifically, four base deep networks are proposed, which include attention-based Convolutional Recurrent Neural Network (A-CRNN), attention-based bidirectional Long Short-Term Memory (A-BiLSTM), attention-based bidirectional Gated Recurrent Unit (A-BiGRU), as well as Convolutional Neural Network (CNN). A Particle Swarm Optimization (PSO) algorithm is used to optimize the training parameters of each network. An ensemble mechanism is used to integrate the outputs of these base networks by averaging the probability predictions of each class. Evaluated using respiratory ICBHI, Coswara breathing, speech, and cough datasets, as well as a combination of ICBHI and Coswara breathing databases, our ensemble model and base networks achieve ICBHI scores ranging from 0.920 to 0.9766. Most importantly, the empirical results indicate that a positive COVID-19 diagnosis can be distinguished to a high degree from other more common respiratory diseases using audio recordings, based on the combined ICBHI and Coswara breathing datasets.

Explainable machine learning for precise fatigue crack tip detection

Data-driven models based on deep learning have led to tremendous breakthroughs in classical computer vision tasks and have recently made their way into natural sciences. However, the absence of domain knowledge in their inherent design significantly hinders the understanding and acceptance of these models. Nevertheless, explainability is crucial to justify the use of deep learning tools in safety-relevant applications such as aircraft component design, service and inspection. In this work, we train convolutional neural networks for crack tip detection in fatigue crack growth experiments using full-field displacement data obtained by digital image correlation. For this, we introduce the novel architecture ParallelNets—a network which combines segmentation and regression of the crack tip coordinates—and compare it with a classical U-Net-based architecture. Aiming for explainability, we use the Grad-CAM interpretability method to visualize the neural attention of several models. Attention heatmaps show that ParallelNets is able to focus on physically relevant areas like the crack tip field, which explains its superior performance in terms of accuracy, robustness, and stability.

Improvise approach for respiratory pathologies classification with multilayer convolutional neural networks

Every respiratory-related checkup includes audio samples collected from the individual, collected through different tools (sonograph, stethoscope). This audio is analyzed to identify pathology, which requires time and effort. The research work proposed in this paper aims at easing the task with deep learning by the diagnosis of lung-related pathologies using Convolutional Neural Network (CNN) with the help of transformed features from the audio samples. International Conference on Biomedical and Health Informatics (ICBHI) corpus dataset was used for lung sound. Here a novel approach is proposed to pre-process the data and pass it through a newly proposed CNN architecture. The combination of pre-processing steps MFCC, Melspectrogram, and Chroma CENS with CNN improvise the performance of the proposed system, which helps to make an accurate diagnosis of lung sounds. The comparative analysis shows how the proposed approach performs better with previous state-of-the-art research approaches. It also shows that there is no need for a wheeze or a crackle to be present in the lung sound to carry out the classification of respiratory pathologies.

The coming era of a new auscultation system for analyzing respiratory sounds

Auscultation with stethoscope has been an essential tool for diagnosing the patients with respiratory disease. Although auscultation is non-invasive, rapid, and inexpensive, it has intrinsic limitations such as inter-listener variability and subjectivity, and the examination must be performed face-to-face. Conventional stethoscope could not record the respiratory sounds, so it was impossible to share the sounds. Recent innovative digital stethoscopes have overcome the limitations and enabled clinicians to store and share the sounds for education and discussion. In particular, the recordable stethoscope made it possible to analyze breathing sounds using artificial intelligence, especially based on neural network. Deep learning-based analysis with an automatic feature extractor and convoluted neural network classifier has been applied for the accurate analysis of respiratory sounds. In addition, the current advances in battery technology, embedded processors with low power consumption, and integrated sensors make possible the development of wearable and wireless stethoscopes, which can help to examine patients living in areas of a shortage of doctors or those who need isolation. There are still challenges to overcome, such as the analysis of complex and mixed respiratory sounds and noise filtering, but continuous research and technological development will facilitate the transition to a new era of a wearable and smart stethoscope.

Recognition of pulmonary diseases from lung sounds using convolutional neural networks and long short-term memory

In this paper, a study is conducted to explore the ability of deep learning in recognizing pulmonary diseases from electronically recorded lung sounds. The selected data-set included a total of 103 patients obtained from locally recorded stethoscope lung sounds acquired at King Abdullah University Hospital, Jordan University of Science and Technology, Jordan. In addition, 110 patients data were added to the data-set from the Int. Conf. on Biomedical Health Informatics publicly available challenge database. Initially, all signals were checked to have a sampling frequency of 4 kHz and segmented into 5 s segments. Then, several preprocessing steps were undertaken to ensure smoother and less noisy signals. These steps included wavelet smoothing, displacement artifact removal, and z-score normalization. The deep learning network architecture consisted of two stages; convolutional neural networks and bidirectional long short-term memory units. The training of the model was evaluated based on a k-fold cross-validation scheme of tenfolds using several performance evaluation metrics including Cohen's kappa, accuracy, sensitivity, specificity, precision, and F1-score. The developed algorithm achieved the highest average accuracy of 99.62% with a precision of 98.85% in classifying patients based on the pulmonary disease types using CNN + BDLSTM. Furthermore, a total agreement of 98.26% was obtained between the predictions and original classes within the training scheme. This study paves the way towards implementing deep learning models in clinical settings to assist clinicians in decision making related to the recognition of pulmonary diseases.

SUPPLEMENTARY INFORMATION

The online version supplementary material available at 10.1007/s12652-021-03184-y.

Review of Artificial Intelligence techniques in Chronic Obstructive Lung Disease

BACKGROUND

Artificial Intelligence (AI) has proven to be an invaluable asset in the healthcare domain, where massive amounts of data are produced. Chronic Obstructive Pulmonary Disease (COPD) is a heterogeneous chronic condition with multiscale manifestations and complex interactions that represents an ideal target for AI.

OBJECTIVE

The aim of this review article is to appraise the adoption of AI in COPD research, and more specifically its applications to date along with reported results, potential challenges and future prospects.

METHODS

We performed a review of the literature from PubMed and DBLP and assembled studies published up to 2020, yielding 156 articles relevant to the scope of this review.

RESULTS

The resulting articles were assessed and organized into four basic contextual categories, namely: i) COPD diagnosis, ii) COPD prognosis, iii) Patient classification, iv) COPD management, and subsequently presented in an orderly manner based on a set of qualitative and quantitative criteria.

CONCLUSIONS

We observed considerable acceleration of research activity utilizing AI techniques in COPD research, especially in the last couple of years, nevertheless, the massive production of large and complex data in COPD calls for broader adoption of AI and more advanced techniques.

LungAttn: advanced lung sound classification using attention mechanism with dual TQWT and triple STFT spectrogram

Objective. Auscultation of lung sound plays an important role in the early diagnosis of lung diseases. This work aims to develop an automated adventitious lung sound detection method to reduce the workload of physicians.Approach. We propose a deep learning architecture, LungAttn, which incorporates augmented attention convolution into ResNet block to improve the classification accuracy of lung sound. We adopt a feature extraction method based on dual tunableQ-factor wavelet transform and triple short-time Fourier transform to obtain a multi-channel spectrogram. Mixup method is introduced to augment adventitious lung sound recordings to address the imbalance dataset problem.Main results. Based on the ICBHI 2017 challenge dataset, we implement our framework and compare with the state-of-the-art works. Experimental results show that LungAttn has achieved theSensitivity, S_e,Specificity, S_pandScoreof 36.36%, 71.44% and 53.90%, respectively. Of which, our work has improved theScoreby 1.69% compared to the state-of-the-art models based on the official ICBHI 2017 dataset splitting method.Significance. Multi-channel spectrogram based on different oscillatory behavior of adventitious lung sound provides necessary information of lung sound recordings. Attention mechanism is introduced to lung sound classification methods and has proved to be effective. The proposed LungAttn model can potentially improve the speed and accuracy of lung sound classification in clinical practice.

Comprehensive Analysis System for Automated Respiratory Cycle Segmentation and Crackle Peak Detection

Digital auscultation is a well-known method for assessing lung sounds, but remains a subjective process in typical practice, relying on the human interpretation. Several methods have been presented for detecting or analyzing crackles but are limited in their real-world application because few have been integrated into comprehensive systems or validated on non-ideal data. This work details a complete signal analysis methodology for analyzing crackles in challenging recordings. The procedure comprises five sequential processing blocks: (1) motion artifact detection, (2) deep learning denoising network, (3) respiratory cycle segmentation, (4) separation of discontinuous adventitious sounds from vesicular sounds, and (5) crackle peak detection. This system uses a collection of new methods and robustness-focused improvements on previous methods to analyze respiratory cycles and crackles therein. To validate the accuracy, the system is tested on a database of 1000 simulated lung sounds with varying levels of motion artifacts, ambient noise, cycle lengths and crackle intensities, in which ground truths are exactly known. The system performs with average F-score of 91.07% for detecting motion artifacts and 94.43% for respiratory cycle extraction, and an overall F-score of 94.08% for detecting the locations of individual crackles. The process also successfully detects healthy recordings. Preliminary validation is also presented on a small set of 20 patient recordings, for which the system performs comparably. These methods provide quantifiable analysis of respiratory sounds to enable clinicians to distinguish between types of crackles, their timing within the respiratory cycle, and the level of occurrence. Crackles are one of the most common abnormal lung sounds, presenting in multiple cardiorespiratory diseases. These features will contribute to a better understanding of disease severity and progression in an objective, simple and non-invasive way.

A Lightweight CNN Model for Detecting Respiratory Diseases From Lung Auscultation Sounds Using EMD-CWT-Based Hybrid Scalogram

Listening to lung sounds through auscultation is vital in examining the respiratory system for abnormalities. Automated analysis of lung auscultation sounds can be beneficial to the health systems in low-resource settings where there is a lack of skilled physicians. In this work, we propose a lightweight convolutional neural network (CNN) architecture to classify respiratory diseases from individual breath cycles using hybrid scalogram-based features of lung sounds. The proposed feature-set utilizes the empirical mode decomposition (EMD) and the continuous wavelet transform (CWT). The performance of the proposed scheme is studied using a patient independent train-validation-test set from the publicly available ICBHI 2017 lung sound dataset. Employing the proposed framework, weighted accuracy scores of 98.92% for three-class chronic classification and 98.70% for six-class pathological classification are achieved, which outperform well-known and much larger VGG16 in terms of accuracy by absolute margins of 1.10% and 1.11%, respectively. The proposed CNN model also outperforms other contemporary lightweight models while being computationally comparable.

The usage of deep neural network improves distinguishing COVID-19 from other suspected viral pneumonia by clinicians on chest CT: a real-world study

OBJECTIVES

Based on the current clinical routine, we aimed to develop a novel deep learning model to distinguish coronavirus disease 2019 (COVID-19) pneumonia from other types of pneumonia and validate it with a real-world dataset (RWD).

METHODS

A total of 563 chest CT scans of 380 patients (227/380 were diagnosed with COVID-19 pneumonia) from 5 hospitals were collected to train our deep learning (DL) model. Lung regions were extracted by U-net, then transformed and fed to pre-trained ResNet-50-based IDANNet (Identification and Analysis of New covid-19 Net) to produce a diagnostic probability. Fivefold cross-validation was employed to validate the application of our model. Another 318 scans of 316 patients (243/316 were diagnosed with COVID-19 pneumonia) from 2 other hospitals were enrolled prospectively as the RWDs to testify our DL model's performance and compared it with that from 3 experienced radiologists.

RESULTS

A three-dimensional DL model was successfully established. The diagnostic threshold to differentiate COVID-19 and non-COVID-19 pneumonia was 0.685 with an AUC of 0.906 (95% CI: 0.886-0.913) in the internal validation group. In the RWD cohort, our model achieved an AUC of 0.868 (95% CI: 0.851-0.876) with the sensitivity of 0.811 and the specificity of 0.822, non-inferior to the performance of 3 experienced radiologists, suggesting promising clinical practical usage.

CONCLUSIONS

The established DL model was able to achieve accurate identification of COVID-19 pneumonia from other suspected ones in the real-world situation, which could become a reliable tool in clinical routine.

KEY POINTS

• In an internal validation set, our DL model achieved the best performance to differentiate COVID-19 from non-COVID-19 pneumonia with a sensitivity of 0.836, a specificity of 0.800, and an AUC of 0.906 (95% CI: 0.886-0.913) when the threshold was set at 0.685. • In the prospective RWD cohort, our DL diagnostic model achieved a sensitivity of 0.811, a specificity of 0.822, and AUC of 0.868 (95% CI: 0.851-0.876), non-inferior to the performance of 3 experienced radiologists. • The attention heatmaps were fully generated by the model without additional manual annotation and the attention regions were highly aligned with the ROIs acquired by human radiologists for diagnosis.

Efficiently Classifying Lung Sounds through Depthwise Separable CNN Models with Fused STFT and MFCC Features

Lung sounds remain vital in clinical diagnosis as they reveal associations with pulmonary pathologies. With COVID-19 spreading across the world, it has become more pressing for medical professionals to better leverage artificial intelligence for faster and more accurate lung auscultation. This research aims to propose a feature engineering process that extracts the dedicated features for the depthwise separable convolution neural network (DS-CNN) to classify lung sounds accurately and efficiently. We extracted a total of three features for the shrunk DS-CNN model: the short-time Fourier-transformed (STFT) feature, the Mel-frequency cepstrum coefficient (MFCC) feature, and the fused features of these two. We observed that while DS-CNN models trained on either the STFT or the MFCC feature achieved an accuracy of 82.27% and 73.02%, respectively, fusing both features led to a higher accuracy of 85.74%. In addition, our method achieved 16 times higher inference speed on an edge device and only 0.45% less accuracy than RespireNet. This finding indicates that the fusion of the STFT and MFCC features and DS-CNN would be a model design for lightweight edge devices to achieve accurate AI-aided detection of lung diseases.

Diabetes detection using deep learning techniques with oversampling and feature augmentation

BACKGROUND AND OBJECTIVE

Diabetes is a chronic pathology which is affecting more and more people over the years. It gives rise to a large number of deaths each year. Furthermore, many people living with the disease do not realize the seriousness of their health status early enough. Late diagnosis brings about numerous health problems and a large number of deaths each year so the development of methods for the early diagnosis of this pathology is essential.

METHODS

In this paper, a pipeline based on deep learning techniques is proposed to predict diabetic people. It includes data augmentation using a variational autoencoder (VAE), feature augmentation using an sparse autoencoder (SAE) and a convolutional neural network for classification. Pima Indians Diabetes Database, which takes into account information on the patients such as the number of pregnancies, glucose or insulin level, blood pressure or age, has been evaluated.

RESULTS

A 92.31% of accuracy was obtained when CNN classifier is trained jointly the SAE for featuring augmentation over a well balanced dataset. This means an increment of 3.17% of accuracy with respect the state-of-the-art.

CONCLUSIONS

Using a full deep learning pipeline for data preprocessing and classification has demonstrate to be very promising in the diabetes detection field outperforming the state-of-the-art proposals.

Inspiratory crackles-early and late-revisited: identifying COPD by crackle characteristics

BACKGROUND

The significance of pulmonary crackles, by their timing during inspiration, was described by Nath and Capel in 1974, with early crackles associated with bronchial obstruction and late crackles with restrictive defects. Crackles are also described as 'fine' or 'coarse'. We aimed to evaluate the usefulness of crackle characteristics in the diagnosis of chronic obstructive pulmonary disease (COPD).

METHODS

In a population-based study, lung sounds were recorded at six auscultation sites and classified in participants aged 40 years or older. Inspiratory crackles were classified as 'early' or 'late and into the types' 'coarse' and 'fine' by two observers. A diagnosis of COPD was based on respiratory symptoms and forced expiratory volume in 1 s/forced inspiratory vital capacity below lower limit of normal, based on Global Lung Function Initiative 2012 reference. Associations between crackle characteristics and COPD were analysed by logistic regression. Kappa statistics was applied for evaluating interobserver agreement.

RESULTS

Of 3684 subjects included in the analysis, 52.9% were female, 50.1% were ≥65 years and 204 (5.5%) had COPD. Basal inspiratory crackles were heard in 306 participants by observer 1 and in 323 by observer 2. When heard bilaterally COPD could be predicted with ORs of 2.59 (95% CI 1.36 to 4.91) and 3.20 (95% CI 1.71 to 5.98), annotated by observer 1 and 2, respectively, adjusted for sex and age. If bilateral crackles were coarse the corresponding ORs were 2.65 (95% CI 1.28 to 5.49) and 3.67 (95% CI 1.58 to 8.52) and when heard early during inspiration the ORs were 6.88 (95% CI 2.59 to 18.29) and 7.63 (95%CI 3.73 to 15.62). The positive predictive value for COPD was 23% when early crackles were heard over one or both lungs. We observed higher kappa values when classifying timing than type.

CONCLUSIONS

'Early' inspiratory crackles predicted COPD more strongly than 'coarse' inspiratory crackles. Identification of early crackles at the lung bases should imply a strong attention to the possibility of COPD.

An Automated System for Classification of Chronic Obstructive Pulmonary Disease and Pneumonia Patients Using Lung Sound Analysis

Chronic obstructive pulmonary disease (COPD) and pneumonia are two of the few fatal lung diseases which share common adventitious lung sounds. Diagnosing the disease from lung sound analysis to design a noninvasive technique for telemedicine is a challenging task. A novel framework is presented to perform a diagnosis of COPD and Pneumonia via application of the signal processing and machine learning approach. This model will help the pulmonologist to accurately detect disease A and B. COPD, normal and pneumonia lung sound (LS) data from the ICBHI respiratory database is used in this research. The performance analysis is evidence of the improved performance of the quadratic discriminate classifier with an accuracy of 99.70% on selected fused features after experimentation. The fusion of time domain, cepstral, and spectral features are employed. Feature selection for fusion is performed through the back-elimination method whereas empirical mode decomposition (EMD) and discrete wavelet transform (DWT)-based techniques are used to denoise and segment the pulmonic signal. Class imbalance is catered with the implementation of the adaptive synthetic (ADASYN) sampling technique.