Recognition of normal–abnormal phonocardiographic signals using deep convolutional neural networks and mel-frequency spectral coefficients

Identifying pediatric heart murmurs and distinguishing innocent from pathologic using deep learning

OBJECTIVE

To develop a deep learning algorithm to perform multi-class classification of normal pediatric heart sounds, innocent murmurs, and pathologic murmurs.

METHODS

We prospectively enrolled children under age 18 being evaluated by the Division of Pediatric Cardiology. Parents provided consent for a deidentified recording of their child's heart sounds with a digital stethoscope. Innocent murmurs were validated by a pediatric cardiologist and pathologic murmurs were validated by echocardiogram. To augment our collection of normal heart sounds, we utilized a public database of pediatric heart sound recordings (Oliveira, 2022). We propose two novel approaches for this audio classification task. We train a vision transformer on either Markov transition field or Gramian angular field image representations of the frequency spectrum. We benchmark our results against a ResNet-50 CNN trained on spectrogram images.

RESULTS

Our final dataset consisted of 366 normal heart sounds, 175 innocent murmurs, and 216 pathologic murmurs. Innocent murmurs collected include Still's murmur, venous hum, and flow murmurs. Pathologic murmurs included ventricular septal defect, tetralogy of Fallot, aortic regurgitation, aortic stenosis, pulmonary stenosis, mitral regurgitation and stenosis, and tricuspid regurgitation. We find that the Vision Transformer consistently outperforms the ResNet-50 on all three image representations, and that the Gramian angular field is the superior image representation for pediatric heart sounds. We calculated a one-vs-rest multi-class ROC curve for each of the three classes. Our best model achieves an area under the curve (AUC) value of 0.92 ± 0.05, 0.83 ± 0.04, and 0.88 ± 0.04 for identifying normal heart sounds, innocent murmurs, and pathologic murmurs, respectively.

CONCLUSION

We present two novel methods for pediatric heart sound classification, which outperforms the current standard of using a convolutional neural network trained on spectrogram images. To our knowledge, we are the first to demonstrate multi-class classification of pediatric murmurs. Multiclass output affords a more explainable and interpretable model, which can facilitate further model improvement in the downstream model development cycle and enhance clinician trust and therefore adoption.

Artificial intelligence framework for heart disease classification from audio signals

As cardiovascular disorders are prevalent, there is a growing demand for reliable and precise diagnostic methods within this domain. Audio signal-based heart disease detection is a promising area of research that leverages sound signals generated by the heart to identify and diagnose cardiovascular disorders. Machine learning (ML) and deep learning (DL) techniques are pivotal in classifying and identifying heart disease from audio signals. This study investigates ML and DL techniques to detect heart disease by analyzing noisy sound signals. This study employed two subsets of datasets from the PASCAL CHALLENGE having real heart audios. The research process and visually depict signals using spectrograms and Mel-Frequency Cepstral Coefficients (MFCCs). We employ data augmentation to improve the model's performance by introducing synthetic noise to the heart sound signals. In addition, a feature ensembler is developed to integrate various audio feature extraction techniques. Several machine learning and deep learning classifiers are utilized for heart disease detection. Among the numerous models studied and previous study findings, the multilayer perceptron model performed best, with an accuracy rate of 95.65%. This study demonstrates the potential of this methodology in accurately detecting heart disease from sound signals. These findings present promising opportunities for enhancing medical diagnosis and patient care.

Markov-Based Neural Networks for Heart Sound Segmentation: Using Domain Knowledge in a Principled Way

This work considers the problem of segmenting heart sounds into their fundamental components. We unify statistical and data-driven solutions by introducing Markov-based Neural Networks (MNNs), a hybrid end-to-end framework that exploits Markov models as statistical inductive biases for an Artificial Neural Network (ANN) discriminator. We show that an MNN leveraging a simple one-dimensional Convolutional ANN significantly outperforms two recent purely data-driven solutions for this task in two publicly available datasets: PhysioNet 2016 (Sensitivity: 0.947 ±0.02; Positive Predictive Value : 0.937 ±0.025) and the CirCor DigiScope 2022 (Sensitivity: 0.950 ±0.008; Positive Predictive Value: 0.943 ±0.012). We also propose a novel gradient-based unsupervised learning algorithm that effectively makes the MNN adaptive to unseen datum sampled from unknown distributions. We perform a cross dataset analysis and show that an MNN pre-trained in the CirCor DigiScope 2022 can benefit from an average improvement of 3.90% Positive Predictive Value on unseen observations from the PhysioNet 2016 dataset using this method.

Heart Sound Classification Network Based on Convolution and Transformer

Electronic auscultation is vital for doctors to detect symptoms and signs of cardiovascular diseases (CVDs), significantly impacting human health. Although progress has been made in heart sound classification, most existing methods require precise segmentation and feature extraction of heart sound signals before classification. To address this, we introduce an innovative approach for heart sound classification. Our method, named Convolution and Transformer Encoder Neural Network (CTENN), simplifies preprocessing, automatically extracting features using a combination of a one-dimensional convolution (1D-Conv) module and a Transformer encoder. Experimental results showcase the superiority of our proposed method in both binary and multi-class tasks, achieving remarkable accuracies of 96.4%, 99.7%, and 95.7% across three distinct datasets compared with that of similar approaches. This advancement holds promise for enhancing CVD diagnosis and treatment.

Parameter-Efficient Densely Connected Dual Attention Network for Phonocardiogram Classification

Cardiac auscultation, exhibited by phonocardiogram (PCG), is a non-invasive and low-cost diagnostic method for cardiovascular diseases (CVDs). However, deploying it in practice is quite challenging, due to the inherent murmurs and a limited number of supervised samples in heart sound data. To solve these problems, not only heart sound analysis based on handcrafted features, but also computer-aided heart sound analysis based on deep learning have been extensively studied in recent years. Though with elaborate design, most of these methods still use additional pre-processing to improve classification performance, which heavily relies on time-consuming experienced engineering. In this article, we propose a parameter-efficient densely connected dual attention network (DDA) for heart sound classification. It combines two advantages simultaneously of the purely end-to-end architecture and enriched contextual representations of the self-attention mechanism. Specifically, the densely connected structure can automatically extract the information flow of heart sound features hierarchically. Alongside, improving contextual modeling capabilities, the dual attention mechanism adaptively aggregates local features with global dependencies via a self-attention mechanism, which captures the semantic interdependencies across position and channel axes respectively. Extensive experiments across stratified 10-fold cross-validation strongly evidence that our proposed DDA model surpasses current 1D deep models on the challenging Cinc2016 benchmark with significant computational efficiency.

Design of ear-contactless stethoscope and improvement in the performance of deep learning based on CNN to classify the heart sound

Cardiac-related disorders are rapidly growing throughout the world. Accurate classification of cardiovascular diseases is an important research topic in healthcare. During COVID-19, auscultating heart sounds was challenging as health workers and doctors wear protective clothing, and direct contact with patients can spread the outbreak. Thus, contactless auscultation of heart sound is necessary. In this paper, a low-cost ear contactless stethoscope is designed where auscultation is done with the help of a bluetooth-enabled micro speaker instead of an earpiece. The PCG recordings are further compared with other standard electronic stethoscopes like Littman 3 M. This work is made to improve the performance of deep learning-based classifiers like recurrent neural networks (RNN) and convolutional neural networks (CNN) for different valvular heart problems using tuning of hyperparameters like learning rate of optimizers, dropout rate, and hidden layer. Hyper-parameter tuning is used to optimize the performances of various deep learning models and their learning curves for real-time analysis. The acoustic, time, and frequency domain features are used in this research. The investigation is made on the heart sounds of normal and diseased patients available from the standard data repository to train the software models. The proposed CNN-based inception network model achieved an accuracy of 99.65 ± 0.06% on the test dataset with a sensitivity of 98.8  ± 0.05% and specificity of 98.2 ± 0.19%. The proposed hybrid CNN-RNN architecture attained 91.17 ± 0.03% accuracy on test data after hyperparameter optimization, whereas the LSTM-based RNN model achieved 82.32 ± 0.11% accuracy. Finally, the evaluated results were compared with machine learning algorithms, and the improved CNN-based Inception Net model is the most effective among others.

Detection of valvular heart diseases combining orthogonal non-negative matrix factorization and convolutional neural networks in PCG signals

BACKGROUND AND OBJECTIVE

Valvular heart disease (VHD) is associated with elevated mortality rates. Although transthoracic echocardiography (TTE) is the gold standard detection tool, phonocardiography (PCG) could be an alternative as it is a cost-effective and noninvasive method for cardiac auscultation. Many researchers have dedicated their efforts to improving the decision-making process and developing robust and precise approaches to assist physicians in providing reliable diagnoses of VHD.

METHODS

This research proposes a novel approach for the detection of anomalous valvular heart sounds from PCG signals. The proposed approach combines orthogonal non-negative matrix factorization (ONMF) and convolutional neural network (CNN) architectures in a three-stage cascade. The aim of the proposal is to improve the learning process by identifying the optimal ONMF temporal or spectral patterns for accurate detection. In the first stage, the time-frequency representation of the input PCG signal is computed. Next, band-pass filtering is performed to locate the spectral range that is most relevant for the presence of such cardiac abnormalities. In the second stage, the temporal and spectral cardiac structures are extracted using the ONMF approach. These structures are utilized in the third stage and fed into the CNN architecture to detect abnormal heart sounds.

RESULTS

Several state-of-the-art CNN architectures, such as LeNet5, AlexNet, ResNet50, VGG16 and GoogLeNet, have been evaluated to determine the effectiveness of using ONMF temporal features for VHD detection. The results reveal that the integration of ONMF temporal features with a CNN classifier significantly improve VHD detection. Specifically, the proposed approach achieves an accuracy improvement of approximately 45% when ONMF spectral features are used and 35% when time-frequency features from the short-time Fourier transform (STFT) spectrogram are used. Additionally, feeding ONMF temporal features into low-complexity CNN architectures yields competitive results comparable to those obtained with complex architectures.

CONCLUSIONS

The temporal structure factorized by ONMF plays a critical role in distinguishing between normal heart sounds and abnormal heart sounds since the repeatability of normal heart cycles is disrupted by the presence of cardiac abnormalities. Consequently, the results highlight the importance of appropriate input data representation in the learning process of CNN models in the biomedical field of valvular heart sound detection.

Beyond Heart Murmur Detection: Automatic Murmur Grading From Phonocardiogram

OBJECTIVE

Murmurs are abnormal heart sounds, identified by experts through cardiac auscultation. The murmur grade, a quantitative measure of the murmur intensity, is strongly correlated with the patient's clinical condition. This work aims to estimate each patient's murmur grade (i.e., absent, soft, loud) from multiple auscultation location phonocardiograms (PCGs) of a large population of pediatric patients from a low-resource rural area.

METHODS

The Mel spectrogram representation of each PCG recording is given to an ensemble of 15 convolutional residual neural networks with channel-wise attention mechanisms to classify each PCG recording. The final murmur grade for each patient is derived based on the proposed decision rule and considering all estimated labels for available recordings. The proposed method is cross-validated on a dataset consisting of 3456 PCG recordings from 1007 patients using a stratified ten-fold cross-validation. Additionally, the method was tested on a hidden test set comprised of 1538 PCG recordings from 442 patients.

RESULTS

The overall cross-validation performances for patient-level murmur gradings are 86.3% and 81.6% in terms of the unweighted average of sensitivities and F1-scores, respectively. The sensitivities (and F1-scores) for absent, soft, and loud murmurs are 90.7% (93.6%), 75.8% (66.8%), and 92.3% (84.2%), respectively. On the test set, the algorithm achieves an unweighted average of sensitivities of 80.4% and an F1-score of 75.8%.

CONCLUSIONS

This study provides a potential approach for algorithmic pre-screening in low-resource settings with relatively high expert screening costs.

SIGNIFICANCE

The proposed method represents a significant step beyond detection of murmurs, providing characterization of intensity, which may provide an enhanced classification of clinical outcomes.

Classifying Heart-Sound Signals Based on CNN Trained on MelSpectrum and Log-MelSpectrum Features

The intelligent classification of heart-sound signals can assist clinicians in the rapid diagnosis of cardiovascular diseases. Mel-frequency cepstral coefficients (MelSpectrums) and log Mel-frequency cepstral coefficients (Log-MelSpectrums) based on a short-time Fourier transform (STFT) can represent the temporal and spectral structures of original heart-sound signals. Recently, various systems based on convolutional neural networks (CNNs) trained on the MelSpectrum and Log-MelSpectrum of segmental heart-sound frames that outperform systems using handcrafted features have been presented and classified heart-sound signals accurately. However, there is no a priori evidence of the best input representation for classifying heart sounds when using CNN models. Therefore, in this study, the MelSpectrum and Log-MelSpectrum features of heart-sound signals combined with a mathematical model of cardiac-sound acquisition were analysed theoretically. Both the experimental results and theoretical analysis demonstrated that the Log-MelSpectrum features can reduce the classification difference between domains and improve the performance of CNNs for heart-sound classification.

An efficient and robust Phonocardiography (PCG)-based Valvular Heart Diseases (VHD) detection framework using Vision Transformer (ViT)

BACKGROUND AND OBJECTIVES

Valvular heart diseases (VHDs) are one of the dominant causes of cardiovascular abnormalities that have been associated with high mortality rates globally. Rapid and accurate diagnosis of the early stage of VHD based on cardiac phonocardiogram (PCG) signal is critical that allows for optimum medication and reduction of mortality rate.

METHODS

To this end, the current study proposes novel deep learning (DL)-based high-performance VHD detection frameworks that are relatively simpler in terms of network structures, yet effective for accurately detecting multiple VHDs. We present three different frameworks considering both 1D and 2D PCG raw signals. For 1D PCG, Mel frequency cepstral coefficients (MFCC) and linear prediction cepstral coefficients (LPCC) features, whereas, for 2D PCG, various deep convolutional neural networks (D-CNNs) features are extracted. Additionally, nature/bio-inspired algorithms (NIA/BIA) including particle swarm optimization (PSO) and genetic algorithm (GA) have been utilized for automatic and efficient feature selection directly from the raw PCG signal. To further improve the performance of the classifier, vision transformer (ViT) has been implemented levering the self-attention mechanism on the time frequency representation (TFR) of 2D PCG signal. Our extensive study presents a comparative performance analysis and the scope of enhancement for the combination of different descriptors, classifiers, and feature selection algorithms.

MAIN RESULTS

Among all classifiers, ViT provides the best performance by achieving mean average accuracy A_cc of 99.90 % and F1-score of 99.95 % outperforming current state-of-the-art VHD classification models.

CONCLUSIONS

The present research provides a robust and efficient DL-based end-to-end PCG signal classification framework for designing a automated high-performance VHD diagnosis system.

Cardiac anomaly detection considering an additive noise and convolutional distortion model of heart sound recordings

Cardiac auscultation is an essential point-of-care method used for the early diagnosis of heart diseases. Automatic analysis of heart sounds for abnormality detection is faced with the challenges of additive noise and sensor-dependent degradation. This paper aims to develop methods to address the cardiac abnormality detection problem when both of these components are present in the cardiac auscultation sound. We first mathematically analyze the effect of additive noise and convolutional distortion on short-term mel-filterbank energy-based features and a Convolutional Neural Network (CNN) layer. Based on the analysis, we propose a combination of linear and logarithmic spectrogram-image features. These 2D features are provided as input to a residual CNN network (ResNet) for heart sound abnormality detection. Experimental validation is performed first on an open-access, multiclass heart sound dataset where we analyzed the effect of additive noise by mixing lung sound noise with the recordings. In noisy conditions, the proposed method outperforms one of the best-performing methods in the literature achieving an Macc (mean of sensitivity and specificity) of 89.55% and an average F-1 score of 82.96%, respectively, when averaged over all noise levels. Next, we perform heart sound abnormality detection (binary classification) experiments on the 2016 Physionet/CinC Challenge dataset that involves noisy recordings obtained from multiple stethoscope sensors. The proposed method achieves significantly improved results compared to the conventional approaches on this dataset, in the presence of both additive noise and channel distortion, with an area under the ROC (receiver operating characteristics) curve (AUC) of 91.36%, F-1 score of 84.09%, and Macc of 85.08%. We also show that the proposed method shows the best mean accuracy across different source domains, including stethoscope and noise variability, demonstrating its effectiveness in different recording conditions. The proposed combination of linear and logarithmic features along with the ResNet classifier effectively minimizes the impact of background noise and sensor variability for classifying phonocardiogram (PCG) signals. The method thus paves the way toward developing computer-aided cardiac auscultation systems in noisy environments using low-cost stethoscopes.

On the analysis of data augmentation methods for spectral imaged based heart sound classification using convolutional neural networks

Background

The application of machine learning to cardiac auscultation has the potential to improve the accuracy and efficiency of both routine and point-of-care screenings. The use of convolutional neural networks (CNN) on heart sound spectrograms in particular has defined state-of-the-art performance. However, the relative paucity of patient data remains a significant barrier to creating models that can adapt to a wide range of potential variability. To that end, we examined a CNN model’s performance on automated heart sound classification, before and after various forms of data augmentation, and aimed to identify the most optimal augmentation methods for cardiac spectrogram analysis.

Results

We built a standard CNN model to classify cardiac sound recordings as either normal or abnormal. The baseline control model achieved a PR AUC of 0.763 ± 0.047. Among the single data augmentation techniques explored, horizontal flipping of the spectrogram image improved the model performance the most, with a PR AUC of 0.819 ± 0.044. Principal component analysis color augmentation (PCA) and perturbations of saturation-value (SV) of the hue-saturation-value (HSV) color scale achieved a PR AUC of 0.779 ± 045 and 0.784 ± 0.037, respectively. Time and frequency masking resulted in a PR AUC of 0.772 ± 0.050. Pitch shifting, time stretching and compressing, noise injection, vertical flipping, and applying random color filters negatively impacted model performance. Concatenating the best performing data augmentation technique (horizontal flip) with PCA and SV perturbations improved model performance.

Conclusion

Data augmentation can improve classification accuracy by expanding and diversifying the dataset, which protects against overfitting to random variance. However, data augmentation is necessarily domain specific. For example, methods like noise injection have found success in other areas of automated sound classification, but in the context of cardiac sound analysis, noise injection can mimic the presence of murmurs and worsen model performance. Thus, care should be taken to ensure clinically appropriate forms of data augmentation to avoid negatively impacting model performance.

CNN-Based Heart Sound Classification with an Imbalance-Compensating Weighted Loss Function

Heart sound auscultation is an effective method for early-stage diagnosis of heart disease. The application of deep neural networks is gaining increasing attention in automated heart sound classification. This paper proposes deep Convolutional Neural Networks (CNNs) to classify normal/abnormal heart sounds, which takes two-dimensional Mel-scale features as input, including Mel frequency cepstral coefficients (MFCCs) and the Log Mel spectrum. We employ two weighted loss functions during the training to mitigate the class imbalance issue. The model was developed on the public PhysioNet/Computing in Cardiology Challenge (CinC) 2016 heart sound database. On the considered test set, the proposed model with Log Mel spectrum as features achieves an Unweighted Average Recall (UAR) of 89.6%, with sensitivity and specificity being 89.5% and 89.7% respectively. This work proposes a CNN-based model to enable automated heart sound classification, which can provide auxiliary assistance for heart auscultation and has the potential to screen for heart pathologies in clinical applications at a relatively low cost.

Efficient screening for severe aortic valve stenosis using understandable artificial intelligence: a prospective diagnostic accuracy study

Aims

The medical need for screening of aortic valve stenosis (AS), which leads to timely and appropriate medical intervention, is rapidly increasing because of the high prevalence of AS in elderly population. This study aimed to establish a screening method using understandable artificial intelligence (AI) to detect severe AS based on heart sounds and to package the built AI into a smartphone application.

Methods and results

In this diagnostic accuracy study, we developed multiple convolutional neural networks (CNNs) using a modified stratified five-fold cross-validation to detect severe AS in electronic heart sound data recorded at three auscultation locations. Clinical validation was performed with the developed smartphone application in an independent cohort (model establishment: n = 556, clinical validation: n = 132). Our ensemble technique integrating the heart sounds from multiple auscultation locations increased the detection accuracy of CNN model by compensating detection errors. The established smartphone application achieved a sensitivity, specificity, accuracy, and F1 value of 97.6% (41/42), 94.4% (85/90), 95.7% (126/132), and 0.93, respectively, which were higher compared with the consensus of cardiologists (81.0%, 93.3%, 89.4%, and 0.829, respectively), implying a good utility for severe AS screening. The Gradient-based Class Activation Map demonstrated that the built AIs could focus on specific heart sounds to differentiate the severity of AS.

Conclusions

Our CNN model combining multiple auscultation locations and exported on smartphone application could efficiently identify severe AS based on heart sounds. The visual explanation of AI decisions for heart sounds was interpretable. These technologies may support medical training and remote consultations.

Assessment of Dual-Tree Complex Wavelet Transform to Improve SNR in Collaboration with Neuro-Fuzzy System for Heart-Sound Identification

The research paper proposes a novel denoising method to improve the outcome of heart-sound (HS)-based heart-condition identification by applying the dual-tree complex wavelet transform (DTCWT) together with the adaptive neuro-fuzzy inference System (ANFIS) classifier. The method consists of three steps: first, preprocessing to eliminate 50 Hz noise; second, applying four successive levels of DTCWT to denoise and reconstruct the time-domain HS signal; third, to evaluate ANFIS on a total of 2735 HS recordings from an international dataset (PhysioNet Challenge 2016). The results show that the signal-to-noise ratio (SNR) with DTCWT was significantly improved (p < 0.001) as compared to original HS recordings. Quantitatively, there was an 11% to many decibel (dB)-fold increase in SNR after DTCWT, representing a significant improvement in denoising HS. In addition, the ANFIS, using six time-domain features, resulted in 55–86% precision, 51–98% recall, 53–86% f-score, and 54–86% MAcc compared to other attempts on the same dataset. Therefore, DTCWT is a successful technique in removing noise from biosignals such as HS recordings. The adaptive property of ANFIS exhibited capability in classifying HS recordings.

The Effect of Signal Duration on the Classification of Heart Sounds: A Deep Learning Approach

Deep learning techniques are the future trend for designing heart sound classification methods, making conventional heart sound segmentation dispensable. However, despite using fixed signal duration for training, no study has assessed its effect on the final performance in detail. Therefore, this study aims at analysing the duration effect on the commonly used deep learning methods to provide insight for future studies in data processing, classifier, and feature selection. The results of this study revealed that (1) very short heart sound signal duration (1 s) weakens the performance of Recurrent Neural Networks (RNNs), whereas no apparent decrease in the tested Convolutional Neural Network (CNN) model was found. (2) RNN outperformed CNN using Mel-frequency cepstrum coefficients (MFCCs) as features. There was no difference between RNN models (LSTM, BiLSTM, GRU, or BiGRU). (3) Adding dynamic information (∆ and ∆²MFCCs) of the heart sound as a feature did not improve the RNNs' performance, and the improvement on CNN was also minimal (≤2.5% in MAcc). The findings provided a theoretical basis for further heart sound classification using deep learning techniques when selecting the input length.

A Robust Interpretable Deep Learning Classifier for Heart Anomaly Detection Without Segmentation

Traditionally, abnormal heart sound classification is framed as a three-stage process. The first stage involves segmenting the phonocardiogram to detect fundamental heart sounds; after which features are extracted and classification is performed. Some researchers in the field argue the segmentation step is an unwanted computational burden, whereas others embrace it as a prior step to feature extraction. When comparing accuracies achieved by studies that have segmented heart sounds before analysis with those who have overlooked that step, the question of whether to segment heart sounds before feature extraction is still open. In this study, we explicitly examine the importance of heart sound segmentation as a prior step for heart sound classification, and then seek to apply the obtained insights to propose a robust classifier for abnormal heart sound detection. Furthermore, recognizing the pressing need for explainable Artificial Intelligence (AI) models in the medical domain, we also unveil hidden representations learned by the classifier using model interpretation techniques. Experimental results demonstrate that the segmentation which can be learned by the model plays an essential role in abnormal heart sound classification. Our new classifier is also shown to be robust, stable and most importantly, explainable, with an accuracy of almost 100% on the widely used PhysioNet dataset.

Deep Learning Methods for Heart Sounds Classification: A Systematic Review

The automated classification of heart sounds plays a significant role in the diagnosis of cardiovascular diseases (CVDs). With the recent introduction of medical big data and artificial intelligence technology, there has been an increased focus on the development of deep learning approaches for heart sound classification. However, despite significant achievements in this field, there are still limitations due to insufficient data, inefficient training, and the unavailability of effective models. With the aim of improving the accuracy of heart sounds classification, an in-depth systematic review and an analysis of existing deep learning methods were performed in the present study, with an emphasis on the convolutional neural network (CNN) and recurrent neural network (RNN) methods developed over the last five years. This paper also discusses the challenges and expected future trends in the application of deep learning to heart sounds classification with the objective of providing an essential reference for further study.

Convolutional and recurrent neural networks for the detection of valvular heart diseases in phonocardiogram recordings

Valvular heart diseases (VHD) are one of the major causes of cardiovascular diseases that are having high mortality rates worldwide. The early diagnosis of VHD prevents the development of cardiac diseases and allows for optimum medication. Despite of the ability of current gold standards in identifying VHD, they still lack the required accuracy and thus, several cases go misdiagnosed. In this vein, a study is conducted herein to investigate the efficiency of deep learning models in identifying VHD through phonocardiography (PCG) recordings. PCG heart sounds were obtained from an open-access data-set representing normal heart sounds along with four major VHD; namely aortic stenosis (AS), mitral stenosis (MS), mitral regurgitation (MR), and mitral valve prolapse (MVP). A total of 1,000 patients were involved in the study with 200 recordings for each class. All recordings were initially trimmed to have 9,600 samples ensuring their coverage of at least 1 cardiac cycle. In addition, they were pre-processed by applying maximal overlap discrete wavelet transform (MODWT) smoothing algorithm and z-score normalization. The neural network architecture was designed to reduce the complexity often found in literature and consisted of a combination of convolutional neural networks (CNN) and recurrent neural networks (RNN) based on Bi-directional long short-term memory (BiLSTM). The model was trained and tested following a k-fold cross-validation scheme of 10-folds utilizing the CNN-BiLSTM network as well as the CNN and BiLSTM, individually. The highest performance was achieved using the CNN-BiLSTM network with an overall Cohen's kappa, accuracy, sensitivity, and specificity of 97.87%, 99.32%, 98.30%, and 99.58%, respectively. In addition, the model had an average area under the curve (AUC) of 0.998. Furthermore, the performance of the model was assessed on the PhysioNet/Computing in Cardiology 2016 challenge data-set and reached an overall accuracy of 87.31% with an AUC of 0.900. This study paves the way towards implementing deep learning models in VHD identification under clinical settings to assist clinicians in decision making and prevent many cases from cardiac abnormalities development.

Classification of heart sound signals using a novel deep WaveNet model

BACKGROUND AND OBJECTIVES

The high mortality rate and increasing prevalence of heart valve diseases globally warrant the need for rapid and accurate diagnosis of such diseases. Phonocardiogram (PCG) signals are used in this study due to the low cost of obtaining the signals. This study classifies five types of heart sounds, namely normal, aortic stenosis, mitral valve prolapse, mitral stenosis, and mitral regurgitation.

METHODS

We have proposed a novel in-house developed deep WaveNet model for automated classification of five types of heart sounds. The model is developed using a total of 1000 PCG recordings belonging to five classes with 200 recordings in each class.

RESULTS

We have achieved a training accuracy of 97% for the classification of heart sounds into five classes. The highest classification accuracy of 98.20% was achieved for the normal class. The developed model was validated with a 10-fold cross-validation, thus affirming its robustness.

CONCLUSION

The study results clearly indicate that the developed model is able to classify five types of heart sounds accurately. The developed system can be used by cardiologists to aid in the detection of heart valve diseases in patients.

HEALTH OF THINGS MODEL FOR CLASSIFYING HUMAN HEART SOUND SIGNALS USING CO-OCCURRENCE MATRIX AND SPECTROGRAM

Cardiovascular diseases have become one of the world’s leading causes of death today. Several decision-making systems have been developed with computer-aided support to help the cardiologists in detecting heart disease and thereby minimizing the mortality rate. This paper uses an unexplored sub-domain related to textural features for classifying phonocardiogram (PCG) as normal or abnormal using Grey Level Co-occurrence Matrix (GLCM). The matrix has been applied to extract features from spectrogram of the PCG signals taken from the Physionet 2016 benchmark dataset. Random Forest, Support Vector Machine, Neural Network, and XGBoost have been applied to assess the status of the human heart using PCG signal spectrogram. The result of GLCM is compared with the two other textural feature extraction methods, viz. structural co-occurrence matrix (SCM), and local binary patterns (LBP). Experimental results have proved that applying machine learning model to classify PCG signal on the dataset where GLCM has extracted the feature-set, the accuracy attained is greater as compared to its peer approaches. Thus, this methodology can go a long way to help the medical specialists in precisely and accurately assessing the heart condition of a patient.

Automatic Segmentation and Classification of Heart Sounds Using Modified Empirical Wavelet Transform and Power Features

A system for the automatic classification of cardiac sounds can be of great help for doctors in the diagnosis of cardiac diseases. Generally speaking, the main stages of such systems are (i) the pre-processing of the heart sound signal, (ii) the segmentation of the cardiac cycles, (iii) feature extraction and (iv) classification. In this paper, we propose methods for each of these stages. The modified empirical wavelet transform (EWT) and the normalized Shannon average energy are used in pre-processing and automatic segmentation to identify the systolic and diastolic intervals in a heart sound recording; then, six power characteristics are extracted (three for the systole and three for the diastole)—the motivation behind using power features is to achieve a low computational cost to facilitate eventual real-time implementations. Finally, different models of machine learning (support vector machine (SVM), k-nearest neighbor (KNN), random forest and multilayer perceptron) are used to determine the classifier with the best performance. The automatic segmentation method was tested with the heart sounds from the Pascal Challenge database. The results indicated an error (computed as the sum of the differences between manual segmentation labels from the database and the segmentation labels obtained by the proposed algorithm) of 843,440.8 for dataset A and 17,074.1 for dataset B, which are better values than those reported with the state-of-the-art methods. For automatic classification, 805 sample recordings from different databases were used. The best accuracy result was 99.26% using the KNN classifier, with a specificity of 100% and a sensitivity of 98.57%. These results compare favorably with similar works using the state-of-the-art methods.

Automated heart sound classification system from unsegmented phonocardiogram (PCG) using deep neural network

Given the patient to doctor ratio of 50,000:1 in low income and middle-income countries, there is a need for automated heart sound classification system that can screen the Phonocardiogram (PCG) records in real-time. This paper proposes deep neural network architectures such as a one-dimensional convolutional neural network (1D-CNN) and Feed-forward Neural Network (F-NN) for the classification of unsegmented phonocardiogram (PCG) signal. The research paper aims to automate the feature engineering and feature selection process used in the analysis of the PCG signal. The original PCG signal is down-sampled at 500 Hz. Then they are divided into smaller time segments of 6 s epochs. Savitzky-Golay filter is used to suppress the high-frequency noises in the signal by data point smoothening. The processed data was then provided as an input to the proposed deep neural network (DNN) architectures. 1081 PCG records were used for training and validating the proposed DNN models. The Feed-forward Neural Network model with five hidden layers provided a better overall accuracy of 0.8565 with a sensitivity of 0.8673, and specificity of 0.8475. The balanced accuracy of the model was found to be 0.8574. The performance of the model was also studied using the Receiver Operating Characteristic (ROC) plot, which produced an Area Under the Curve (AUC) value of 0.857. The classification accuracy of the proposed models was compared to the related works on PCG signal analysis for cardiovascular disease detection. The DNN models studied in this study provided comparable performance in heart sound classification without the requirement of feature engineering and segmentation of heart sound signals.

Classification of Heart Sounds Using Convolutional Neural Network

Heart sounds play an important role in the diagnosis of cardiac conditions. Due to the low signal-to-noise ratio (SNR), it is problematic and time-consuming for experts to discriminate different kinds of heart sounds. Thus, objective classification of heart sounds is essential. In this study, we combined a conventional feature engineering method with deep learning algorithms to automatically classify normal and abnormal heart sounds. First, 497 features were extracted from eight domains. Then, we fed these features into the designed convolutional neural network (CNN), in which the fully connected layers that are usually used before the classification layer were replaced with a global average pooling layer to obtain global information about the feature maps and avoid overfitting. Considering the class imbalance, the class weights were set in the loss function during the training process to improve the classification algorithm’s performance. Stratified five-fold cross-validation was used to evaluate the performance of the proposed method. The mean accuracy, sensitivity, specificity and Matthews correlation coefficient observed on the PhysioNet/CinC Challenge 2016 dataset were 86.8%, 87%, 86.6% and 72.1% respectively. The proposed algorithm’s performance achieves an appropriate trade-off between sensitivity and specificity.

Towards Domain Invariant Heart Sound Abnormality Detection Using Learnable Filterbanks

OBJECTIVE

Cardiac auscultation is the most practiced non-invasive and cost-effective procedure for the early diagnosis of heart diseases. While machine learning based systems can aid in automatically screening patients, the robustness of these systems is affected by numerous factors including the stethoscope/sensor, environment, and data collection protocol. This article studies the adverse effect of domain variability on heart sound abnormality detection and develops strategies to address this problem.

METHODS

We propose a novel Convolutional Neural Network (CNN) layer, consisting of time-convolutional (tConv) units, that emulate Finite Impulse Response (FIR) filters. The filter coefficients can be updated via backpropagation and be stacked in the front-end of the network as a learnable filterbank.

RESULTS

On publicly available multi-domain datasets, the proposed method surpasses the top-scoring systems found in the literature for heart sound abnormality detection (a binary classification task). We utilized sensitivity, specificity, F-1 score and Macc (average of sensitivity and specificity) as performance metrics. Our systems achieved relative improvements of up to 11.84% in terms of MAcc, compared to state-of-the-art methods.

CONCLUSION

The results demonstrate the effectiveness of the proposed learnable filterbank CNN architecture in achieving robustness towards sensor/domain variability in PCG signals.

SIGNIFICANCE

The proposed methods pave the way for deploying automated cardiac screening systems in diversified and underserved communities.

A Review of Computer-Aided Heart Sound Detection Techniques

Cardiovascular diseases have become one of the most prevalent threats to human health throughout the world. As a noninvasive assistant diagnostic tool, the heart sound detection techniques play an important role in the prediction of cardiovascular diseases. In this paper, the latest development of the computer-aided heart sound detection techniques over the last five years has been reviewed. There are mainly the following aspects: the theories of heart sounds and the relationship between heart sounds and cardiovascular diseases; the key technologies used in the processing and analysis of heart sound signals, including denoising, segmentation, feature extraction and classification; with emphasis, the applications of deep learning algorithm in heart sound processing. In the end, some areas for future research in computer-aided heart sound detection techniques are explored, hoping to provide reference to the prediction of cardiovascular diseases.

Machine Listening for Heart Status Monitoring: Introducing and Benchmarking HSS - the Heart Sounds Shenzhen Corpus

Auscultation of the heart is a widely studied technique, which requires precise hearing from practitioners as a means of distinguishing subtle differences in heart-beat rhythm. This technique is popular due to its non-invasive nature, and can be an early diagnosis aid for a range of cardiac conditions. Machine listening approaches can support this process, monitoring continuously and allowing for a representation of both mild and chronic heart conditions. Despite this potential, relevant databases and benchmark studies are scarce. In this paper, we introduce our publicly accessible database, the Heart Sounds Shenzhen Corpus (HSS), which was first released during the recent INTERSPEECH 2018 ComParE Heart Sound sub-challenge. Additionally, we provide a survey of machine learning work in the area of heart sound recognition, as well as a benchmark for HSS utilising standard acoustic features and machine learning models. At best our support vector machine with Log Mel features achieves 49.7% unweighted average recall on a three category task (normal, mild, moderate/severe).

Heart sound classification using the SNMFNet classifier

OBJECTIVE

Heart sound classification still suffers from the challenges involved in achieving high accuracy in the case of small samples. Dimension reduction attempts to extract low-dimensional features with more discriminability from high-dimensional spaces or raw data, and is popular in learning predictive models that target small sample problems. However, it can also be harmful to classification, because any reduction has the potential to lose information containing category attributes.

APPROACH

For this, a novel SNMFNet classifier is designed to directly associate the dimension reduction process with the classification procedure used for promoting feature dimension reduction to follow the approach that is beneficial for classification, thus making the low-dimensional features more distinguishable and addressing the challenge facing heart sound classification in small samples.

MAIN RESULTS

We evaluated our method and representative methods using a public heart sound dataset. The experimental results demonstrate that our method outperforms all comparative models with an obvious improvement in small samples. Furthermore, even if used with relatively sufficient samples, our method performs at least as well as the baseline that uses the same high-dimensional features.

SIGNIFICANCE

The proposed SNMFNet classifier significantly to improves the small sample problem in heart sound classification.

Phonocardiogram classification using deep neural networks and weighted probability comparisons

Cardiac auscultation is one of the most conventional approaches for the initial assessment of heart disease, however the technique is highly user-dependent and with low repeatability. Several computational approaches based on the analysis of the phonocardiograms (PCG) have been proposed to classify heart sounds into normal or abnormal, but most often do not achieve acceptable levels of sensitivity (Se) and specificity (Sp) or require the use of special hardware. We propose a novel approach for classification of PCG. First, the system makes use of deep neural networks for computing individual cardiac cycle probabilities, followed by classification using weighted probability comparisons. The system was tested on an extended dataset consisting of a balanced sample of 18179 normal and abnormal cycles, achieving Se and Sp values of 91.3% and 93.8% respectively. In addition, the system overcomes previous limitations since it was trained with a balanced sample; also, the decision factor used during the classification stage allows to control the trade-off between Se and Sp, making the proposed system suitable for clinical applications.

Deep Unsupervised Representation Learning for Abnormal Heart Sound Classification

Given the world-wide prevalence of heart disease, the robust and automatic detection of abnormal heart sounds could have profound effects on patient care and outcomes. In this regard, a comparison of conventional and state-of-theart deep learning based computer audition paradigms for the audio classification task of normal, mild abnormalities, and moderate/severe abnormalities as present in phonocardiogram recordings, is presented herein. In particular, we explore the suitability of deep feature representations as learnt by sequence to sequence autoencoders based on the auDeep toolkit. Key results, gained on the new Heart Sounds Shenzhen corpus, indicate that a fused combination of deep unsupervised features is well suited to the three-way classification problem, achieving our highest unweighted average recall of 47.9% on the test partition.

Deep Learning on 1-D Biosignals: a Taxonomy-based Survey

OBJECTIVES

 Deep learning models such as convolutional neural networks (CNNs) have been applied successfully to medical imaging, but biomedical signal analysis has yet to fully benefit from this novel approach. Our survey aims at (i) reviewing deep learning techniques for biosignal analysis in computer- aided diagnosis; and (ii) deriving a taxonomy for organizing the growing number of applications in the field.

METHODS

 A comprehensive literature research was performed using PubMed, Scopus, and ACM. Deep learning models were classified with respect to the (i) origin, (ii) dimension, and (iii) type of the biosignal as input to the deep learning model; (iv) the goal of the application; (v) the size and (vi) type of ground truth data; (vii) the type and (viii) schedule of learning the network; and (ix) the topology of the model.

RESULTS

 Between January 2010 and December 2017, a total 71 papers were published on the topic. The majority (n = 36) of papers are on electrocariography (ECG) signals. Most applications (n = 25) aim at detection of patterns, while only a few (n = 6) at predection of events. Out of 36 ECG-based works, many (n = 17) relate to multi-lead ECG. Other biosignals that have been identified in the survey are electromyography, phonocardiography, photoplethysmography, electrooculography, continuous glucose monitoring, acoustic respiratory signal, blood pressure, and electrodermal activity signal, while ballistocardiography or seismocardiography have yet to be analyzed using deep learning techniques. In supervised and unsupervised applications, CNNs and restricted Boltzmann machines are the most and least frequently used, (n = 34) and (n = 15), respectively.

CONCLUSION

 Our key-code classification of relevant papers was used to cluster the approaches that have been published to date and demonstrated a large variability of research with respect to data, application, and network topology. Future research is expected to focus on the standardization of deep learning architectures and on the optimization of the network parameters to increase performance and robustness. Furthermore, application-driven approaches and updated training data from mobile recordings are needed.

Learning Front-end Filter-bank Parameters using Convolutional Neural Networks for Abnormal Heart Sound Detection

Automatic heart sound abnormality detection can play a vital role in the early diagnosis of heart diseases, particularly in low-resource settings. The state-of-the-art algorithms for this task utilize a set of Finite Impulse Response (FIR) band-pass filters as a front-end followed by a Convolutional Neural Network (CNN) model. In this work, we propound a novel CNN architecture that integrates the front-end band-pass filters within the network using time-convolution (tConv) layers, which enables the FIR filter-bank parameters to become learnable. Different initialization strategies for the learnable filters, including random parameters and a set of predefined FIR filter-bank coefficients, are examined. Using the proposed tConv layers, we add constraints to the learnable FIR filters to ensure linear and zero phase responses. Experimental evaluations are performed on a balanced 4-fold cross-validation task prepared using the PhysioNet/CinC 2016 dataset. Results demonstrate that the proposed models yield superior performance compared to the state-of-the-art system, while the linear phase FIR filter-bank method provides an absolute improvement of 9.54% over the baseline in terms of an overall accuracy metric.