Towards the classification of heart sounds based on convolutional deep neural network

Identifying pediatric heart murmurs and distinguishing innocent from pathologic using deep learning

OBJECTIVE

To develop a deep learning algorithm to perform multi-class classification of normal pediatric heart sounds, innocent murmurs, and pathologic murmurs.

METHODS

We prospectively enrolled children under age 18 being evaluated by the Division of Pediatric Cardiology. Parents provided consent for a deidentified recording of their child's heart sounds with a digital stethoscope. Innocent murmurs were validated by a pediatric cardiologist and pathologic murmurs were validated by echocardiogram. To augment our collection of normal heart sounds, we utilized a public database of pediatric heart sound recordings (Oliveira, 2022). We propose two novel approaches for this audio classification task. We train a vision transformer on either Markov transition field or Gramian angular field image representations of the frequency spectrum. We benchmark our results against a ResNet-50 CNN trained on spectrogram images.

RESULTS

Our final dataset consisted of 366 normal heart sounds, 175 innocent murmurs, and 216 pathologic murmurs. Innocent murmurs collected include Still's murmur, venous hum, and flow murmurs. Pathologic murmurs included ventricular septal defect, tetralogy of Fallot, aortic regurgitation, aortic stenosis, pulmonary stenosis, mitral regurgitation and stenosis, and tricuspid regurgitation. We find that the Vision Transformer consistently outperforms the ResNet-50 on all three image representations, and that the Gramian angular field is the superior image representation for pediatric heart sounds. We calculated a one-vs-rest multi-class ROC curve for each of the three classes. Our best model achieves an area under the curve (AUC) value of 0.92 ± 0.05, 0.83 ± 0.04, and 0.88 ± 0.04 for identifying normal heart sounds, innocent murmurs, and pathologic murmurs, respectively.

CONCLUSION

We present two novel methods for pediatric heart sound classification, which outperforms the current standard of using a convolutional neural network trained on spectrogram images. To our knowledge, we are the first to demonstrate multi-class classification of pediatric murmurs. Multiclass output affords a more explainable and interpretable model, which can facilitate further model improvement in the downstream model development cycle and enhance clinician trust and therefore adoption.

A novel feature-level fusion scheme with multimodal attention CNN for heart sound classification

BACKGROUND AND OBJECTIVE

Most of the existing machine learning-based heart sound classification methods achieve limited accuracy. Since they primarily depend on single domain feature information and tend to focus equally on each part of the signal rather than employing a selective attention mechanism. In addition, they fail to exploit convolutional neural network (CNN) - based features with an effective fusion strategy.

METHODS

In order to overcome these limitations, a novel multimodal attention convolutional neural network (MACNN) with a feature-level fusion strategy, in which Mel-cepstral domain as well as general frequency domain features are incorporated to increase the diversity of the features, is proposed in this paper. In the proposed method, DilationAttenNet is first utilized to construct attention-based CNN feature extractors and then these feature extractors are jointly optimized in MACNN at the feature-level. The attention mechanism aims to suppress irrelevant information and focus on crucial diverse features extracted from the CNN.

RESULTS

Extensive experiments are carried out to study the efficacy of the feature level fusion in comparison to that with early fusion. The results show that the proposed MACNN method significantly outperforms the state-of-the-art approaches in terms of accuracy and score for the two publicly available Github and Physionet datasets.

CONCLUSION

The findings of our experiments demonstrated the high performance for heart sound classification based on the proposed MACNN, and hence have potential clinical usefulness in the identification of heart diseases. This technique can assist cardiologists and researchers in the design and development of heart sound classification methods.

Crossmixed convolutional neural network for digital speech recognition

Digital speech recognition is a challenging problem that requires the ability to learn complex signal characteristics such as frequency, pitch, intensity, timbre, and melody, which traditional methods often face issues in recognizing. This article introduces three solutions based on convolutional neural networks (CNN) to solve the problem: 1D-CNN is designed to learn directly from digital data; 2DS-CNN and 2DM-CNN have a more complex architecture, transferring raw waveform into transformed images using Fourier transform to learn essential features. Experimental results on four large data sets, containing 30,000 samples for each, show that the three proposed models achieve superior performance compared to well-known models such as GoogLeNet and AlexNet, with the best accuracy of 95.87%, 99.65%, and 99.76%, respectively. With 5-10% higher performance than other models, the proposed solution has demonstrated the ability to effectively learn features, improve recognition accuracy and speed, and open up the potential for broad applications in virtual assistants, medical recording, and voice commands.

Deep-Learning-Based Arrhythmia Detection Using ECG Signals: A Comparative Study and Performance Evaluation

Heart diseases is the world's principal cause of death, and arrhythmia poses a serious risk to the health of the patient. Electrocardiogram (ECG) signals can be used to detect arrhythmia early and accurately, which is essential for immediate treatment and intervention. Deep learning approaches have played an important role in automatically identifying complicated patterns from ECG data, which can be further used to identify arrhythmia. In this paper, deep-learning-based methods for arrhythmia identification using ECG signals are thoroughly studied and their performances evaluated on the basis of accuracy, specificity, precision, and F1 score. We propose the development of a small CNN, and its performance is compared against pretrained models like GoogLeNet. The comparative study demonstrates the promising potential of deep-learning-based arrhythmia identification using ECG signals.

Detection of valvular heart diseases combining orthogonal non-negative matrix factorization and convolutional neural networks in PCG signals

BACKGROUND AND OBJECTIVE

Valvular heart disease (VHD) is associated with elevated mortality rates. Although transthoracic echocardiography (TTE) is the gold standard detection tool, phonocardiography (PCG) could be an alternative as it is a cost-effective and noninvasive method for cardiac auscultation. Many researchers have dedicated their efforts to improving the decision-making process and developing robust and precise approaches to assist physicians in providing reliable diagnoses of VHD.

METHODS

This research proposes a novel approach for the detection of anomalous valvular heart sounds from PCG signals. The proposed approach combines orthogonal non-negative matrix factorization (ONMF) and convolutional neural network (CNN) architectures in a three-stage cascade. The aim of the proposal is to improve the learning process by identifying the optimal ONMF temporal or spectral patterns for accurate detection. In the first stage, the time-frequency representation of the input PCG signal is computed. Next, band-pass filtering is performed to locate the spectral range that is most relevant for the presence of such cardiac abnormalities. In the second stage, the temporal and spectral cardiac structures are extracted using the ONMF approach. These structures are utilized in the third stage and fed into the CNN architecture to detect abnormal heart sounds.

RESULTS

Several state-of-the-art CNN architectures, such as LeNet5, AlexNet, ResNet50, VGG16 and GoogLeNet, have been evaluated to determine the effectiveness of using ONMF temporal features for VHD detection. The results reveal that the integration of ONMF temporal features with a CNN classifier significantly improve VHD detection. Specifically, the proposed approach achieves an accuracy improvement of approximately 45% when ONMF spectral features are used and 35% when time-frequency features from the short-time Fourier transform (STFT) spectrogram are used. Additionally, feeding ONMF temporal features into low-complexity CNN architectures yields competitive results comparable to those obtained with complex architectures.

CONCLUSIONS

The temporal structure factorized by ONMF plays a critical role in distinguishing between normal heart sounds and abnormal heart sounds since the repeatability of normal heart cycles is disrupted by the presence of cardiac abnormalities. Consequently, the results highlight the importance of appropriate input data representation in the learning process of CNN models in the biomedical field of valvular heart sound detection.

The Reliability and Accuracy of a Fall Risk Assessment Procedure Using Mobile Smartphone Sensors Compared with a Physiological Profile Assessment

Falls in older people are a major health concern as the leading cause of disability and the second most common cause of accidental death. We developed a rapid fall risk assessment based on a combination of physical performance measurements made with an inertial sensor embedded in a smartphone. This study aimed to evaluate and validate the reliability and accuracy of an easy-to-use smartphone fall risk assessment by comparing it with the Physiological Profile Assessment (PPA) results. Sixty-five participants older than 55 performed a variation of the Timed Up and Go test using smartphone sensors. Balance and gait parameters were calculated, and their reliability was assessed by the (ICC) and compared with the PPAs. Since the PPA allows classification into six levels of fall risk, the data obtained from the smartphone assessment were categorised into six equivalent levels using different parametric and nonparametric classifier models with neural networks. The F1 score and geometric mean of each model were also calculated. All selected parameters showed ICCs around 0.9. The best classifier, in terms of accuracy, was the nonparametric mixed input data model with a 100% success rate in the classification category. In conclusion, fall risk can be reliably assessed using a simple, fast smartphone protocol that allows accurate fall risk classification among older people and can be a useful screening tool in clinical settings.

Classification of Parkinson's disease stages with a two-stage deep neural network

Introduction

Parkinson's disease is one of the most prevalent neurodegenerative diseases. In the most advanced stages, PD produces motor dysfunction that impairs basic activities of daily living such as balance, gait, sitting, or standing. Early identification allows healthcare personnel to intervene more effectively in rehabilitation. Understanding the altered aspects and impact on the progression of the disease is important for improving the quality of life. This study proposes a two-stage neural network model for the classifying the initial stages of PD using data recorded with smartphone sensors during a modified Timed Up & Go test.

Methods

The proposed model consists on two stages: in the first stage, a semantic segmentation of the raw sensor signals classifies the activities included in the test and obtains biomechanical variables that are considered clinically relevant parameters for functional assessment. The second stage is a neural network with three input branches: one with the biomechanical variables, one with the spectrogram image of the sensor signals, and the third with the raw sensor signals.

Results

This stage employs convolutional layers and long short-term memory. The results show a mean accuracy of 99.64% for the stratified k-fold training/validation process and 100% success rate of participants in the test phase.

Discussion

The proposed model is capable of identifying the three initial stages of Parkinson's disease using a 2-min functional test. The test easy instrumentation requirements and short duration make it feasible for use feasible in the clinical context.

Heart sound classification based on improved mel-frequency spectral coefficients and deep residual learning

Heart sound classification plays a critical role in the early diagnosis of cardiovascular diseases. Although there have been many advances in heart sound classification in the last few years, most of them are still based on conventional segmented features and shallow structure-based classifiers. Therefore, we propose a new heart sound classification method based on improved mel-frequency cepstrum coefficient features and deep residual learning. Firstly, the heart sound signal is preprocessed, and its improved features are computed. Then, these features are used as input features of the neural network. The pathological information in the heart sound signal is further extracted by the deep residual network. Finally, the heart sound signal is classified into different categories according to the features learned by the neural network. This paper presents comprehensive analyses of different network parameters and network connection strategies. The proposed method achieves an accuracy of 94.43% on the dataset in this paper.

Cardi-Net: A deep neural network for classification of cardiac disease using phonocardiogram signal

BACKGROUND AND OBJECTIVES

The lack of medical facilities in isolated areas makes many patients remain aloof from quick and timely diagnosis of cardiovascular diseases, leading to high mortality rates. A deep learning based method for automatic diagnosis of multiple cardiac diseases from Phonocardiogram (PCG) signals is proposed in this paper.

METHODS

The proposed system is a combination of deep learning based convolutional neural network (CNN) and power spectrogram Cardi-Net, which can extract deep discriminating features of PCG signals from the power spectrogram to identify the diseases. The choice of Power Spectral Density (PSD) makes the model extract highly discriminatory features significant for the multi-classification of four common cardiac disorders.

RESULTS

Data augmentation techniques are applied to make the model robust, and the model undergoes 10-fold cross-validation to yield an overall accuracy of 98.879% on the test dataset to diagnose multi heart diseases from PCG signals.

CONCLUSION

The proposed model is completely automatic, where signal pre-processing and feature engineering are not required. The conversion time of power spectrogram from PCG signals is very low range from 0.10 s to 0.11 s. This reduces the complexity of the model, making it highly reliable and robust for real-time applications. The proposed architecture can be deployed on cloud and a low cost processor, desktop, android app leading to proper access to the dispensaries in remote areas.

A Wavelet Packet Transform and Convolutional Neural Network Method Based Ultrasonic Detection Signals Recognition of Concrete

This paper proposes a new intelligent recognition method for concrete ultrasonic detection based on wavelet packet transform and a convolutional neural network (CNN). To validate the proposed data-based method, a case study is presented where the K-fold cross-validation was adopted to produce the performance analysis and classification experiments. Moreover, three evaluation indicators, precision, recall, and F-score, are calculated for analyzing the classification performance of the trained models. As a result, the obtained four-classifying CNN reaches more than 99% detection accuracy while the lowest recognition accuracy is not less than 92.5% on the testing dataset for the six-classifying CNN model. Compared with the existing stochastic configuration network (SCN) models, the presented method achieves the design objective with better recognition performance. The calculation results of the six-classifying and five-classifying models and related research clearly indicate the remaining challenging tasks for intelligent recognition algorithms in extracting features and classifying mass data from various concrete defects precisely and efficiently.

A multi-variate heart disease optimization and recognition framework

Cardiovascular diseases (CVD) are the most widely spread diseases all over the world among the common chronic diseases. CVD represents one of the main causes of morbidity and mortality. Therefore, it is vital to accurately detect the existence of heart diseases to help to save the patient life and prescribe a suitable treatment. The current evolution in artificial intelligence plays an important role in helping physicians diagnose different diseases. In the present work, a hybrid framework for the detection of heart diseases using medical voice records is suggested. A framework that consists of four layers, namely “Segmentation” Layer, “Features Extraction” Layer, “Learning and Optimization” Layer, and “Export and Statistics” Layer is proposed. In the first layer, a novel segmentation technique based on the segmentation of variable durations and directions (i.e., forward and backward) is suggested. Using the proposed technique, 11 datasets with 14,416 numerical features are generated. The second layer is responsible for feature extraction. Numerical and graphical features are extracted from the resulting datasets. In the third layer, numerical features are passed to 5 different Machine Learning (ML) algorithms, while graphical features are passed to 8 different Convolutional Neural Networks (CNN) with transfer learning to select the most suitable configurations. Grid Search and Aquila Optimizer (AO) are used to optimize the hyperparameters of ML and CNN configurations, respectively. In the last layer, the output of the proposed hybrid framework is validated using different performance metrics. The best-reported metrics are (1) 100% accuracy using ML algorithms including Extra Tree Classifier (ETC) and Random Forest Classifier (RFC) and (2) 99.17% accuracy using CNN.

Identification of Characteristic Points in Multivariate Physiological Signals by Sensor Fusion and Multi-Task Deep Networks

Identification of characteristic points in physiological signals, such as the peak of the R wave in the electrocardiogram and the peak of the systolic wave of the photopletismogram, is a fundamental step for the quantification of clinical parameters, such as the pulse transit time. In this work, we presented a novel neural architecture, called eMTUnet, to automate point identification in multivariate signals acquired with a chest-worn device. The eMTUnet consists of a single deep network capable of performing three tasks simultaneously: (i) localization in time of characteristic points (labeling task), (ii) evaluation of the quality of signals (classification task); (iii) estimation of the reliability of classification (reliability task). Preliminary results in overnight monitoring showcased the ability to detect characteristic points in the four signals with a recall index of about 1.00, 0.90, 0.90, and 0.80, respectively. The accuracy of the signal quality classification was about 0.90, on average over four different classes. The average confidence of the correctly classified signals, against the misclassifications, was 0.93 vs. 0.52, proving the worthiness of the confidence index, which may better qualify the point identification. From the achieved outcomes, we point out that high-quality segmentation and classification are both ensured, which brings the use of a multi-modal framework, composed of wearable sensors and artificial intelligence, incrementally closer to clinical translation.

Deep Learning-Based Heart Sound Analysis for Left Ventricular Diastolic Dysfunction Diagnosis

The aggravation of left ventricular diastolic dysfunction (LVDD) could lead to ventricular remodeling, wall stiffness, reduced compliance, and progression to heart failure with a preserved ejection fraction. A non-invasive method based on convolutional neural networks (CNN) and heart sounds (HS) is presented for the early diagnosis of LVDD in this paper. A deep convolutional generative adversarial networks (DCGAN) model-based data augmentation (DA) method was proposed to expand a HS database of LVDD for model training. Firstly, the preprocessing of HS signals was performed using the improved wavelet denoising method. Secondly, the logistic regression based hidden semi-Markov model was utilized to segment HS signals, which were subsequently converted into spectrograms for DA using the short-time Fourier transform (STFT). Finally, the proposed method was compared with VGG-16, VGG-19, ResNet-18, ResNet-50, DenseNet-121, and AlexNet in terms of performance for LVDD diagnosis. The result shows that the proposed method has a reasonable performance with an accuracy of 0.987, a sensitivity of 0.986, and a specificity of 0.988, which proves the effectiveness of HS analysis for the early diagnosis of LVDD and demonstrates that the DCGAN-based DA method could effectively augment HS data.

Deep learning based classification of unsegmented phonocardiogram spectrograms leveraging transfer learning

OBJECTIVE

Cardiovascular diseases (CVDs) are the main cause of deaths all over the world. This research focuses on computer-aided analysis based on deep learning of phonocardiogram (PCG) signals that can enable improved and timely detection of heart abnormalities. The two widely used publicly available PCG datasets are from the PhysioNet/CinC (2016) and PASCAL (2011) challenges. The datasets are significantly different in terms of the tools used for data acquisition, clinical protocols, digital storages and signal qualities, making it challenging to process and analyze.

APPROACH

In this work, we have used short-time Fourier transform (STFT) based spectrograms to learn the representative patterns of the normal and abnormal PCG signals. Spectrograms generated from both the datasets are utilized to perform four different studies: (i) train, validate and test different variants of convolutional neural network (CNN) models with PhysioNet dataset, (ii) train, validate and test the best performing CNN structure on PASCAL dataset as well as, (iii) on the combined PhysioNet-PASCAL dataset and (iv) finally, transfer learning technique is employed to train the best performing pre-trained network from the first study with PASCAL dataset.

MAIN RESULTS

The first study achieves an accuracy, sensitivity, specificity, precision and F1 score of 95.75%, 96.3%, 94.1%, 97.52% , and 96.93% respectively while the second study shows accuracy, sensitivity, specificity, precision and F1 score of 75.25%, 74.2%, 76.4%, 76.73%, and 75.42% respectively. The third study shows accuracy, sensitivity, specificity, precision and F1 score of 92.7%, 94.98%, 89.95%, 95.3% and 94.6% respectively. Finally, the fourth study shows a precision of 96.98% on the noisy PASCAL dataset with transfer learning approach.

SIGNIFICANCE

The proposed approach employs a less complex and relatively light custom CNN model that outperforms most of the recent competing studies by achieving comparatively high classification accuracy and precision, making it suitable for screening CVDs using PCG signals.

Signals classification based on IA-optimal CNN

The versatility of the existing A-optimal-based CNN for solving multiple types of signals classification problems has not been verified by different signals datasets. Moreover, the existing A-optimal-based CNN uses a simplified approximate function as the optimization objective function instead of precise analytical function, which affects the signals classification accuracy to a certain extent. In this paper, a classification method called IA-optimal CNN is proposed. To improve the stability of the classifier, the trace of the covariance matrix of the weights of the fully connected layer is used as the optimization objective function, and the parameter optimization model is established without any simplification of the optimization objective function. In addition, to avoid the difficulty of not being able to obtain the analytical expression formula of the partial derivative of the inverse matrix with regard to the networks parameters, a novel dual function is introduced to transform the optimization problem into an equivalent binary function optimization problem. Furthermore, based on the above analytical solution results, the parameters are updated using the alternate iterative optimization method and the accurate weight update formula is deduced in detail. Five signals datasets are used to test the universality of the IA-optimal CNN in signals classification fields. The performance of IA-optimal CNN is showed, and the experimental results are compared with the existing A-optimal-based classification algorithm. Lastly, the following conclusion is proved theoretically: For the A-optimal-based CNN, the trace of the covariance matrix will continue to decrease and approach a convergence value in the iterative process, but it is impossible for the networks to strictly reach the A-optimal state.

Deep Learning Methods for Heart Sounds Classification: A Systematic Review

The automated classification of heart sounds plays a significant role in the diagnosis of cardiovascular diseases (CVDs). With the recent introduction of medical big data and artificial intelligence technology, there has been an increased focus on the development of deep learning approaches for heart sound classification. However, despite significant achievements in this field, there are still limitations due to insufficient data, inefficient training, and the unavailability of effective models. With the aim of improving the accuracy of heart sounds classification, an in-depth systematic review and an analysis of existing deep learning methods were performed in the present study, with an emphasis on the convolutional neural network (CNN) and recurrent neural network (RNN) methods developed over the last five years. This paper also discusses the challenges and expected future trends in the application of deep learning to heart sounds classification with the objective of providing an essential reference for further study.

Automatic recognition of murmurs of ventricular septal defect using convolutional recurrent neural networks with temporal attentive pooling

Recognizing specific heart sound patterns is important for the diagnosis of structural heart diseases. However, the correct recognition of heart murmur depends largely on clinical experience. Accurately identifying abnormal heart sound patterns is challenging for young and inexperienced clinicians. This study is aimed at the development of a novel algorithm that can automatically recognize systolic murmurs in patients with ventricular septal defects (VSDs). Heart sounds from 51 subjects with VSDs and 25 subjects without a significant heart malformation were obtained in this study. Subsequently, the soundtracks were divided into different training and testing sets to establish the recognition system and evaluate the performance. The automatic murmur recognition system was based on a novel temporal attentive pooling-convolutional recurrent neural network (TAP-CRNN) model. On analyzing the performance using the test data that comprised 178 VSD heart sounds and 60 normal heart sounds, a sensitivity rate of 96.0% was obtained along with a specificity of 96.7%. When analyzing the heart sounds recorded in the second aortic and tricuspid areas, both the sensitivity and specificity were 100%. We demonstrated that the proposed TAP-CRNN system can accurately recognize the systolic murmurs of VSD patients, showing promising potential for the development of software for classifying the heart murmurs of several other structural heart diseases.