Heart sound classification based on improved MFCC features and convolutional recurrent neural networks

Feature-Based Fusion Using CNN for Lung and Heart Sound Classification

Lung or heart sound classification is challenging due to the complex nature of audio data, its dynamic properties of time, and frequency domains. It is also very difficult to detect lung or heart conditions with small amounts of data or unbalanced and high noise in data. Furthermore, the quality of data is a considerable pitfall for improving the performance of deep learning. In this paper, we propose a novel feature-based fusion network called FDC-FS for classifying heart and lung sounds. The FDC-FS framework aims to effectively transfer learning from three different deep neural network models built from audio datasets. The innovation of the proposed transfer learning relies on the transformation from audio data to image vectors and from three specific models to one fused model that would be more suitable for deep learning. We used two publicly available datasets for this study, i.e., lung sound data from ICHBI 2017 challenge and heart challenge data. We applied data augmentation techniques, such as noise distortion, pitch shift, and time stretching, dealing with some data issues in these datasets. Importantly, we extracted three unique features from the audio samples, i.e., Spectrogram, MFCC, and Chromagram. Finally, we built a fusion of three optimal convolutional neural network models by feeding the image feature vectors transformed from audio features. We confirmed the superiority of the proposed fusion model compared to the state-of-the-art works. The highest accuracy we achieved with FDC-FS is 99.1% with Spectrogram-based lung sound classification while 97% for Spectrogram and Chromagram based heart sound classification.

Rheumatic Heart Disease Screening Based on Phonocardiogram

Rheumatic heart disease (RHD) is one of the most common causes of cardiovascular complications in developing countries. It is a heart valve disease that typically affects children. Impaired heart valves stop functioning properly, resulting in a turbulent blood flow within the heart known as a murmur. This murmur can be detected by cardiac auscultation. However, the specificity and sensitivity of manual auscultation were reported to be low. The other alternative is echocardiography, which is costly and requires a highly qualified physician. Given the disease's current high prevalence rate (the latest reported rate in the study area (Ethiopia) was 5.65%), there is a pressing need for early detection of the disease through mass screening programs. This paper proposes an automated RHD screening approach using machine learning that can be used by non-medically trained persons outside of a clinical setting. Heart sound data was collected from 124 persons with RHD (PwRHD) and 46 healthy controls (HC) in Ethiopia with an additional 81 HC records from an open-access dataset. Thirty-one distinct features were extracted to correctly represent RHD. A support vector machine (SVM) classifier was evaluated using two nested cross-validation approaches to quantitatively assess the generalization of the system to previously unseen subjects. For regular nested 10-fold cross-validation, an f1-score of 96.0 ± 0.9%, recall 95.8 ± 1.5%, precision 96.2 ± 0.6% and a specificity of 96.0 ± 0.6% were achieved. In the imbalanced nested cross-validation at a prevalence rate of 5%, it achieved an f1-score of 72.2 ± 0.8%, recall 92.3 ± 0.4%, precision 59.2 ± 3.6%, and a specificity of 94.8 ± 0.6%. In screening tasks where the prevalence of the disease is small, recall is more important than precision. The findings are encouraging, and the proposed screening tool can be inexpensive, easy to deploy, and has an excellent detection rate. As a result, it has the potential for mass screening and early detection of RHD in developing countries.

Detection of dementia on voice recordings using deep learning: a Framingham Heart Study

BACKGROUND

Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available.

METHODS AND FINDINGS

We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE.

CONCLUSION

This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.

Detection of dementia on voice recordings using deep learning: a Framingham Heart Study

BACKGROUND

Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available.

METHODS AND FINDINGS

We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE.

CONCLUSION

This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.

Prediction of exercise sudden death in rabbit exhaustive swimming using deep neural network

Background and objective

Moderate exercise contributes to good health. However, excessive exercise may lead to cardiac fatigue, myocardial damage and even exercise sudden death. Monitoring the heart health has important implication to prevent exercise sudden death. Diagnosis methods such as electrocardiogram, echocardiogram, blood pressure and histological analysis have shown that arrhythmia and left ventricular fibrosis are early warning symptoms of exercise sudden death. Heart sounds (HS) can reflect the changes of cardiac valve, cardiac blood flow and myocardial function. Deep learning has drawn wide attention because of its ability to recognize disease. Therefore, a deep learning method combined with HS was proposed to predict exercise sudden death in New Zealand rabbits. The objective is to develop a method to predict exercise sudden death in New Zealand rabbits.

Methods

This paper proposed a method to predict exercise sudden death in New Zealand rabbits based on convolutional neural network (CNN) and gated recurrent unit (GRU). The weight-bearing exhaustive swimming experiment was conducted to obtain the HS of exercise sudden death and surviving New Zealand rabbits (n = 11/10) at four different time points. Then, the improved Viola integral method and double threshold method were employed to segment HS signals. The segmented HS frames at different time points were taken as the input of a combined CNN and GRU called CNN–GRU network to complete the prediction of exercise sudden death.

Results

In order to evaluate the performance of proposed network, CNN and GRU were used for comparison. When the fourth time point segmented HS frames were taken as input, the result shows that the proposed network has better performance with an accuracy of 89.57%, a sensitivity of 89.38% and a specificity of 92.20%. In addition, the segmented HS frames at different time points were input into CNN–GRU network, and the result shows that with the progress of the experiment, the prediction accuracy of exercise sudden death in New Zealand rabbits increased from 50.98 to 89.57%.

Conclusion

The proposed network shows good performance in classifying HS, which proves the feasibility of deep learning in exploring exercise sudden death. Further, it may have important implications in helping humans explore exercise sudden death.

Benchmarking of eight recurrent neural network variants for breath phase and adventitious sound detection on a self-developed open-access lung sound database-HF_Lung_V1

A reliable, remote, and continuous real-time respiratory sound monitor with automated respiratory sound analysis ability is urgently required in many clinical scenarios-such as in monitoring disease progression of coronavirus disease 2019-to replace conventional auscultation with a handheld stethoscope. However, a robust computerized respiratory sound analysis algorithm for breath phase detection and adventitious sound detection at the recording level has not yet been validated in practical applications. In this study, we developed a lung sound database (HF_Lung_V1) comprising 9,765 audio files of lung sounds (duration of 15 s each), 34,095 inhalation labels, 18,349 exhalation labels, 13,883 continuous adventitious sound (CAS) labels (comprising 8,457 wheeze labels, 686 stridor labels, and 4,740 rhonchus labels), and 15,606 discontinuous adventitious sound labels (all crackles). We conducted benchmark tests using long short-term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (BiLSTM), bidirectional GRU (BiGRU), convolutional neural network (CNN)-LSTM, CNN-GRU, CNN-BiLSTM, and CNN-BiGRU models for breath phase detection and adventitious sound detection. We also conducted a performance comparison between the LSTM-based and GRU-based models, between unidirectional and bidirectional models, and between models with and without a CNN. The results revealed that these models exhibited adequate performance in lung sound analysis. The GRU-based models outperformed, in terms of F1 scores and areas under the receiver operating characteristic curves, the LSTM-based models in most of the defined tasks. Furthermore, all bidirectional models outperformed their unidirectional counterparts. Finally, the addition of a CNN improved the accuracy of lung sound analysis, especially in the CAS detection tasks.

Signals classification based on IA-optimal CNN

The versatility of the existing A-optimal-based CNN for solving multiple types of signals classification problems has not been verified by different signals datasets. Moreover, the existing A-optimal-based CNN uses a simplified approximate function as the optimization objective function instead of precise analytical function, which affects the signals classification accuracy to a certain extent. In this paper, a classification method called IA-optimal CNN is proposed. To improve the stability of the classifier, the trace of the covariance matrix of the weights of the fully connected layer is used as the optimization objective function, and the parameter optimization model is established without any simplification of the optimization objective function. In addition, to avoid the difficulty of not being able to obtain the analytical expression formula of the partial derivative of the inverse matrix with regard to the networks parameters, a novel dual function is introduced to transform the optimization problem into an equivalent binary function optimization problem. Furthermore, based on the above analytical solution results, the parameters are updated using the alternate iterative optimization method and the accurate weight update formula is deduced in detail. Five signals datasets are used to test the universality of the IA-optimal CNN in signals classification fields. The performance of IA-optimal CNN is showed, and the experimental results are compared with the existing A-optimal-based classification algorithm. Lastly, the following conclusion is proved theoretically: For the A-optimal-based CNN, the trace of the covariance matrix will continue to decrease and approach a convergence value in the iterative process, but it is impossible for the networks to strictly reach the A-optimal state.

Deep Learning Methods for Heart Sounds Classification: A Systematic Review

The automated classification of heart sounds plays a significant role in the diagnosis of cardiovascular diseases (CVDs). With the recent introduction of medical big data and artificial intelligence technology, there has been an increased focus on the development of deep learning approaches for heart sound classification. However, despite significant achievements in this field, there are still limitations due to insufficient data, inefficient training, and the unavailability of effective models. With the aim of improving the accuracy of heart sounds classification, an in-depth systematic review and an analysis of existing deep learning methods were performed in the present study, with an emphasis on the convolutional neural network (CNN) and recurrent neural network (RNN) methods developed over the last five years. This paper also discusses the challenges and expected future trends in the application of deep learning to heart sounds classification with the objective of providing an essential reference for further study.

Can empirical mode decomposition improve heartbeat detection in fetal phonocardiography signals?

BACKGROUND AND OBJECTIVE

A fetal phonocardiography signal can be hard to interpret and classify due to various sources of additive noise in the womb, spanning from fetal movement to maternal heart sounds. Nevertheless, the non-invasive nature of the method makes it potentially suitable for long-term monitoring of fetal health, especially since it can be implemented on ubiquitous devices such as smartphones. We have employed empirical mode decomposition for the extraction of intrinsic mode functions that would enable the utilization of additional characteristics from the signal.

METHODS

Fetal heart recordings from 7 pregnant women in the 3rd trimester or pregnancy were taken in parallel with a measurement microphone and a portable Doppler device. Signal peaks positions from the Doppler were taken as the locations of S1 heart sounds and subsequently used as classification labels for the microphone signal. After employing a moving window approach for segmentation, more than 7600 observations were stored in the final dataset. The 135 extracted features consisted of typical audio temporal and spectral characteristics, each taken from separate sets of audio signals and intrinsic mode functions. We have used a number of metrics and methods to validate the usability of features, including univariate analysis of feature ranking and importance. Furthermore, we have used machine learning to train a number of classifiers to validate the usability of features based on intrinsic mode functions, taking prediction accuracy as the comparison metric.

RESULTS

Features extracted from intrinsic mode functions combined with audio features significantly improve accuracy in comparison to using only audio features. The improvements of detection accuracy obtained with a selected set of combined features spanned from 3.8% to even 10.3% based on the employed classifier.

CONCLUSIONS

We have utilized empirical mode decomposition as a method of extracting features relevant for fetal heartbeat classification. The results show consistent improvements in detection accuracy when these characteristics are added to a set of conventional audio features. This implies substantial benefits of applying empirical mode decomposition and lays the groundwork for future research on fetal heartbeat detection.

A FUSION FRAMEWORK BASED ON CEPSTRAL DOMAIN FEATURES FROM PHONOCARDIOGRAM TO PREDICT HEART HEALTH STATUS

A great increase in the number of cardiovascular cases has been a cause of serious concern for the medical experts all over the world today. In order to achieve valuable risk stratification for patients, early prediction of heart health can benefit specialists to make effective decisions. Heart sound signals help to know about the condition of heart of a patient. Motivated by the success of cepstral features in speech signal classification, authors have used here three different cepstral features, viz. Mel-frequency cepstral coefficients (MFCCs), gammatone frequency cepstral coefficients (GFCCs), and Mel-spectrogram for classifying phonocardiogram into normal and abnormal. Existing research has explored only MFCCs and Mel-feature set extensively for classifying the phonocardiogram. However, in this work, the authors have used a fusion of GFCCs with MFCCs and Mel-spectrogram, and achieved a better accuracy score of 0.96 with sensitivity and specificity scores as 0.91 and 0.98, respectively. The proposed model has been validated on the publicly available benchmark dataset PhysioNet 2016.

Low-Power Audio Keyword Spotting Using Tsetlin Machines

The emergence of artificial intelligence (AI) driven keyword spotting (KWS) technologies has revolutionized human to machine interaction. Yet, the challenge of end-to-end energy efficiency, memory footprint and system complexity of current neural network (NN) powered AI-KWS pipelines has remained ever present. This paper evaluates KWS utilizing a learning automata powered machine learning algorithm called the Tsetlin Machine (TM). Through significant reduction in parameter requirements and choosing logic over arithmetic-based processing, the TM offers new opportunities for low-power KWS while maintaining high learning efficacy. In this paper, we explore a TM-based keyword spotting (KWS) pipeline to demonstrate low complexity with faster rate of convergence compared to NNs. Further, we investigate the scalability with increasing keywords and explore the potential for enabling low-power on-chip KWS.

Lightweight End-to-End Neural Network Model for Automatic Heart Sound Classification

Heart sounds play an important role in the initial screening of heart diseases. However, the accurate diagnosis with heart sound signals requires doctors to have many years of clinical experience and relevant professional knowledge. In this study, we proposed an end-to-end lightweight neural network model that does not require heart sound segmentation and has very few parameters. We segmented the original heart sound signal and performed a short-time Fourier transform (STFT) to obtain the frequency domain features. These features were sent to the improved two-dimensional convolutional neural network (CNN) model for features learning and classification. Considering the imbalance of positive and negative samples, we introduced FocalLoss as the loss function, verified our network model with multiple random verifications, and, hence, obtained a better classification result. Our main purpose is to design a lightweight network structure that is easy for hardware implementation. Compared with the results of the latest literature, our model only uses 4.29 K parameters, which is 1/10 of the size of the state-of-the-art work.

Identification of Heart Sounds with an Interpretable Evolving Fuzzy Neural Network

Heart problems are responsible for the majority of deaths worldwide. The use of intelligent techniques to assist in the identification of existing patterns in these diseases can facilitate treatments and decision making in the field of medicine. This work aims to extract knowledge from a dataset based on heart noise behaviors in order to determine whether heart murmur predilection exists or not in the analyzed patients. A heart murmur can be pathological due to defects in the heart, so the use of an evolving hybrid technique can assist in detecting this comorbidity team, and at the same time, extract knowledge through fuzzy linguistic rules, facilitating the understanding of the nature of the evaluated data. Heart disease detection tests were performed to compare the proposed hybrid model's performance with state of the art for the subject. The results obtained (90.75% accuracy) prove that in addition to great assertiveness in detecting heart murmurs, the evolving hybrid model could be concomitant with the extraction of knowledge from data submitted to an intelligent approach.