Heart sound classification based on improved MFCC features and convolutional recurrent neural networks

Improved recovery of cardiac auscultation sounds using modified cosine transform and LSTM-based masking

Obtaining accurate cardiac auscultation signals, including basic heart sounds (S1 and S2) and subtle signs of disease, is crucial for improving cardiac diagnoses and making the most of telehealth. This research paper introduces an innovative approach that utilizes a modified cosine transform (MCT) and a masking strategy based on long short-term memory (LSTM) to effectively distinguish heart sounds and murmurs from background noise and interfering sounds. The MCT is used to capture the repeated pattern of the heart sounds, while the LSTMs are trained to construct masking based on the repeated MCT spectrum. The proposed strategy's performance in maintaining the clinical relevance of heart sounds continues to demonstrate effectiveness, even in environments marked by increased noise and complex disruptions. The present work highlights the clinical significance and reliability of the suggested methodology through in-depth signal visualization and rigorous statistical performance evaluations. In comparative assessments, the proposed approach has demonstrated superior performance compared to recent algorithms, such as LU-Net and PC-DAE. Furthermore, the system's adaptability to various datasets enhances its reliability and practicality. The suggested method is a potential way to improve the accuracy of cardiovascular diagnostics in an era of rapid advancement in medical signal processing. The proposed approach showed an enhancement in the average signal-to-noise ratio (SNR) by 9.6 dB at an input SNR of - 6 dB and by 3.3 dB at an input SNR of 10 dB. The average signal distortion ratio (SDR) achieved across a variety of input SNR values was 8.56 dB.

Effects of precise cardio sounds on the success rate of phonocardiography

This work investigates whether inclusion of the low-frequency components of heart sounds can increase the accuracy, sensitivity and specificity of diagnosis of cardiovascular disorders. We standardized the measurement method to minimize changes in signal characteristics. We used the Continuous Wavelet Transform to analyze changing frequency characteristics over time and to allocate frequencies appropriately between the low-frequency and audible frequency bands. We used a Convolutional Neural Network (CNN) and deep-learning (DL) for image classification, and a CNN equipped with long short-term memory to enable sequential feature extraction. The accuracy of the learning model was validated using the PhysioNet 2016 CinC dataset, then we used our collected dataset to show that incorporating low-frequency components in the dataset increased the DL model's accuracy by 2% and sensitivity by 4%. Furthermore, the LSTM layer was 0.8% more accurate than the dense layer.

Automated detection of myocardial infarction based on an improved state refinement module for LSTM/GRU

Myocardial infarction (MI) is a common cardiovascular disease caused by the blockages of coronary arteries. The visual inspection of electrocardiogram (ECG) is the main diagnosis pattern, while it is taxing and time-consuming. Motivated from state refinement module for long short term memory (SRM-LSTM), we proposed two improved state refinement frameworks based on LSTM and gated recurrent unit (GRU) called ISRM-LSTM and ISRM-GRU. Both are capable of adaptively refining current states of sample points in ECG with a message passing mechanism than existing LSTM. To evaluate the validity, both are installed into convolutional network architecture and standard LSTM, GRU and Residual networks are employed as control groups across the Physikalisch-Technische Bundesanstalt database. Empirical results confirm noticeable performance improvements than control groups and several existing algorithms with an accuracy of 99.1%. To our knowledge, both modules are the first attempt to consider the interaction characteristics into deep network and improve interpretability exhibiting considerable potentials on lightweight devices thanks to only utilization of three channel ECGs.

Hybrid Sneaky algorithm-based deep neural networks for Heart sound classification using phonocardiogram

In the diagnosis of cardiac disorders Heart sound has a major role, and early detection is crucial to safeguard the patients. Computerized strategies of heart sound classification advocate intensive and more exact results in a quick and better manner. Using a hybrid optimization-controlled deep learning strategy this paper proposed an automatic heart sound classification module. The parameter tuning of the Deep Neural Network (DNN) classifier in a satisfactory manner is the importance of this research which depends on the Hybrid Sneaky optimization algorithm. The developed sneaky optimization algorithm inherits the traits of questing and societal search agents. Moreover, input data from the Phonocardiogram (PCG) database undergoes the process of feature extraction which extract the important features, like statistical, Heart Rate Variability (HRV), and to enhance the performance of this model, the features of Mel frequency Cepstral coefficients (MFCC) are assisted. The developed Sneaky optimization-based DNN classifier's performance is determined in respect of the metrics, namely precision, accuracy, specificity, and sensitivity, which are around 97%, 96.98%, 97%, and 96.9%, respectively.

Role of artificial intelligence in early detection of congenital heart diseases in neonates

In the domain of healthcare, most importantly pediatric healthcare, the role of artificial intelligence (AI) has significantly impacted the medical field. Congenital heart diseases represent a group of heart diseases that are known to be some of the most critical cardiac conditions present at birth. These heart diseases need a swift diagnosis as well as an intervention to ensure the wellbeing of newborns. Fortunately, with the help of AI, including the highly advanced algorithms, analytics and imaging involved, it provides us with a promising era for neonatal care. This article reviewed published data in PubMed, Science Direct, UpToDate, and Google Scholar between the years 2015-2023. To conclude The use of artificial intelligence in detecting congenital heart diseases has shown great promise in improving the accuracy and efficiency of diagnosis. Several studies have demonstrated the efficacy of AI-based approaches for diagnosing congenital heart diseases, with results indicating that the systems can achieve high levels of sensitivity and specificity. In addition, AI can help reduce the workload of healthcare professionals allowing them to focus on other critical aspects of patient care. Despite the potential benefits of using AI, in addition to detecting congenital heart disease, there are still some challenges to overcome, such as the need for large amounts of high-quality data and the requirement for careful validation of the algorithms. Nevertheless, with ongoing research and development, AI is likely to become an increasingly valuable tool for improving the diagnosis and treatment of congenital heart diseases.

Enhancing heart disease prediction using a self-attention-based transformer model

Cardiovascular diseases (CVDs) continue to be the leading cause of more than 17 million mortalities worldwide. The early detection of heart failure with high accuracy is crucial for clinical trials and therapy. Patients will be categorized into various types of heart disease based on characteristics like blood pressure, cholesterol levels, heart rate, and other characteristics. With the use of an automatic system, we can provide early diagnoses for those who are prone to heart failure by analyzing their characteristics. In this work, we deploy a novel self-attention-based transformer model, that combines self-attention mechanisms and transformer networks to predict CVD risk. The self-attention layers capture contextual information and generate representations that effectively model complex patterns in the data. Self-attention mechanisms provide interpretability by giving each component of the input sequence a certain amount of attention weight. This includes adjusting the input and output layers, incorporating more layers, and modifying the attention processes to collect relevant information. This also makes it possible for physicians to comprehend which features of the data contributed to the model's predictions. The proposed model is tested on the Cleveland dataset, a benchmark dataset of the University of California Irvine (UCI) machine learning (ML) repository. Comparing the proposed model to several baseline approaches, we achieved the highest accuracy of 96.51%. Furthermore, the outcomes of our experiments demonstrate that the prediction rate of our model is higher than that of other cutting-edge approaches used for heart disease prediction.

Clustering Methods for Vibro-Acoustic Sensing Features as a Potential Approach to Tissue Characterisation in Robot-Assisted Interventions

This article provides a comprehensive analysis of the feature extraction methods applied to vibro-acoustic signals (VA signals) in the context of robot-assisted interventions. The primary objective is to extract valuable information from these signals to understand tissue behaviour better and build upon prior research. This study is divided into three key stages: feature extraction using the Cepstrum Transform (CT), Mel-Frequency Cepstral Coefficients (MFCCs), and Fast Chirplet Transform (FCT); dimensionality reduction employing techniques such as Principal Component Analysis (PCA), t-Distributed Stochastic Neighbour Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP); and, finally, classification using a nearest neighbours classifier. The results demonstrate that using feature extraction techniques, especially the combination of CT and MFCC with dimensionality reduction algorithms, yields highly efficient outcomes. The classification metrics (Accuracy, Recall, and F1-score) approach 99%, and the clustering metric is 0.61. The performance of the CT-UMAP combination stands out in the evaluation metrics.

Heart Sound Classification Network Based on Convolution and Transformer

Electronic auscultation is vital for doctors to detect symptoms and signs of cardiovascular diseases (CVDs), significantly impacting human health. Although progress has been made in heart sound classification, most existing methods require precise segmentation and feature extraction of heart sound signals before classification. To address this, we introduce an innovative approach for heart sound classification. Our method, named Convolution and Transformer Encoder Neural Network (CTENN), simplifies preprocessing, automatically extracting features using a combination of a one-dimensional convolution (1D-Conv) module and a Transformer encoder. Experimental results showcase the superiority of our proposed method in both binary and multi-class tasks, achieving remarkable accuracies of 96.4%, 99.7%, and 95.7% across three distinct datasets compared with that of similar approaches. This advancement holds promise for enhancing CVD diagnosis and treatment.

A learnable front-end based efficient channel attention network for heart sound classification

Objective. To enhance the accuracy of heart sound classification, this study aims to overcome the limitations of common models which rely on handcrafted feature extraction. These traditional methods may distort or discard crucial pathological information within heart sounds due to their requirement of tedious parameter settings.Approach.We propose a learnable front-end based Efficient Channel Attention Network (ECA-Net) for heart sound classification. This novel approach optimizes the transformation of waveform-to-spectrogram, enabling adaptive feature extraction from heart sound signals without domain knowledge. The features are subsequently fed into an ECA-Net based convolutional recurrent neural network, which emphasizes informative features and suppresses irrelevant information. To address data imbalance, Focal loss is employed in our model.Main results.Using the well-known public PhysioNet challenge 2016 dataset, our method achieved a classification accuracy of 97.77%, outperforming the majority of previous studies and closely rivaling the best model with a difference of just 0.57%.Significance.The learnable front-end facilitates end-to-end training by replacing the conventional heart sound feature extraction module. This provides a novel and efficient approach for heart sound classification research and applications, enhancing the practical utility of end-to-end models in this field.

Parameter-Efficient Densely Connected Dual Attention Network for Phonocardiogram Classification

Cardiac auscultation, exhibited by phonocardiogram (PCG), is a non-invasive and low-cost diagnostic method for cardiovascular diseases (CVDs). However, deploying it in practice is quite challenging, due to the inherent murmurs and a limited number of supervised samples in heart sound data. To solve these problems, not only heart sound analysis based on handcrafted features, but also computer-aided heart sound analysis based on deep learning have been extensively studied in recent years. Though with elaborate design, most of these methods still use additional pre-processing to improve classification performance, which heavily relies on time-consuming experienced engineering. In this article, we propose a parameter-efficient densely connected dual attention network (DDA) for heart sound classification. It combines two advantages simultaneously of the purely end-to-end architecture and enriched contextual representations of the self-attention mechanism. Specifically, the densely connected structure can automatically extract the information flow of heart sound features hierarchically. Alongside, improving contextual modeling capabilities, the dual attention mechanism adaptively aggregates local features with global dependencies via a self-attention mechanism, which captures the semantic interdependencies across position and channel axes respectively. Extensive experiments across stratified 10-fold cross-validation strongly evidence that our proposed DDA model surpasses current 1D deep models on the challenging Cinc2016 benchmark with significant computational efficiency.

Design of ear-contactless stethoscope and improvement in the performance of deep learning based on CNN to classify the heart sound

Cardiac-related disorders are rapidly growing throughout the world. Accurate classification of cardiovascular diseases is an important research topic in healthcare. During COVID-19, auscultating heart sounds was challenging as health workers and doctors wear protective clothing, and direct contact with patients can spread the outbreak. Thus, contactless auscultation of heart sound is necessary. In this paper, a low-cost ear contactless stethoscope is designed where auscultation is done with the help of a bluetooth-enabled micro speaker instead of an earpiece. The PCG recordings are further compared with other standard electronic stethoscopes like Littman 3 M. This work is made to improve the performance of deep learning-based classifiers like recurrent neural networks (RNN) and convolutional neural networks (CNN) for different valvular heart problems using tuning of hyperparameters like learning rate of optimizers, dropout rate, and hidden layer. Hyper-parameter tuning is used to optimize the performances of various deep learning models and their learning curves for real-time analysis. The acoustic, time, and frequency domain features are used in this research. The investigation is made on the heart sounds of normal and diseased patients available from the standard data repository to train the software models. The proposed CNN-based inception network model achieved an accuracy of 99.65 ± 0.06% on the test dataset with a sensitivity of 98.8  ± 0.05% and specificity of 98.2 ± 0.19%. The proposed hybrid CNN-RNN architecture attained 91.17 ± 0.03% accuracy on test data after hyperparameter optimization, whereas the LSTM-based RNN model achieved 82.32 ± 0.11% accuracy. Finally, the evaluated results were compared with machine learning algorithms, and the improved CNN-based Inception Net model is the most effective among others.

Detection of valvular heart diseases combining orthogonal non-negative matrix factorization and convolutional neural networks in PCG signals

BACKGROUND AND OBJECTIVE

Valvular heart disease (VHD) is associated with elevated mortality rates. Although transthoracic echocardiography (TTE) is the gold standard detection tool, phonocardiography (PCG) could be an alternative as it is a cost-effective and noninvasive method for cardiac auscultation. Many researchers have dedicated their efforts to improving the decision-making process and developing robust and precise approaches to assist physicians in providing reliable diagnoses of VHD.

METHODS

This research proposes a novel approach for the detection of anomalous valvular heart sounds from PCG signals. The proposed approach combines orthogonal non-negative matrix factorization (ONMF) and convolutional neural network (CNN) architectures in a three-stage cascade. The aim of the proposal is to improve the learning process by identifying the optimal ONMF temporal or spectral patterns for accurate detection. In the first stage, the time-frequency representation of the input PCG signal is computed. Next, band-pass filtering is performed to locate the spectral range that is most relevant for the presence of such cardiac abnormalities. In the second stage, the temporal and spectral cardiac structures are extracted using the ONMF approach. These structures are utilized in the third stage and fed into the CNN architecture to detect abnormal heart sounds.

RESULTS

Several state-of-the-art CNN architectures, such as LeNet5, AlexNet, ResNet50, VGG16 and GoogLeNet, have been evaluated to determine the effectiveness of using ONMF temporal features for VHD detection. The results reveal that the integration of ONMF temporal features with a CNN classifier significantly improve VHD detection. Specifically, the proposed approach achieves an accuracy improvement of approximately 45% when ONMF spectral features are used and 35% when time-frequency features from the short-time Fourier transform (STFT) spectrogram are used. Additionally, feeding ONMF temporal features into low-complexity CNN architectures yields competitive results comparable to those obtained with complex architectures.

CONCLUSIONS

The temporal structure factorized by ONMF plays a critical role in distinguishing between normal heart sounds and abnormal heart sounds since the repeatability of normal heart cycles is disrupted by the presence of cardiac abnormalities. Consequently, the results highlight the importance of appropriate input data representation in the learning process of CNN models in the biomedical field of valvular heart sound detection.

Cutting Weights of Deep Learning Models for Heart Sound Classification: Introducing a Knowledge Distillation Approach

Cardiovascular diseases (CVDs) are the number one cause of death worldwide. In recent years, intelligent auxiliary diagnosis of CVDs based on computer audition has become a popular research field, and intelligent diagnosis technology is increasingly mature. Neural networks used to monitor CVDs are becoming more complex, requiring more computing power and memory, and are difficult to deploy in wearable devices. This paper proposes a lightweight model for classifying heart sounds based on knowledge distillation, which can be deployed in wearable devices to monitor the heart sounds of wearers. The network model is designed based on Convolutional Neural Networks (CNNs). Model performance is evaluated by extracting Mel Frequency Cepstral Coefficients (MFCCs) features from the PhysioNet/CinC Challenge 2016 dataset. The experimental results show that knowledge distillation can improve a lightweight network's accuracy, and our model performs well on the test set. Especially, when the knowledge distillation temperature is 7 and the weight α is 0.1, the accuracy is 88.5 %, the recall is 83.8 %, and the specificity is 93.6 %.Clinical relevance- A lightweight model of heart sound classification based on knowledge distillation can be deployed on various hardware devices for timely monitoring and feedback of the physical condition of patients with CVDs for timely provision of medical advice. When the model is deployed on the medical instruments of the hospital, the condition of severe and hospitalised patients can be timely fed back and clinical treatment advice can be provided to the clinicians.

Time-Frequency Fragment Selection for Disease Detection from Imbalanced Phonocardiogram Data

Cardiovascular disease (CVD) has become the most concerning disease worldwide. A Phonocardiogram (PCG), the graphical representation of heart sound, is a non-invasive method that helps to detect CVD by analyzing its characteristics. Several machine learning (ML) approaches have been proposed in the last decade to assist practitioners in interpreting this disease accurately. However, the ML-based method requires a considerable amount of PCG data with a balance between data categories for unbiased performance. Moreover, PCG data in the literature is scarce, and the available database has a strong imbalance between the normal and abnormal categories. This data imbalance causes outcomes to be severely biased towards classes with greater samples. This work proposes a variable-hop fragment selection method with a pre-trained CNN model to counter the issues of data scarcity and imbalance. The proposed framework improves 7.12% of unweighted average recall (UAR) value for assessing an imbalanced PCG dataset compared to the state-of-the-art method and reports an overall UAR of 92.46% on the PhysioNet/CinC Challenge 2016 dataset. The improved performance signifies the clinical relevance of the work providing reliable assistance for heart auscultation and has the potential to screen for heart pathologies in data constraint applications.

Al-Biruni Earth Radius Optimization with Transfer Learning Based Histopathological Image Analysis for Lung and Colon Cancer Detection

An early diagnosis of lung and colon cancer (LCC) is critical for improved patient outcomes and effective treatment. Histopathological image (HSI) analysis has emerged as a robust tool for cancer diagnosis. HSI analysis for a LCC diagnosis includes the analysis and examination of tissue samples attained from the LCC to recognize lesions or cancerous cells. It has a significant role in the staging and diagnosis of this tumor, which aids in the prognosis and treatment planning, but a manual analysis of the image is subject to human error and is also time-consuming. Therefore, a computer-aided approach is needed for the detection of LCC using HSI. Transfer learning (TL) leverages pretrained deep learning (DL) algorithms that have been trained on a larger dataset for extracting related features from the HIS, which are then used for training a classifier for a tumor diagnosis. This manuscript offers the design of the Al-Biruni Earth Radius Optimization with Transfer Learning-based Histopathological Image Analysis for Lung and Colon Cancer Detection (BERTL-HIALCCD) technique. The purpose of the study is to detect LCC effectually in histopathological images. To execute this, the BERTL-HIALCCD method follows the concepts of computer vision (CV) and transfer learning for accurate LCC detection. When using the BERTL-HIALCCD technique, an improved ShuffleNet model is applied for the feature extraction process, and its hyperparameters are chosen by the BER system. For the effectual recognition of LCC, a deep convolutional recurrent neural network (DCRNN) model is applied. Finally, the coati optimization algorithm (COA) is exploited for the parameter choice of the DCRNN approach. For examining the efficacy of the BERTL-HIALCCD technique, a comprehensive group of experiments was conducted on a large dataset of histopathological images. The experimental outcomes demonstrate that the combination of AER and COA algorithms attain an improved performance in cancer detection over the compared models.

Classifying Heart-Sound Signals Based on CNN Trained on MelSpectrum and Log-MelSpectrum Features

The intelligent classification of heart-sound signals can assist clinicians in the rapid diagnosis of cardiovascular diseases. Mel-frequency cepstral coefficients (MelSpectrums) and log Mel-frequency cepstral coefficients (Log-MelSpectrums) based on a short-time Fourier transform (STFT) can represent the temporal and spectral structures of original heart-sound signals. Recently, various systems based on convolutional neural networks (CNNs) trained on the MelSpectrum and Log-MelSpectrum of segmental heart-sound frames that outperform systems using handcrafted features have been presented and classified heart-sound signals accurately. However, there is no a priori evidence of the best input representation for classifying heart sounds when using CNN models. Therefore, in this study, the MelSpectrum and Log-MelSpectrum features of heart-sound signals combined with a mathematical model of cardiac-sound acquisition were analysed theoretically. Both the experimental results and theoretical analysis demonstrated that the Log-MelSpectrum features can reduce the classification difference between domains and improve the performance of CNNs for heart-sound classification.

Tree-structured neural networks: Spatiotemporal dynamics and optimal control

How the network topology drives the response dynamic is a basic question that has not yet been fully answered in neural networks. Elucidating the internal relation between topological structures and dynamics is instrumental in our understanding of brain function. Recent studies have revealed that the ring structure and star structure have a great influence on the dynamical behavior of neural networks. In order to further explore the role of topological structures in the response dynamic, we construct a new tree structure that differs from the ring structure and star structure of traditional neural networks. Considering the diffusion effect, we propose a diffusion neural network model with binary tree structure and multiple delays. How to design control strategies to optimize brain function has also been an open question. Thus, we put forward a novel full-dimensional nonlinear state feedback control strategy to optimize relevant neurodynamics. Some conditions about the local stability and Hopf bifurcation are obtained, and it is proved that the Turing instability does not occur. Moreover, for the formation of the spatially homogeneous periodic solution, some diffusion conditions are also fused together. Finally, several numerical examples are carried out to illustrate the results' correctness. Meanwhile, some comparative experiments are rendered to reveal the effectiveness of the proposed control strategy.

Aortic Stenosis Detection Using Spectral Statistical Features of Heart Sound Signals

INTRODUCTION: Aortic stenosis (AS) is a severe complicated heart valve disease. This valve abnormality is a slow-progressive condition and mostly asymptomatic. Hence, there is a need for a rapid non-invasive AS diagnosis method with minimal feature extraction.  OBJECTIVE: In this paper, we proposed a rapid spectral analysis-based statistical feature extraction method to identify the AS stages with the minimum number of features. METHODS: In this study, the heart sound signals were collected from the medical database and transformed into the frequency domain for further spectral feature analysis. We used the windowing technique to condition the heart signals before spectral analysis. The spectral statistical features were extracted from the computed frequency spectrum. The range of statistical features was compared for normal, early, and delayed AS groups. RESULTS: In experiments, the normal, early, and delayed AS heart sound signals were used. The experimental results show the statistical difference between the normal and AS heart sound signal spectrums. The normal/unhealthy condition of a heart was identified using the statistical features of the frequency spectrum. CONCLUSION:  The experimental results confirmed that the statistical features derived from the heart sound signal spectrums were varied according to the AS condition. Hence, the spectral statistical features can be considered as a rapid predictor of AS.

Heart sound classification based on equal scale frequency cepstral coefficients and deep learning

Heart diseases represent a serious medical condition that can be fatal. Therefore, it is critical to investigate the measures of its early prevention. The Mel-scale frequency cepstral coefficients (MFCC) feature has been widely used in the early diagnosis of heart abnormity and achieved promising results. During feature extraction, the Mel-scale triangular overlapping filter set is applied, which makes the frequency response more in line with the human auditory property. However, the frequency of the heart sound signals has no specific relationship with the human auditory system, which may not be suitable for processing of heart sound signals. To overcome this issue and obtain a more objective feature that can better adapt to practical use, in this work, we propose an equal scale frequency cepstral coefficients (EFCC) feature based on replacing the Mel-scale filter set with a set of equally spaced triangular overlapping filters. We further designed classifiers combining convolutional neural network (CNN), recurrent neural network (RNN) and random forest (RF) layers, which can extract both the spatial and temporal information of the input features. We evaluated the proposed algorithm on our database and the PhysioNet Computational Cardiology (CinC) 2016 Challenge Database. Results from ten-fold cross-validation reveal that the EFCC-based features show considerably better performance and robustness than the MFCC-based features on the task of classifying heart sounds from novel patients. Our algorithm can be further used in wearable medical devices to monitor the heart status of patients in real time with high precision, which is of great clinical importance.

A machine learning approach to the development and prospective evaluation of a pediatric lung sound classification model

Auscultation, a cost-effective and non-invasive part of physical examination, is essential to diagnose pediatric respiratory disorders. Electronic stethoscopes allow transmission, storage, and analysis of lung sounds. We aimed to develop a machine learning model to classify pediatric respiratory sounds. Lung sounds were digitally recorded during routine physical examinations at a pediatric pulmonology outpatient clinic from July to November 2019 and labeled as normal, crackles, or wheezing. Ensemble support vector machine models were trained and evaluated for four classification tasks (normal vs. abnormal, crackles vs. wheezing, normal vs. crackles, and normal vs. wheezing) using K-fold cross-validation (K = 10). Model performance on a prospective validation set (June to July 2021) was compared with those of pediatricians and non-pediatricians. Total 680 clips were used for training and internal validation. The model accuracies during internal validation for normal vs. abnormal, crackles vs. wheezing, normal vs. crackles, and normal vs. wheezing were 83.68%, 83.67%, 80.94%, and 90.42%, respectively. The prospective validation (n = 90) accuracies were 82.22%, 67.74%, 67.80%, and 81.36%, respectively, which were comparable to pediatrician and non-pediatrician performance. An automated classification model of pediatric lung sounds is feasible and maybe utilized as a screening tool for respiratory disorders in this pandemic era.

Heart sound classification based on improved mel-frequency spectral coefficients and deep residual learning

Heart sound classification plays a critical role in the early diagnosis of cardiovascular diseases. Although there have been many advances in heart sound classification in the last few years, most of them are still based on conventional segmented features and shallow structure-based classifiers. Therefore, we propose a new heart sound classification method based on improved mel-frequency cepstrum coefficient features and deep residual learning. Firstly, the heart sound signal is preprocessed, and its improved features are computed. Then, these features are used as input features of the neural network. The pathological information in the heart sound signal is further extracted by the deep residual network. Finally, the heart sound signal is classified into different categories according to the features learned by the neural network. This paper presents comprehensive analyses of different network parameters and network connection strategies. The proposed method achieves an accuracy of 94.43% on the dataset in this paper.

Classification of Heart Sounds Using Chaogram Transform and Deep Convolutional Neural Network Transfer Learning

Heart sounds convey important information regarding potential heart diseases. Currently, heart sound classification attracts many researchers from the fields of telemedicine, digital signal processing, and machine learning-among others-mainly to identify cardiac pathology as quickly as possible. This article proposes chaogram as a new transform to convert heart sound signals to colour images. In the proposed approach, the output image is, therefore, the projection of the reconstructed phase space representation of the phonocardiogram (PCG) signal on three coordinate planes. This has two major benefits: (1) it makes possible to apply deep convolutional neural networks to heart sounds and (2) it is also possible to employ a transfer learning scheme by converting a heart sound signal to an image. The performance of the proposed approach was verified on the PhysioNet dataset. Due to the imbalanced data on this dataset, it is common to assess the results quality using the average of sensitivity and specificity, which is known as score, instead of accuracy. In this study, the best results were achieved using the InceptionV3 model, which achieved a score of 88.06%.

Cardiac anomaly detection considering an additive noise and convolutional distortion model of heart sound recordings

Cardiac auscultation is an essential point-of-care method used for the early diagnosis of heart diseases. Automatic analysis of heart sounds for abnormality detection is faced with the challenges of additive noise and sensor-dependent degradation. This paper aims to develop methods to address the cardiac abnormality detection problem when both of these components are present in the cardiac auscultation sound. We first mathematically analyze the effect of additive noise and convolutional distortion on short-term mel-filterbank energy-based features and a Convolutional Neural Network (CNN) layer. Based on the analysis, we propose a combination of linear and logarithmic spectrogram-image features. These 2D features are provided as input to a residual CNN network (ResNet) for heart sound abnormality detection. Experimental validation is performed first on an open-access, multiclass heart sound dataset where we analyzed the effect of additive noise by mixing lung sound noise with the recordings. In noisy conditions, the proposed method outperforms one of the best-performing methods in the literature achieving an Macc (mean of sensitivity and specificity) of 89.55% and an average F-1 score of 82.96%, respectively, when averaged over all noise levels. Next, we perform heart sound abnormality detection (binary classification) experiments on the 2016 Physionet/CinC Challenge dataset that involves noisy recordings obtained from multiple stethoscope sensors. The proposed method achieves significantly improved results compared to the conventional approaches on this dataset, in the presence of both additive noise and channel distortion, with an area under the ROC (receiver operating characteristics) curve (AUC) of 91.36%, F-1 score of 84.09%, and Macc of 85.08%. We also show that the proposed method shows the best mean accuracy across different source domains, including stethoscope and noise variability, demonstrating its effectiveness in different recording conditions. The proposed combination of linear and logarithmic features along with the ResNet classifier effectively minimizes the impact of background noise and sensor variability for classifying phonocardiogram (PCG) signals. The method thus paves the way toward developing computer-aided cardiac auscultation systems in noisy environments using low-cost stethoscopes.

Classification of audio signals using spectrogram surfaces and extrinsic distortion measures

Representation of one-dimensional (1D) signals as surfaces and higher-dimensional manifolds reveals geometric structures that can enhance assessment of signal similarity and classification of large sets of signals. Motivated by this observation, we propose a novel robust algorithm for extraction of geometric features, by mapping the obtained geometric objects into a reference domain. This yields a set of highly descriptive features that are instrumental in feature engineering and in analysis of 1D signals. Two examples illustrate applications of our approach to well-structured audio signals: Lung sounds were chosen because of the interest in respiratory pathologies caused by the coronavirus and environmental conditions; accent detection was selected as a challenging speech analysis problem. Our approach outperformed baseline models under all measured metrics. It can be further extended by considering higher-dimensional distortion measures. We provide access to the code for those who are interested in other applications and different setups (Code: https://github.com/jeremy-levy/Classification-of-audio-signals-using-spectrogram-surfaces-and-extrinsic-distortion-measures).

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1186/s13634-022-00933-9.

Diagnosis of cardiac abnormalities based on phonocardiogram using a novel fuzzy matching feature extraction method

BACKGROUND

The diagnosis of cardiac abnormalities based on heart sound signal is a research hotspot in recent years. The early diagnosis of cardiac abnormalities has a crucial significance for the treatment of heart diseases.

METHODS

For the sake of achieving more practical clinical applications of automatic recognition of cardiac abnormalities, here we proposed a novel fuzzy matching feature extraction method. First of all, a group of Gaussian wavelets are selected and then optimized based on a template signal. Convolutional features of test signal and the template signal are then computed. Matching degree and matching energy features between template signal and test signal in time domain and frequency domain are then extracted. To test performance of proposed feature extraction method, machine learning algorithms such as K-nearest neighbor, support vector machine, random forest and multilayer perceptron with grid search parameter optimization are constructed to recognize heart disease using the extracted features based on phonocardiogram signals.

RESULTS

As a result, we found that the best classification accuracy of random forest reaches 96.5% under tenfold cross validation using the features extracted by the proposed method. Further, Mel-Frequency Cepstral Coefficients of phonocardiogram signals combing with features extracted by our algorithm are evaluated. Accuracy, sensitivity and specificity of integrated features reaches 99.0%, 99.4% and 99.7% respectively when using support vector machine, which achieves the best performance among all reported algorithms based on the same dataset. On several common features, we used independent sample t-tests. The results revealed that there are significant differences (p < 0.05) between 5 categories.

CONCLUSION

It can be concluded that our proposed fuzzy matching feature extraction method is a practical approach to extract powerful and interpretable features from one-dimensional signals for heart sound diagnostics and other pattern recognition task.

Korotkoff sounds dynamically reflect changes in cardiac function based on deep learning methods

Korotkoff sounds (K-sounds) have been around for over 100 years and are considered the gold standard for blood pressure (BP) measurement. K-sounds are also unique for the diagnosis and treatment of cardiovascular diseases; however, their efficacy is limited. The incidences of heart failure (HF) are increasing, which necessitate the development of a rapid and convenient pre-hospital screening method. In this review, we propose a deep learning (DL) method and the possibility of using K-methods to predict cardiac function changes for the detection of cardiac dysfunctions.

A lightweight hybrid deep learning system for cardiac valvular disease classification

Cardiovascular diseases (CVDs) are a prominent cause of death globally. The introduction of medical big data and Artificial Intelligence (AI) technology encouraged the effort to develop and deploy deep learning models for distinguishing heart sound abnormalities. These systems employ phonocardiogram (PCG) signals because of their lack of sophistication and cost-effectiveness. Automated and early diagnosis of cardiovascular diseases (CVDs) helps alleviate deadly complications. In this research, a cardiac diagnostic system that combined CNN and LSTM components was developed, it uses phonocardiogram (PCG) signals, and utilizes either augmented or non-augmented datasets. The proposed model discriminates five heart valvular conditions, namely normal, Aortic Stenosis (AS), Mitral Regurgitation (MR), Mitral Stenosis (MS), and Mitral Valve Prolapse (MVP). The findings demonstrate that the suggested end-to-end architecture yields outstanding performance concerning all important evaluation metrics. For the five classes problem using the open heart sound dataset, accuracy was 98.5%, F1-score was 98.501%, and Area Under the Curve (AUC) was 0.9978 for the non-augmented dataset and accuracy was 99.87%, F1-score was 99.87%, and AUC was 0.9985 for the augmented dataset. Model performance was further evaluated using the PhysioNet/Computing in Cardiology 2016 challenge dataset, for the two classes problem, accuracy was 93.76%, F1-score was 85.59%, and AUC was 0.9505. The achieved results show that the proposed system outperforms all previous works that use the same audio signal databases. In the future, the findings will help build a multimodal structure that uses both PCG and ECG signals.

LightFD: Real-Time Fault Diagnosis with Edge Intelligence for Power Transformers

Power fault monitoring based on acoustic waves has gained a great deal of attention in industry. Existing methods for fault diagnosis typically collect sound signals on site and transmit them to a back-end server for analysis, which may fail to provide a real-time response due to transmission packet loss and latency. However, the limited computing power of edge devices and the existing methods for feature extraction pose a significant challenge to performing diagnosis on the edge. In this paper, we propose a fast Lightweight Fault Diagnosis method for power transformers, referred to as LightFD, which integrates several technical components. Firstly, before feature extraction, we design an asymmetric Hamming-cosine window function to reduce signal spectrum leakage and ensure data integrity. Secondly, we design a multidimensional spatio-temporal feature extraction method to extract acoustic features. Finally, we design a parallel dual-layer, dual-channel lightweight neural network to realize the classification of different fault types on edge devices with limited computing power. Extensive simulation and experimental results show that the diagnostic precision and recall of LightFD reach 94.64% and 95.33%, which represent an improvement of 4% and 1.6% over the traditional SVM method, respectively.

A multimodal parallel method for left ventricular dysfunction identification based on phonocardiogram and electrocardiogram signals synchronous analysis

Heart failure (HF) is widely acknowledged as the terminal stage of cardiac disease and represents a global clinical and public health problem. Left ventricular ejection fraction (LVEF) measured by echocardiography is an important indicator of HF diagnosis and treatment. Early identification of LVEF reduction and early treatment is of great significance to improve LVEF and the prognosis of HF. This research aims to introduce a new method for left ventricular dysfunction (LVD) identification based on phonocardiogram (ECG) and electrocardiogram (PCG) signals synchronous analysis. In the present study, we established a database called Synchronized ECG and PCG Database for Patients with Left Ventricular Dysfunction (SEP-LVDb) consisting of 1046 synchronous ECG and PCG recordings from patients with reduced (n = 107) and normal (n = 699) LVEF. 173 and 873 recordings were available from the reduced and normal LVEF group, respectively. Then, we proposed a parallel multimodal method for LVD identification based on synchronous analysis of PCG and ECG signals. Two-layer bidirectional gate recurrent unit (Bi-GRU) was used to extract features in the time domain, and the data were classified using residual network 18 (ResNet-18). This research confirmed that fused ECG and PCG signals yielded better performance than ECG or PCG signals alone, with an accuracy of 93.27%, precision of 93.34%, recall of 93.27%, and F1-score of 93.27%. Verification of the model's performance with an independent dataset achieved an accuracy of 80.00%, precision of 79.38%, recall of 80.00% and F1-score of 78.67%. The Bi-GRU model outperformed Bi-directional long short-term memory (Bi-LSTM) and recurrent neural network (RNN) models with a best selection frame length of 3.2 s. The Saliency Maps showed that SEP-LVDPN could effectively learn features from the data.

Heart Sound Classification based on Residual Shrinkage Networks

Heart sound classification is one of the non-invasive methods for early detection of the cardiovascular diseases (CVDs), the leading cause for deaths. In recent years, Computer Audition (CA) technology has become increasingly sophisticated, auxiliary diagnosis technology of heart disease based on CA has become a popular research area. This paper proposes a deep Convolutional Neural Network (CNN) model for heart sound classification. To improve the classification accuracy of heart sound, we design a classification algorithm combining classical Residual Network (ResNet) and Long Short-Term Memory (LSTM). The model performance is evaluated in the PhysioNet/CinC Challenges 2016 datasets using a 2D time-frequency feature. We extract the four features from different filter-bank coefficients, including Filterbank (Fbank), Mel-Frequency Spectral Coefficients (MFSCs), and Mel-Frequency Cepstral Coefficients (MFCCs). The experimental results show the MFSCs feature outperforms the other features in the proposed CNN model. The proposed model performs well on the test set, particularly the F1 score of 84.3 % - the accuracy of 84.4 %, the sensitivity of 84.3 %, and the specificity of 85.6 %. Compared with the classical ResNet model, an accuracy of 4.9 % improvement is observed in the proposed model.

Classification of Children's Heart Sounds With Noise Reduction Based on Variational Modal Decomposition

Purpose

Children's heart sounds were denoised to improve the performance of the intelligent diagnosis.

Methods

A combined noise reduction method based on variational modal decomposition (VMD) and wavelet soft threshold algorithm (WST) was proposed, and used to denoise 103 phonocardiogram samples. Features were extracted after denoising and employed for an intelligent diagnosis model to verify the effect of the denoising method.

Results

The noise in children's phonocardiograms, especially crying noise, was suppressed. The signal-to-noise ratio obtained by the method for normal heart sounds was 14.69 dB at 5 dB Gaussian noise, which was higher than that obtained by WST only and the other VMD denoising method. Intelligent classification showed that the accuracy, sensitivity and specificity of the classification system for congenital heart diseases were 92.23, 92.42, and 91.89%, respectively and better than those with WST only.

Conclusion

The proposed noise reduction method effectively eliminates noise in children's phonocardiograms and improves the performance of intelligent screening for the children with congenital heart diseases.

A deep neural network using audio files for detection of aortic stenosis

BACKGROUND

Although aortic stenosis (AS) is the most common valvular heart disease in the western world, many affected patients remain undiagnosed. Auscultation is a readily available screening tool for AS. However, it requires a high level of professional expertise.

HYPOTHESIS

An AI algorithm can detect AS using audio files with the same accuracy as experienced cardiologists.

METHODS

A deep neural network (DNN) was trained by preprocessed audio files of 100 patients with AS and 100 controls. The DNN's performance was evaluated with a test data set of 40 patients. The primary outcome measures were sensitivity, specificity, and F1-score. Results of the DNN were compared with the performance of cardiologists, residents, and medical students.

RESULTS

Eighteen percent of patients without AS and 22% of patients with AS showed an additional moderate or severe mitral regurgitation. The DNN showed a sensitivity of 0.90 (0.81-0.99), a specificity of 1, and an F1-score of 0.95 (0.89-1.0) for the detection of AS. In comparison, we calculated an F1-score of 0.94 (0.86-1.0) for cardiologists, 0.88 (0.78-0.98) for residents, and 0.88 (0.78-0.98) for students.

CONCLUSIONS

The present study shows that deep learning-guided auscultation predicts significant AS with similar accuracy as cardiologists. The results of this pilot study suggest that AI-assisted auscultation may help general practitioners without special cardiology training in daily practice.

Toward Realigning Automatic Speaker Verification in the Era of COVID-19

The use of face masks has increased dramatically since the COVID-19 pandemic started in order to to curb the spread of the disease. Additionally, breakthrough infections caused by the Delta and Omicron variants have further increased the importance of wearing a face mask, even for vaccinated individuals. However, the use of face masks also induces attenuation in speech signals, and this change may impact speech processing technologies, e.g., automated speaker verification (ASV) and speech to text conversion. In this paper we examine Automatic Speaker Verification (ASV) systems against the speech samples in the presence of three different types of face mask: surgical, cloth, and filtered N95, and analyze the impact on acoustics and other factors. In addition, we explore the effect of different microphones, and distance from the microphone, and the impact of face masks when speakers use ASV systems in real-world scenarios. Our analysis shows a significant deterioration in performance when an ASV system encounters different face masks, microphones, and variable distance between the subject and microphone. To address this problem, this paper proposes a novel framework to overcome performance degradation in these scenarios by realigning the ASV system. The novelty of the proposed ASV framework is as follows: first, we propose a fused feature descriptor by concatenating the novel Ternary Deviated overlapping Patterns (TDoP), Mel Frequency Cepstral Coefficients (MFCC), and Gammatone Cepstral Coefficients (GTCC), which are used by both the ensemble learning-based ASV and anomaly detection system in the proposed ASV architecture. Second, this paper proposes an anomaly detection model for identifying vocal samples produced in the presence of face masks. Next, it presents a Peak Norm (PN) filter to approximate the signal of the speaker without a face mask in order to boost the accuracy of ASV systems. Finally, the features of filtered samples utilizing the PN filter and samples without face masks are passed to the proposed ASV to test for improved accuracy. The proposed ASV system achieved an accuracy of 0.99 and 0.92, respectively, on samples recorded without a face mask and with different face masks. Although the use of face masks affects the ASV system, the PN filtering solution overcomes this deficiency up to 4%. Similarly, when exposed to different microphones and distances, the PN approach enhanced system accuracy by up to 7% and 9%, respectively. The results demonstrate the effectiveness of the presented framework against an in-house prepared, diverse Multi Speaker Face Masks (MSFM) dataset, (IRB No. FY2021-83), consisting of samples of subjects taken with a variety of face masks and microphones, and from different distances.

An automatic approach for heart failure typing based on heart sounds and convolutional recurrent neural networks

Heart failure (HF) is a complex clinical syndrome that poses a major hazard to human health. Patients with different types of HF have great differences in pathogenesis and treatment options. Therefore, HF typing is of great significance for timely treatment of patients. In this paper, we proposed an automatic approach for HF typing based on heart sounds (HS) and convolutional recurrent neural networks, which provides a new non-invasive and convenient way for HF typing. Firstly, the collected HS signals were preprocessed with adaptive wavelet denoising. Then, the logistic regression based hidden semi-Markov model was utilized to segment HS frames. For the distinction between normal subjects and the HF patients with preserved ejection fraction or reduced ejection fraction, a model based on convolutional neural network and recurrent neural network was built. The model can automatically learn the spatial and temporal characteristics of HS signals. The results show that the proposed model achieved a superior performance with an accuracy of 97.64%. This study suggests the proposed method could be a useful tool for HF recognition and as a supplement for HF typing.

Assessment of Dual-Tree Complex Wavelet Transform to Improve SNR in Collaboration with Neuro-Fuzzy System for Heart-Sound Identification

The research paper proposes a novel denoising method to improve the outcome of heart-sound (HS)-based heart-condition identification by applying the dual-tree complex wavelet transform (DTCWT) together with the adaptive neuro-fuzzy inference System (ANFIS) classifier. The method consists of three steps: first, preprocessing to eliminate 50 Hz noise; second, applying four successive levels of DTCWT to denoise and reconstruct the time-domain HS signal; third, to evaluate ANFIS on a total of 2735 HS recordings from an international dataset (PhysioNet Challenge 2016). The results show that the signal-to-noise ratio (SNR) with DTCWT was significantly improved (p < 0.001) as compared to original HS recordings. Quantitatively, there was an 11% to many decibel (dB)-fold increase in SNR after DTCWT, representing a significant improvement in denoising HS. In addition, the ANFIS, using six time-domain features, resulted in 55–86% precision, 51–98% recall, 53–86% f-score, and 54–86% MAcc compared to other attempts on the same dataset. Therefore, DTCWT is a successful technique in removing noise from biosignals such as HS recordings. The adaptive property of ANFIS exhibited capability in classifying HS recordings.

The Effect of Signal Duration on the Classification of Heart Sounds: A Deep Learning Approach

Deep learning techniques are the future trend for designing heart sound classification methods, making conventional heart sound segmentation dispensable. However, despite using fixed signal duration for training, no study has assessed its effect on the final performance in detail. Therefore, this study aims at analysing the duration effect on the commonly used deep learning methods to provide insight for future studies in data processing, classifier, and feature selection. The results of this study revealed that (1) very short heart sound signal duration (1 s) weakens the performance of Recurrent Neural Networks (RNNs), whereas no apparent decrease in the tested Convolutional Neural Network (CNN) model was found. (2) RNN outperformed CNN using Mel-frequency cepstrum coefficients (MFCCs) as features. There was no difference between RNN models (LSTM, BiLSTM, GRU, or BiGRU). (3) Adding dynamic information (∆ and ∆²MFCCs) of the heart sound as a feature did not improve the RNNs' performance, and the improvement on CNN was also minimal (≤2.5% in MAcc). The findings provided a theoretical basis for further heart sound classification using deep learning techniques when selecting the input length.

Feature-Based Fusion Using CNN for Lung and Heart Sound Classification

Lung or heart sound classification is challenging due to the complex nature of audio data, its dynamic properties of time, and frequency domains. It is also very difficult to detect lung or heart conditions with small amounts of data or unbalanced and high noise in data. Furthermore, the quality of data is a considerable pitfall for improving the performance of deep learning. In this paper, we propose a novel feature-based fusion network called FDC-FS for classifying heart and lung sounds. The FDC-FS framework aims to effectively transfer learning from three different deep neural network models built from audio datasets. The innovation of the proposed transfer learning relies on the transformation from audio data to image vectors and from three specific models to one fused model that would be more suitable for deep learning. We used two publicly available datasets for this study, i.e., lung sound data from ICHBI 2017 challenge and heart challenge data. We applied data augmentation techniques, such as noise distortion, pitch shift, and time stretching, dealing with some data issues in these datasets. Importantly, we extracted three unique features from the audio samples, i.e., Spectrogram, MFCC, and Chromagram. Finally, we built a fusion of three optimal convolutional neural network models by feeding the image feature vectors transformed from audio features. We confirmed the superiority of the proposed fusion model compared to the state-of-the-art works. The highest accuracy we achieved with FDC-FS is 99.1% with Spectrogram-based lung sound classification while 97% for Spectrogram and Chromagram based heart sound classification.