Deep Learning Methods for Heart Sounds Classification: A Systematic Review

Effects of precise cardio sounds on the success rate of phonocardiography

This work investigates whether inclusion of the low-frequency components of heart sounds can increase the accuracy, sensitivity and specificity of diagnosis of cardiovascular disorders. We standardized the measurement method to minimize changes in signal characteristics. We used the Continuous Wavelet Transform to analyze changing frequency characteristics over time and to allocate frequencies appropriately between the low-frequency and audible frequency bands. We used a Convolutional Neural Network (CNN) and deep-learning (DL) for image classification, and a CNN equipped with long short-term memory to enable sequential feature extraction. The accuracy of the learning model was validated using the PhysioNet 2016 CinC dataset, then we used our collected dataset to show that incorporating low-frequency components in the dataset increased the DL model's accuracy by 2% and sensitivity by 4%. Furthermore, the LSTM layer was 0.8% more accurate than the dense layer.

Identifying pediatric heart murmurs and distinguishing innocent from pathologic using deep learning

OBJECTIVE

To develop a deep learning algorithm to perform multi-class classification of normal pediatric heart sounds, innocent murmurs, and pathologic murmurs.

METHODS

We prospectively enrolled children under age 18 being evaluated by the Division of Pediatric Cardiology. Parents provided consent for a deidentified recording of their child's heart sounds with a digital stethoscope. Innocent murmurs were validated by a pediatric cardiologist and pathologic murmurs were validated by echocardiogram. To augment our collection of normal heart sounds, we utilized a public database of pediatric heart sound recordings (Oliveira, 2022). We propose two novel approaches for this audio classification task. We train a vision transformer on either Markov transition field or Gramian angular field image representations of the frequency spectrum. We benchmark our results against a ResNet-50 CNN trained on spectrogram images.

RESULTS

Our final dataset consisted of 366 normal heart sounds, 175 innocent murmurs, and 216 pathologic murmurs. Innocent murmurs collected include Still's murmur, venous hum, and flow murmurs. Pathologic murmurs included ventricular septal defect, tetralogy of Fallot, aortic regurgitation, aortic stenosis, pulmonary stenosis, mitral regurgitation and stenosis, and tricuspid regurgitation. We find that the Vision Transformer consistently outperforms the ResNet-50 on all three image representations, and that the Gramian angular field is the superior image representation for pediatric heart sounds. We calculated a one-vs-rest multi-class ROC curve for each of the three classes. Our best model achieves an area under the curve (AUC) value of 0.92 ± 0.05, 0.83 ± 0.04, and 0.88 ± 0.04 for identifying normal heart sounds, innocent murmurs, and pathologic murmurs, respectively.

CONCLUSION

We present two novel methods for pediatric heart sound classification, which outperforms the current standard of using a convolutional neural network trained on spectrogram images. To our knowledge, we are the first to demonstrate multi-class classification of pediatric murmurs. Multiclass output affords a more explainable and interpretable model, which can facilitate further model improvement in the downstream model development cycle and enhance clinician trust and therefore adoption.

Recognition of Patient Gender: A Machine Learning Preliminary Analysis Using Heart Sounds from Children and Adolescents

Research has shown that X-rays and fundus images can classify gender, age group, and race, raising concerns about bias and fairness in medical AI applications. However, the potential for physiological sounds to classify sociodemographic traits has not been investigated. Exploring this gap is crucial for understanding the implications and ensuring fairness in the field of medical sound analysis. We aimed to develop classifiers to determine gender (men/women) based on heart sound recordings and using machine learning (ML). Data-driven ML analysis. We utilized the open-access CirCor DigiScope Phonocardiogram Dataset obtained from cardiac screening programs in Brazil. Volunteers < 21 years of age. Each participant completed a questionnaire and underwent a clinical examination, including electronic auscultation at four cardiac points: aortic (AV), mitral (MV), pulmonary (PV), and tricuspid (TV). We used Mel-frequency cepstral coefficients (MFCCs) to develop the ML classifiers. From each patient and from each auscultation sound recording, we extracted 10 MFCCs. In sensitivity analysis, we additionally extracted 20, 30, 40, and 50 MFCCs. The most effective gender classifier was developed using PV recordings (AUC ROC = 70.3%). The second best came from MV recordings (AUC ROC = 58.8%). AV and TV recordings produced classifiers with an AUC ROC of 56.4% and 56.1%, respectively. Using more MFCCs did not substantially improve the classifiers. It is possible to classify between males and females using phonocardiogram data. As health-related audio recordings become more prominent in ML applications, research is required to explore if these recordings contain signals that could distinguish sociodemographic features.

Machine Learning Algorithms for Processing and Classifying Unsegmented Phonocardiographic Signals: An Efficient Edge Computing Solution Suitable for Wearable Devices

The phonocardiogram (PCG) can be used as an affordable way to monitor heart conditions. This study proposes the training and testing of several classifiers based on SVMs (support vector machines), k-NN (k-Nearest Neighbor), and NNs (neural networks) to perform binary ("Normal"/"Pathologic") and multiclass ("Normal", "CAD" (coronary artery disease), "MVP" (mitral valve prolapse), and "Benign" (benign murmurs)) classification of PCG signals, without heart sound segmentation algorithms. Two datasets of 482 and 826 PCG signals from the Physionet/CinC 2016 dataset are used to train the binary and multiclass classifiers, respectively. Each PCG signal is pre-processed, with spike removal, denoising, filtering, and normalization; afterward, it is divided into 5 s frames with a 1 s shift. Subsequently, a feature set is extracted from each frame to train and test the binary and multiclass classifiers. Concerning the binary classification, the trained classifiers yielded accuracies ranging from 92.4 to 98.7% on the test set, with memory occupations from 92.7 kB to 11.1 MB. Regarding the multiclass classification, the trained classifiers achieved accuracies spanning from 95.3 to 98.6% on the test set, occupying a memory portion from 233 kB to 14.1 MB. The NNs trained and tested in this work offer the best trade-off between performance and memory occupation, whereas the trained k-NN models obtained the best performance at the cost of large memory occupation (up to 14.1 MB). The classifiers' performance slightly depends on the signal quality, since a denoising step is performed during pre-processing. To this end, the signal-to-noise ratio (SNR) was acquired before and after the denoising, indicating an improvement between 15 and 30 dB. The trained and tested models occupy relatively little memory, enabling their implementation in resource-limited systems.

Developing an AI-assisted digital auscultation tool for automatic assessment of the severity of mitral regurgitation: protocol for a cross-sectional, non-interventional study

INTRODUCTION

Mitral regurgitation (MR) is the most common valvular heart disorder, with a morbidity rate of 2.5%. While echocardiography is commonly used in assessing MR, it has many limitations, especially for large-scale MR screening. Cardiac auscultation with electronic stethoscope and artificial intelligence (AI) can be a fast and economical modality for assessing MR severity. Our objectives are (1) to establish a deep neural network (DNN)-based cardiac auscultation method for assessing the severity of MR; and (2) to quantitatively measure the performance of the developed AI-based MR assessment method by virtual clinical trial.

METHODS AND ANALYSIS

In a cross-sectional design, phonocardiogram will be recorded at the mitral valve auscultation area of outpatients. The enrolled patients will be checked by echocardiography to confirm the diagnosis of MR or no MR. Echocardiographic parameters will be used as gold standard to assess the severity of MR, classified into four levels: none, mild, moderate and severe. The study consists of two stages. First, an MR-related cardiac sound database will be created on which a DNN-based MR severity classifier will be trained. The automatic MR severity classifier will be integrated with the Smartho-D2 electronic stethoscope. Second, the performance of the developed smart device will be assessed in an independent clinical validation data set. Sensitivity, specificity, precision, accuracy and F1 score of the developed smart MR assessment device will be evaluated. Agreement on the performance of the smart device between cardiologist users and patient users will be inspected. The interpretability of the developed model will also be studied with statistical comparisons of occlusion map-guided variables among the four severity groups.

ETHICS AND DISSEMINATION

The study protocol was approved by the Medical Ethics Committee of Huzhou Central Hospital, China (registration number: 202302009-01). Informed consent is required from all participants. Dissemination will be through conference presentations and peer-reviewed journals.

TRIAL REGISTRATION NUMBER

ChiCTR2300069496.

Accurate Localization of First and Second Heart Sounds via Template Matching in Forcecardiography Signals

Cardiac auscultation is an essential part of physical examination and plays a key role in the early diagnosis of many cardiovascular diseases. The analysis of phonocardiography (PCG) recordings is generally based on the recognition of the main heart sounds, i.e., S1 and S2, which is not a trivial task. This study proposes a method for an accurate recognition and localization of heart sounds in Forcecardiography (FCG) recordings. FCG is a novel technique able to measure subsonic vibrations and sounds via small force sensors placed onto a subject's thorax, allowing continuous cardio-respiratory monitoring. In this study, a template-matching technique based on normalized cross-correlation was used to automatically recognize heart sounds in FCG signals recorded from six healthy subjects at rest. Distinct templates were manually selected from each FCG recording and used to separately localize S1 and S2 sounds, as well as S1-S2 pairs. A simultaneously recorded electrocardiography (ECG) trace was used for performance evaluation. The results show that the template matching approach proved capable of separately classifying S1 and S2 sounds in more than 96% of all heartbeats. Linear regression, correlation, and Bland-Altman analyses showed that inter-beat intervals were estimated with high accuracy. Indeed, the estimation error was confined within 10 ms, with negligible impact on heart rate estimation. Heart rate variability (HRV) indices were also computed and turned out to be almost comparable with those obtained from ECG. The preliminary yet encouraging results of this study suggest that the template matching approach based on normalized cross-correlation allows very accurate heart sounds localization and inter-beat intervals estimation.

Algorithm for predicting valvular heart disease from heart sounds in an unselected cohort

Objective

This study aims to assess the ability of state-of-the-art machine learning algorithms to detect valvular heart disease (VHD) from digital heart sound recordings in a general population that includes asymptomatic cases and intermediate stages of disease progression.

Methods

We trained a recurrent neural network to predict murmurs from heart sound audio using annotated recordings collected with digital stethoscopes from four auscultation positions in 2,124 participants from the Tromsø7 study. The predicted murmurs were used to predict VHD as determined by echocardiography.

Results

The presence of aortic stenosis (AS) was detected with a sensitivity of 90.9%, a specificity of 94.5%, and an area under the curve (AUC) of 0.979 (CI: 0.963-0.995). At least moderate AS was detected with an AUC of 0.993 (CI: 0.989-0.997). Moderate or greater aortic and mitral regurgitation (AR and MR) were predicted with AUC values of 0.634 (CI: 0.565-703) and 0.549 (CI: 0.506-0.593), respectively, which increased to 0.766 and 0.677 when clinical variables were added as predictors. The AUC for predicting symptomatic cases was higher for AR and MR, 0.756 and 0.711, respectively. Screening jointly for symptomatic regurgitation or presence of stenosis resulted in an AUC of 0.86, with 97.7% of AS cases (n = 44) and all 12 MS cases detected.

Conclusions

The algorithm demonstrated excellent performance in detecting AS in a general cohort, surpassing observations from similar studies on selected cohorts. The detection of AR and MR based on HS audio was poor, but accuracy was considerably higher for symptomatic cases, and the inclusion of clinical variables improved the performance of the model significantly.

Machine Learning for Automated Classification of Abnormal Lung Sounds Obtained from Public Databases: A Systematic Review

Pulmonary auscultation is essential for detecting abnormal lung sounds during physical assessments, but its reliability depends on the operator. Machine learning (ML) models offer an alternative by automatically classifying lung sounds. ML models require substantial data, and public databases aim to address this limitation. This systematic review compares characteristics, diagnostic accuracy, concerns, and data sources of existing models in the literature. Papers published from five major databases between 1990 and 2022 were assessed. Quality assessment was accomplished with a modified QUADAS-2 tool. The review encompassed 62 studies utilizing ML models and public-access databases for lung sound classification. Artificial neural networks (ANN) and support vector machines (SVM) were frequently employed in the ML classifiers. The accuracy ranged from 49.43% to 100% for discriminating abnormal sound types and 69.40% to 99.62% for disease class classification. Seventeen public databases were identified, with the ICBHI 2017 database being the most used (66%). The majority of studies exhibited a high risk of bias and concerns related to patient selection and reference standards. Summarizing, ML models can effectively classify abnormal lung sounds using publicly available data sources. Nevertheless, inconsistent reporting and methodologies pose limitations to advancing the field, and therefore, public databases should adhere to standardized recording and labeling procedures.

Beyond Heart Murmur Detection: Automatic Murmur Grading From Phonocardiogram

OBJECTIVE

Murmurs are abnormal heart sounds, identified by experts through cardiac auscultation. The murmur grade, a quantitative measure of the murmur intensity, is strongly correlated with the patient's clinical condition. This work aims to estimate each patient's murmur grade (i.e., absent, soft, loud) from multiple auscultation location phonocardiograms (PCGs) of a large population of pediatric patients from a low-resource rural area.

METHODS

The Mel spectrogram representation of each PCG recording is given to an ensemble of 15 convolutional residual neural networks with channel-wise attention mechanisms to classify each PCG recording. The final murmur grade for each patient is derived based on the proposed decision rule and considering all estimated labels for available recordings. The proposed method is cross-validated on a dataset consisting of 3456 PCG recordings from 1007 patients using a stratified ten-fold cross-validation. Additionally, the method was tested on a hidden test set comprised of 1538 PCG recordings from 442 patients.

RESULTS

The overall cross-validation performances for patient-level murmur gradings are 86.3% and 81.6% in terms of the unweighted average of sensitivities and F1-scores, respectively. The sensitivities (and F1-scores) for absent, soft, and loud murmurs are 90.7% (93.6%), 75.8% (66.8%), and 92.3% (84.2%), respectively. On the test set, the algorithm achieves an unweighted average of sensitivities of 80.4% and an F1-score of 75.8%.

CONCLUSIONS

This study provides a potential approach for algorithmic pre-screening in low-resource settings with relatively high expert screening costs.

SIGNIFICANCE

The proposed method represents a significant step beyond detection of murmurs, providing characterization of intensity, which may provide an enhanced classification of clinical outcomes.

Classifying Heart-Sound Signals Based on CNN Trained on MelSpectrum and Log-MelSpectrum Features

The intelligent classification of heart-sound signals can assist clinicians in the rapid diagnosis of cardiovascular diseases. Mel-frequency cepstral coefficients (MelSpectrums) and log Mel-frequency cepstral coefficients (Log-MelSpectrums) based on a short-time Fourier transform (STFT) can represent the temporal and spectral structures of original heart-sound signals. Recently, various systems based on convolutional neural networks (CNNs) trained on the MelSpectrum and Log-MelSpectrum of segmental heart-sound frames that outperform systems using handcrafted features have been presented and classified heart-sound signals accurately. However, there is no a priori evidence of the best input representation for classifying heart sounds when using CNN models. Therefore, in this study, the MelSpectrum and Log-MelSpectrum features of heart-sound signals combined with a mathematical model of cardiac-sound acquisition were analysed theoretically. Both the experimental results and theoretical analysis demonstrated that the Log-MelSpectrum features can reduce the classification difference between domains and improve the performance of CNNs for heart-sound classification.

Technical characterisation of digital stethoscopes: towards scalable artificial intelligence-based auscultation

Digital stethoscopes can enable the development of integrated artificial intelligence (AI) systems that can remove the subjectivity of manual auscultation, improve diagnostic accuracy, and compensate for diminishing auscultatory skills. Developing scalable AI systems can be challenging, especially when acquisition devices differ and thus introduce sensor bias. To address this issue, a precise knowledge of these differences, i.e., frequency responses of these devices, is needed, but the manufacturers often do not provide complete device specifications. In this study, we reported an effective methodology for determining the frequency response of a digital stethoscope and used it to characterise three common digital stethoscopes: Littmann 3200, Eko Core, and Thinklabs One. Our results show significant inter-device variability in that the frequency responses of the three studied stethoscopes were distinctly different. A moderate intra-device variability was seen when comparing two separate units of Littmann 3200. The study highlights the need for normalisation across devices for developing successful AI-assisted auscultation and provides a technical characterisation approach as a first step to accomplish it.

Heartbeat sound classification using a hybrid adaptive neuro-fuzzy inferences system (ANFIS) and artificial bee colony

Cardiovascular disease is one of the main causes of death worldwide which can be easily diagnosed by listening to the murmur sound of heartbeat sounds using a stethoscope. The murmur sound happens at the Lub-Dub, which indicates there are abnormalities in the heart. However, using the stethoscope for listening to the heartbeat sound requires a long time of training then only the physician can detect the murmuring sound. The existing studies show that young physicians face difficulties in this heart sound detection. Use of computerized methods and data analytics for detection and classification of heartbeat sounds will improve the overall quality of sound detection. Many studies have been worked on classifying the heartbeat sound; however, they lack the method with high accuracy. Therefore, this research aims to classify the heartbeat sound using a novel optimized Adaptive Neuro-Fuzzy Inferences System (ANFIS) by artificial bee colony (ABC). The data is cleaned, pre-processed, and MFCC is extracted from the heartbeat sounds. Then the proposed ABC-ANFIS is used to run the pre-processed heartbeat sound, and accuracy is calculated for the model. The results indicate that the proposed ABC-ANFIS model achieved 93% accuracy for the murmur class. The proposed ABC-ANFIS has higher accuracy in compared to ANFIS, PSO ANFIS, SVM, KSTM, KNN, and other existing studies. Thus, this study can assist physicians to classify heartbeat sounds for detecting cardiovascular disease in the early stages.

An Optimal Approach for Heart Sound Classification Using Grid Search in Hyperparameter Optimization of Machine Learning

Heart-sound auscultation is one of the most widely used approaches for detecting cardiovascular disorders. Diagnosing abnormalities of heart sound using a stethoscope depends on the physician's skill and judgment. Several studies have shown promising results in automatically detecting cardiovascular disorders based on heart-sound signals. However, the accuracy performance needs to be enhanced as automated heart-sound classification aids in the early detection and prevention of the dangerous effects of cardiovascular problems. In this study, an optimal heart-sound classification method based on machine learning technologies for cardiovascular disease prediction is performed. It consists of three steps: pre-processing that sets the 5 s duration of the PhysioNet Challenge 2016 and 2022 datasets, feature extraction using Mel frequency cepstrum coefficients (MFCC), and classification using grid search for hyperparameter tuning of several classifier algorithms including k-nearest neighbor (K-NN), random forest (RF), artificial neural network (ANN), and support vector machine (SVM). The five-fold cross-validation was used to evaluate the performance of the proposed method. The best model obtained classification accuracy of 95.78% and 76.31%, which was assessed using PhysioNet Challenge 2016 and 2022, respectively. The findings demonstrate that the suggested approach obtained excellent classification results using PhysioNet Challenge 2016 and showed promising results using PhysioNet Challenge 2022. Therefore, the proposed method has been potentially developed as an additional tool to facilitate the medical practitioner in diagnosing the abnormality of the heart sound.

Robust classification of heart valve sound based on adaptive EMD and feature fusion

Cardiovascular disease (CVD) is considered one of the leading causes of death worldwide. In recent years, this research area has attracted researchers' attention to investigate heart sounds to diagnose the disease. To effectively distinguish heart valve defects from normal heart sounds, adaptive empirical mode decomposition (EMD) and feature fusion techniques were used to analyze the classification of heart sounds. Based on the correlation coefficient and Root Mean Square Error (RMSE) method, adaptive EMD was proposed under the condition of screening the intrinsic mode function (IMF) components. Adaptive thresholds based on Hausdorff Distance were used to choose the IMF components used for reconstruction. The multidimensional features extracted from the reconstructed signal were ranked and selected. The features of waveform transformation, energy and heart sound signal can indicate the state of heart activity corresponding to various heart sounds. Here, a set of ordinary features were extracted from the time, frequency and nonlinear domains. To extract more compelling features and achieve better classification results, another four cardiac reserve time features were fused. The fusion features were sorted using six different feature selection algorithms. Three classifiers, random forest, decision tree, and K-nearest neighbor, were trained on open source and our databases. Compared to the previous work, our extensive experimental evaluations show that the proposed method can achieve the best results and have the highest accuracy of 99.3% (1.9% improvement in classification accuracy). The excellent results verified the robustness and effectiveness of the fusion features and proposed method.

Classification of Heart Sounds Using Chaogram Transform and Deep Convolutional Neural Network Transfer Learning

Heart sounds convey important information regarding potential heart diseases. Currently, heart sound classification attracts many researchers from the fields of telemedicine, digital signal processing, and machine learning-among others-mainly to identify cardiac pathology as quickly as possible. This article proposes chaogram as a new transform to convert heart sound signals to colour images. In the proposed approach, the output image is, therefore, the projection of the reconstructed phase space representation of the phonocardiogram (PCG) signal on three coordinate planes. This has two major benefits: (1) it makes possible to apply deep convolutional neural networks to heart sounds and (2) it is also possible to employ a transfer learning scheme by converting a heart sound signal to an image. The performance of the proposed approach was verified on the PhysioNet dataset. Due to the imbalanced data on this dataset, it is common to assess the results quality using the average of sensitivity and specificity, which is known as score, instead of accuracy. In this study, the best results were achieved using the InceptionV3 model, which achieved a score of 88.06%.

A Review of Automated Bioacoustics and General Acoustics Classification Research

Automated bioacoustics classification has received increasing attention from the research community in recent years due its cross-disciplinary nature and its diverse application. Applications in bioacoustics classification range from smart acoustic sensor networks that investigate the effects of acoustic vocalizations on species to context-aware edge devices that anticipate changes in their environment adapt their sensing and processing accordingly. The research described here is an in-depth survey of the current state of bioacoustics classification and monitoring. The survey examines bioacoustics classification alongside general acoustics to provide a representative picture of the research landscape. The survey reviewed 124 studies spanning eight years of research. The survey identifies the key application areas in bioacoustics research and the techniques used in audio transformation and feature extraction. The survey also examines the classification algorithms used in bioacoustics systems. Lastly, the survey examines current challenges, possible opportunities, and future directions in bioacoustics.

A Study on the Association between Korotkoff Sound Signaling and Chronic Heart Failure (CHF) Based on Computer-Assisted Diagnoses

Background

Korotkoff sound (KS) is an important indicator of hypertension when monitoring blood pressure. However, its utility in noninvasive diagnosis of Chronic heart failure (CHF) has rarely been studied.

Purpose

In this study, we proposed a method for signal denoising, segmentation, and feature extraction for KS, and a Bayesian optimization-based support vector machine algorithm for KS classification.

Methods

The acquired KS signal was resampled and denoised to extract 19 energy features, 12 statistical features, 2 entropy features, and 13 Mel Frequency Cepstrum Coefficient (MFCCs) features. A controlled trial based on the VALSAVA maneuver was carried out to investigate the relationship between cardiac function and KS. To classify these feature sets, the K-Nearest Neighbors (KNN), decision tree (DT), Naive Bayes (NB), ensemble (EM) classifiers, and the proposed BO-SVM were employed and evaluated using the accuracy (Acc), sensitivity (Se), specificity (Sp), Precision (Ps), and F1 score (F1).

Results

The ALSAVA maneuver indicated that the KS signal could play an important role in the diagnosis of CHF. Through comparative experiments, it was shown that the best performance of the classifier was obtained by BO-SVM, with Acc (85.0%), Se (85.3%), and Sp (84.6%).

Conclusions

In this study, a method for noise reduction, segmentation, and classification of KS was established. In the measured data set, our method performed well in terms of classification accuracy, sensitivity, and specificity. In light of this, we believed that the methods described in this paper can be applied to the early, noninvasive detection of heart disease as well as a supplementary monitoring technique for the prognosis of patients with CHF.

Wavelet and Spectral Analysis of Normal and Abnormal Heart Sound for Diagnosing Cardiac Disorders

Body auscultation is a frequent clinical diagnostic procedure used to diagnose heart problems. The key advantage of this clinical method is that it provides a cheap and effective solution that enables medical professionals to interpret heart sounds for the diagnosis of cardiac diseases. Signal processing can quantify the distribution of amplitude and frequency content for diagnostic purposes. In this experiment, the use of signal processing and wavelet analysis in screening cardiac disorders provided enough evidence to distinguish between the heart sounds of a healthy and unhealthy heart. Real-time data was collected using an IoT device, and the noise was reduced using the REES52 sensor. It was found that mean frequency is sufficiently discriminatory to distinguish between a healthy and unhealthy heart, according to features derived from signal amplitude distribution in the time and frequency domain analysis. The results of the present study indicate the adequate discrimination between the characteristics of heart sounds for automatic detection of cardiac problems by signal processing from normal and abnormal heart sounds.

COVID-19 Classification from Chest X-Ray Images: A Framework of Deep Explainable Artificial Intelligence

COVID-19 detection and classification using chest X-ray images is a current hot research topic based on the important application known as medical image analysis. To halt the spread of COVID-19, it is critical to identify the infection as soon as possible. Due to time constraints and the expertise of radiologists, manually diagnosing this infection from chest X-ray images is a difficult and time-consuming process. Artificial intelligence techniques have had a significant impact on medical image analysis and have also introduced several techniques for COVID-19 diagnosis. Deep learning and explainable AI have shown significant popularity among AL techniques for COVID-19 detection and classification. In this work, we propose a deep learning and explainable AI technique for the diagnosis and classification of COVID-19 using chest X-ray images. Initially, a hybrid contrast enhancement technique is proposed and applied to the original images that are later utilized for the training of two modified deep learning models. The deep transfer learning concept is selected for the training of pretrained modified models that are later employed for feature extraction. Features of both deep models are fused using improved canonical correlation analysis that is further optimized using a hybrid algorithm named Whale-Elephant Herding. Through this algorithm, the best features are selected and classified using an extreme learning machine (ELM). Moreover, the modified deep models are utilized for Grad-CAM visualization. The experimental process was conducted on three publicly available datasets and achieved accuracies of 99.1, 98.2, and 96.7%, respectively. Moreover, the ablation study was performed and showed that the proposed accuracy is better than the other methods.

Unsupervised detection and classification of heartbeats using the dissimilarity matrix in PCG signals

BACKGROUND AND OBJECTIVE

Auscultation is the first technique applied to the early diagnose of any cardiovascular disease (CVD) in rural areas and poor-resources countries because of its low cost and non-invasiveness. However, it highly depends on the physician's expertise to recognize specific heart sounds heard through the stethoscope. The analysis of phonocardiogram (PCG) signals attempts to segment each cardiac cycle into the four cardiac states (S1, systole, S2 and diastole) in order to develop automatic systems applied to an efficient and reliable detection and classification of heartbeats. In this work, we propose an unsupervised approach, based on time-frequency characteristics shown by cardiac sounds, to detect and classify heartbeats S1 and S2.

METHODS

The proposed system consists of a two-stage cascade. The first stage performs a rough heartbeat detection while the second stage refines the previous one, improving the temporal localization and also classifying the heartbeats into types S1 and S2. The first contribution is a novel approach that combines the dissimilarity matrix with the frame-level spectral divergence to locate heartbeats using the repetitiveness shown by the heart sounds and the temporal relationships between the intervals defined by the events S1/S2 and non-S1/S2 (systole and diastole). The second contribution is a verification-correction-classification process based on a sliding window that allows the preservation of the temporal structure of the cardiac cycle in order to be applied in the heart sound classification. The proposed method has been assessed using the open access databases PASCAL, CirCor DigiScope Phonocardiogram and an additional sound mixing procedure considering both Additive White Gaussian Noise (AWGN) and different kinds of clinical ambient noises from a commercial database.

RESULTS

The proposed method outperforms the detection and classification performance of other recent state-of-the-art methods. Although our proposal achieves the best average accuracy for PCG signals without cardiac abnormalities, 99.4% in heartbeat detection and 97.2% in heartbeat classification, its worst average accuracy is always above 92% for PCG signals with cardiac abnormalities, signifying an improvement in heartbeat detection/classification above 10% compared to the other state-of-the-art methods evaluated.

CONCLUSIONS

The proposed method provides the best detection/classification performance in realistic scenarios where the presence of cardiac anomalies as well as different types of clinical environmental noises are active in the PCG signal. Of note, the promising modelling of the temporal structures of the heart provided by the dissimilarity matrix together with the frame-level spectral divergence, as well as the removal of a significant number of spurious heart events and recovery of missing heart events, both corrected by the proposed verification-correction-classification algorithm, suggest that our proposal is a successful tool to be applied in heart segmentation.

An automatic approach for heart failure typing based on heart sounds and convolutional recurrent neural networks

Heart failure (HF) is a complex clinical syndrome that poses a major hazard to human health. Patients with different types of HF have great differences in pathogenesis and treatment options. Therefore, HF typing is of great significance for timely treatment of patients. In this paper, we proposed an automatic approach for HF typing based on heart sounds (HS) and convolutional recurrent neural networks, which provides a new non-invasive and convenient way for HF typing. Firstly, the collected HS signals were preprocessed with adaptive wavelet denoising. Then, the logistic regression based hidden semi-Markov model was utilized to segment HS frames. For the distinction between normal subjects and the HF patients with preserved ejection fraction or reduced ejection fraction, a model based on convolutional neural network and recurrent neural network was built. The model can automatically learn the spatial and temporal characteristics of HS signals. The results show that the proposed model achieved a superior performance with an accuracy of 97.64%. This study suggests the proposed method could be a useful tool for HF recognition and as a supplement for HF typing.

Big Data Analysis and Prediction System Based on Improved Convolutional Neural Network

This paper presents a big data analysis and prediction system based on convolutional neural networks. Continuous template matching technology is used to analyze the distributed data structure of big data, and the information fusion processing of cloud service combination big data is combined with matching related detection methods, frequent item detection, and association rule feature extraction of high-dimensional fusion data. A clustering method is adopted to realize the classification and mining of cloud service portfolio big data. The hardware equipment of the car to detect the surrounding environment is complicated, and the combination of the convolutional neural network and the camera to detect the surrounding environment has become a research hotspot. However, simply using the convolutional neural network to process the camera data to control the turning angle of the car has the problems of long training time and low accuracy. An improved convolutional neural network is proposed. The experimental results show that the accuracy of data mining by this method is 12.43% and 21.76% higher than that of traditional methods, and the number of iteration steps is shorter, indicating that the timeliness of mining is higher. This network structure can effectively improve the training speed of the network and improve the accuracy of the network. It is proven that the convolutional neural network has faster training speed and higher accuracy.

Feature-Based Fusion Using CNN for Lung and Heart Sound Classification

Lung or heart sound classification is challenging due to the complex nature of audio data, its dynamic properties of time, and frequency domains. It is also very difficult to detect lung or heart conditions with small amounts of data or unbalanced and high noise in data. Furthermore, the quality of data is a considerable pitfall for improving the performance of deep learning. In this paper, we propose a novel feature-based fusion network called FDC-FS for classifying heart and lung sounds. The FDC-FS framework aims to effectively transfer learning from three different deep neural network models built from audio datasets. The innovation of the proposed transfer learning relies on the transformation from audio data to image vectors and from three specific models to one fused model that would be more suitable for deep learning. We used two publicly available datasets for this study, i.e., lung sound data from ICHBI 2017 challenge and heart challenge data. We applied data augmentation techniques, such as noise distortion, pitch shift, and time stretching, dealing with some data issues in these datasets. Importantly, we extracted three unique features from the audio samples, i.e., Spectrogram, MFCC, and Chromagram. Finally, we built a fusion of three optimal convolutional neural network models by feeding the image feature vectors transformed from audio features. We confirmed the superiority of the proposed fusion model compared to the state-of-the-art works. The highest accuracy we achieved with FDC-FS is 99.1% with Spectrogram-based lung sound classification while 97% for Spectrogram and Chromagram based heart sound classification.

Deep Learning-Based Heart Sound Analysis for Left Ventricular Diastolic Dysfunction Diagnosis

The aggravation of left ventricular diastolic dysfunction (LVDD) could lead to ventricular remodeling, wall stiffness, reduced compliance, and progression to heart failure with a preserved ejection fraction. A non-invasive method based on convolutional neural networks (CNN) and heart sounds (HS) is presented for the early diagnosis of LVDD in this paper. A deep convolutional generative adversarial networks (DCGAN) model-based data augmentation (DA) method was proposed to expand a HS database of LVDD for model training. Firstly, the preprocessing of HS signals was performed using the improved wavelet denoising method. Secondly, the logistic regression based hidden semi-Markov model was utilized to segment HS signals, which were subsequently converted into spectrograms for DA using the short-time Fourier transform (STFT). Finally, the proposed method was compared with VGG-16, VGG-19, ResNet-18, ResNet-50, DenseNet-121, and AlexNet in terms of performance for LVDD diagnosis. The result shows that the proposed method has a reasonable performance with an accuracy of 0.987, a sensitivity of 0.986, and a specificity of 0.988, which proves the effectiveness of HS analysis for the early diagnosis of LVDD and demonstrates that the DCGAN-based DA method could effectively augment HS data.

A systematic review on application of deep learning in digestive system image processing

With the advent of the big data era, the application of artificial intelligence represented by deep learning in medicine has become a hot topic. In gastroenterology, deep learning has accomplished remarkable accomplishments in endoscopy, imageology, and pathology. Artificial intelligence has been applied to benign gastrointestinal tract lesions, early cancer, tumors, inflammatory bowel diseases, livers, pancreas, and other diseases. Computer-aided diagnosis significantly improve diagnostic accuracy and reduce physicians' workload and provide a shred of evidence for clinical diagnosis and treatment. In the near future, artificial intelligence will have high application value in the field of medicine. This paper mainly summarizes the latest research on artificial intelligence in diagnosing and treating digestive system diseases and discussing artificial intelligence's future in digestive system diseases. We sincerely hope that our work can become a stepping stone for gastroenterologists and computer experts in artificial intelligence research and facilitate the application and development of computer-aided image processing technology in gastroenterology.