Snoring classified: The Munich-Passau Snore Sound Corpus

HEAR4Health: a blueprint for making computer audition a staple of modern healthcare

Recent years have seen a rapid increase in digital medicine research in an attempt to transform traditional healthcare systems to their modern, intelligent, and versatile equivalents that are adequately equipped to tackle contemporary challenges. This has led to a wave of applications that utilise AI technologies; first and foremost in the fields of medical imaging, but also in the use of wearables and other intelligent sensors. In comparison, computer audition can be seen to be lagging behind, at least in terms of commercial interest. Yet, audition has long been a staple assistant for medical practitioners, with the stethoscope being the quintessential sign of doctors around the world. Transforming this traditional technology with the use of AI entails a set of unique challenges. We categorise the advances needed in four key pillars: Hear, corresponding to the cornerstone technologies needed to analyse auditory signals in real-life conditions; Earlier, for the advances needed in computational and data efficiency; Attentively, for accounting to individual differences and handling the longitudinal nature of medical data; and, finally, Responsibly, for ensuring compliance to the ethical standards accorded to the field of medicine. Thus, we provide an overview and perspective of HEAR4Health: the sketch of a modern, ubiquitous sensing system that can bring computer audition on par with other AI technologies in the strive for improved healthcare systems.

Automatic snoring detection using a hybrid 1D-2D convolutional neural network

Snoring, as a prevalent symptom, seriously interferes with life quality of patients with sleep disordered breathing only (simple snorers), patients with obstructive sleep apnea (OSA) and their bed partners. Researches have shown that snoring could be used for screening and diagnosis of OSA. Therefore, accurate detection of snoring sounds from sleep respiratory audio at night has been one of the most important parts. Considered that the snoring is somewhat dangerously overlooked around the world, an automatic and high-precision snoring detection algorithm is required. In this work, we designed a non-contact data acquire equipment to record nocturnal sleep respiratory audio of subjects in their private bedrooms, and proposed a hybrid convolutional neural network (CNN) model for the automatic snore detection. This model consists of a one-dimensional (1D) CNN processing the original signal and a two-dimensional (2D) CNN representing images mapped by the visibility graph method. In our experiment, our algorithm achieves an average classification accuracy of 89.3%, an average sensitivity of 89.7%, an average specificity of 88.5%, and an average AUC of 0.947, which surpasses some state-of-the-art models trained on our data. In conclusion, our results indicate that the proposed method in this study could be effective and significance for massive screening of OSA patients in daily life. And our work provides an alternative framework for time series analysis.

Pilot study of contactless sleep apnea detection based on snore signals with hardware implementation

Objective.Sleep apnea has a high incidence and is a potentially dangerous disease, and its early detection and diagnosis are challenging. Polysomnography (PSG) is considered the best approach for sleep apnea detection, but it requires cumbersome and complicated operations. Thus, it cannot satisfy the family healthcare needs.Approach.To facilitate the initial detection of sleep apnea in the home environment, we developed a sleep apnea classification model based on snoring and hybrid neural network, and implemented the well trained model in an embedded hardware platform. We used snore signals from 32 patients at Shenzhen People's Hospital. The Mel-Fbank features were extracted from snore signals to build a sleep apnea classification model based on Bi-LSTM with attention mechanism.Main results.The proposed model classified snore signals into four types: hypopnea, normal condition, obstructive sleep apnea, and central sleep apnea, with 83.52% and 62.31% accuracies, corresponding to the subject-dependence and subject-independence validation, respectively. After pruning and model quantization, at the cost of 0.81% and 0.95% accuracy loss of the subject dependence and subject independence classification, respectively, the number of model parameters and model storage space were reduced by 32.12% and 60.37%, respectively. The model exhibited accuracies of 82.71% and 61.36% based on the subject dependence and subject independence validations, respectively. When the well trained model was successfully porting and running on an STM32 ARM-embedded platform, the model accuracy was 58.85% for the four classifications based on leave-one-subject-out validation.Significance.The proposed sleep apnea detection model can be used in home healthcare for the initial detection of sleep apnea.

Tidal Volume Level Estimation Using Respiratory Sounds

Respiratory sounds have been used as a noninvasive and convenient method to estimate respiratory flow and tidal volume. However, current methods need calibration, making them difficult to use in a home environment. A respiratory sound analysis method is proposed to estimate tidal volume levels during sleep qualitatively. Respiratory sounds are filtered and segmented into one-minute clips, all clips are clustered into three categories: normal breathing/snoring/uncertain with agglomerative hierarchical clustering (AHC). Formant parameters are extracted to classify snoring clips into simple snoring and obstructive snoring with the K-means algorithm. For simple snoring clips, the tidal volume level is calculated based on snoring last time. For obstructive snoring clips, the tidal volume level is calculated by the maximum breathing pause interval. The performance of the proposed method is evaluated on an open dataset, PSG-Audio, in which full-night polysomnography (PSG) and tracheal sound were recorded simultaneously. The calculated tidal volume levels are compared with the corresponding lowest nocturnal oxygen saturation (LoO₂) data. Experiments show that the proposed method calculates tidal volume levels with high accuracy and robustness.

DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing From Decentralized Data

Deep neural speech and audio processing systems have a large number of trainable parameters, a relatively complex architecture, and require a vast amount of training data and computational power. These constraints make it more challenging to integrate such systems into embedded devices and utilize them for real-time, real-world applications. We tackle these limitations by introducing DeepSpectrumLite, an open-source, lightweight transfer learning framework for on-device speech and audio recognition using pre-trained image Convolutional Neural Networks (CNNs). The framework creates and augments Mel spectrogram plots on the fly from raw audio signals which are then used to finetune specific pre-trained CNNs for the target classification task. Subsequently, the whole pipeline can be run in real-time with a mean inference lag of 242.0 ms when a DenseNet121 model is used on a consumer-grade Motorola moto e7 plus smartphone. DeepSpectrumLite operates decentralized, eliminating the need for data upload for further processing. We demonstrate the suitability of the proposed transfer learning approach for embedded audio signal processing by obtaining state-of-the-art results on a set of paralinguistic and general audio tasks, including speech and music emotion recognition, social signal processing, COVID-19 cough and COVID-19 speech analysis, and snore sound classification. We provide an extensive command-line interface for users and developers which is comprehensively documented and publicly available at https://github.com/DeepSpectrum/DeepSpectrumLite.

Towards Sleep Study Automation: Detection Evaluation of Respiratory-Related Events

The diagnosis of sleep disordered breathing depends on the detection of several respiratory-related events: apneas, hypopneas, snores, or respiratory event-related arousals from sleep studies. While a number of automatic detection methods have been proposed, reproducibility of these methods has been an issue, in part due to the absence of a generally accepted protocol for evaluating their results. With sleep measurements this is usually treated as a classification problem and the accompanying issue of localization is not treated as similarly critical. To address these problems we present a detection evaluation protocol that is able to qualitatively assess the match between two annotations of respiratory-related events. This protocol relies on measuring the relative temporal overlap between two annotations in order to find an alignment that maximizes their F1-score at the sequence level. This protocol can be used in applications which require a precise estimate of the number of events, total event duration, and a joint estimate of event number and duration. We assess its application using a data set that contains over 10,000 manually annotated snore events from 9 subjects, and show that when using the American Academy of Sleep Medicine Manual standard, two sleep technologists can achieve an F1-score of 0.88 when identifying the presence of snore events. In addition, we drafted rules for marking snore boundaries and showed that one sleep technologist can achieve F1-score of 0.94 at the same tasks. Finally, we compared our protocol against the protocol that is used to evaluate sleep spindle detection and highlighted the differences.

Computer Audition for Fighting the SARS-CoV-2 Corona Crisis-Introducing the Multitask Speech Corpus for COVID-19

Computer audition (CA) has experienced a fast development in the past decades by leveraging advanced signal processing and machine learning techniques. In particular, for its noninvasive and ubiquitous character by nature, CA-based applications in healthcare have increasingly attracted attention in recent years. During the tough time of the global crisis caused by the coronavirus disease 2019 (COVID-19), scientists and engineers in data science have collaborated to think of novel ways in prevention, diagnosis, treatment, tracking, and management of this global pandemic. On the one hand, we have witnessed the power of 5G, Internet of Things, big data, computer vision, and artificial intelligence in applications of epidemiology modeling, drug and/or vaccine finding and designing, fast CT screening, and quarantine management. On the other hand, relevant studies in exploring the capacity of CA are extremely lacking and underestimated. To this end, we propose a novel multitask speech corpus for COVID-19 research usage. We collected 51 confirmed COVID-19 patients' in-the-wild speech data in Wuhan city, China. We define three main tasks in this corpus, i.e., three-category classification tasks for evaluating the physical and/or mental status of patients, i.e., sleep quality, fatigue, and anxiety. The benchmarks are given by using both classic machine learning methods and state-of-the-art deep learning techniques. We believe this study and corpus cannot only facilitate the ongoing research on using data science to fight against COVID-19, but also the monitoring of contagious diseases for general purpose.

Prediction of the obstruction sites in the upper airway in sleep-disordered breathing based on snoring sound parameters: a systematic review

BACKGROUND

Identification of the obstruction site in the upper airway may help in treatment selection for patients with sleep-disordered breathing. Because of limitations of existing techniques, there is a continuous search for more feasible methods. Snoring sound parameters were hypothesized to be potential predictors of the obstruction site. Therefore, this review aims to i) investigate the association between snoring sound parameters and the obstruction sites; and ii) analyze the methodology of reported prediction models of the obstruction sites.

METHODS

The literature search was conducted in PubMed, Embase.com, CENTRAL, Web of Science, and Scopus in collaboration with a medical librarian. Studies were eligible if they investigated the associations between snoring sound parameters and the obstruction sites, and/or reported prediction models of the obstruction sites based on snoring sound.

RESULTS

Of the 1016 retrieved references, 28 eligible studies were included. It was found that the characteristic frequency components generated from lower-level obstructions of the upper airway were higher than those generated from upper-level obstructions. Prediction models were built mainly based on snoring sound parameters in frequency domain. The reported accuracies ranged from 60.4% to 92.2%.

CONCLUSIONS

Available evidence points toward associations between the snoring sound parameters in the frequency domain and the obstruction sites in the upper airway. It is promising to build a prediction model of the obstruction sites based on snoring sound parameters and participant characteristics, but so far snoring sound analysis does not seem to be a viable diagnostic modality for treatment selection.

PSG-Audio, a scored polysomnography dataset with simultaneous audio recordings for sleep apnea studies

The sleep apnea syndrome is a chronic condition that affects the quality of life and increases the risk of severe health conditions such as cardiovascular diseases. However, the prevalence of the syndrome in the general population is considered to be heavily underestimated due to the restricted number of people seeking diagnosis, with the leading cause for this being the inconvenience of the current reference standard for apnea diagnosis: Polysomnography. To enhance patients' awareness of the syndrome, a great endeavour is conducted in the literature. Various home-based apnea detection systems are being developed, profiting from information in a restricted set of polysomnography signals. In particular, breathing sound has been proven highly effective in detecting apneic events during sleep. The development of accurate systems requires multitudinous datasets of audio recordings and polysomnograms. In this work, we provide the first open access dataset, comprising 212 polysomnograms along with synchronized high-quality tracheal and ambient microphone recordings. We envision this dataset to be widely used for the development of home-based apnea detection techniques and frameworks.

Association of Snoring Characteristics with Predominant Site of Collapse of Upper Airway in Obstructive Sleep Apnoea Patients

STUDY OBJECTIVES

Acoustic analysis of isolated events and snoring by previous researchers suggests a correlation between individual acoustic features and individual site of collapse events. In this study, we hypothesised that multi-parameter evaluation of snore sounds during natural sleep would provide a robust prediction of the predominant site of airway collapse.

METHODS

The audio signals of 58 OSA patients were recorded simultaneously with full night polysomnography. The site of collapse was determined by manual analysis of the shape of the airflow signal during hypopnoea events and corresponding audio signal segments containing snore were manually extracted and processed. Machine learning algorithms were developed to automatically annotate the site of collapse of each hypopnoea event into three classes (lateral wall, palate and tongue-base). The predominant site of collapse for a sleep period was determined from the individual hypopnoea annotations and compared to the manually determined annotations. This was a retrospective study that used cross-validation to estimate performance.

RESULTS

Cluster analysis showed that the data fits well in two clusters with a mean silhouette coefficient of 0.79 and an accuracy of 68% for classifying tongue/non-tongue collapse. A classification model using linear discriminants achieved an overall accuracy of 81% for discriminating tongue/non-tongue predominant site of collapse and accuracy of 64% for all site of collapse classes.

CONCLUSIONS

Our results reveal that the snore signal during hypopnoea can provide information regarding the predominant site of collapse in the upper airway. Therefore, the audio signal recorded during sleep could potentially be used as a new tool in identifying the predominant site of collapse and consequently improving the treatment selection and outcome.

Application of substitution box of present cipher for automated detection of snoring sounds

BACKGROUND AND PURPOSE

Snoring is one of the sleep disorders, and snoring sounds have been used to diagnose many sleep-related diseases. However, the snoring sound classification is done manually which is time-consuming and prone to human errors. An automated snoring sound classification model is proposed to overcome these problems.

MATERIAL AND METHOD

This work proposes an automated snoring sound classification method using three new methods. These methods are maximum absolute pooling (MAP), the nonlinear present pattern, and two-layered neighborhood component analysis, and iterative neighborhood component analysis (NCAINCA) selector. Using these methods, a new snoring sound classification (SSC) model is presented. The MAP decomposition model is applied to snoring sounds to extract both low and high-level features. The presented model aims to attain high performance for SSC problem. The developed present pattern (Present-Pat) uses substitution box (SBox) and statistical feature generator. By deploying these feature generators, both textural and statistical features are generated. NCAINCA chooses the most informative/valuable features, and these selected features are fed to k-nearest neighbor (kNN) classifier with leave-one-out cross-validation (LOOCV). The Present-Pat based SSC system is developed using Munich-Passau Snore Sound Corpus (MPSSC) dataset comprising of four categories.

RESULTS

Our model reached an accuracy and unweighted average recall (UAR) of 97.10 % and 97.60 %, respectively, using LOOCV. Moreover, a nocturnal sound dataset is used to show the universal success of the presented model. Our model attained an accuracy of 98.14 % using the used nocturnal sound dataset.

CONCLUSIONS

Our developed classification model is ready to be tested with more data and can be used by sleep specialists to diagnose the sleep disorders based on snoring sounds.

Automatic classification of excitation location of snoring sounds

STUDY OBJECTIVES

For surgical treatment of patients with obstructive sleep apnea-hypopnea syndrome, it is crucial to locate accurately the obstructive sites in the upper airway; however, noninvasive methods for locating the obstructive sites have not been well explored. Snoring, as the cardinal symptom of obstructive sleep apnea-hypopnea syndrome, should contain information that reflects the state of the upper airway. Through the classification of snores produced at four different locations, this study aimed to test the hypothesis that snores generated by various obstructive sites differ.

METHODS

We trained and tested our model on a public data set that comprised 219 participants. For each snore episode, an acoustic and a physiological feature were extracted and concatenated, forming a 59-dimensional fusion feature. A principal component analysis and a support machine vector were used for dimensional reduction and snore classification. The performance of the proposed model was evaluated using several metrics: sensitivity, precision, specificity, area under the receiver operating characteristic curve, and F1 score.

RESULTS

The unweighted average values of sensitivity, precision, specificity, area under the curve, and F1 were 86.36%, 89.09%, 96.4%, 87.9%, and 87.63%, respectively. The model achieved 98.04%, 80.56%, 72.73%, and 94.12% sensitivity for types V (velum), O (oropharyngeal), T (tongue), and E (epiglottis) snores.

CONCLUSIONS

The characteristics of snores are related to the state of the upper airway. The machine-learning-based model can be used to locate the vibration sites in the upper airway.

Can Machine Learning Assist Locating the Excitation of Snore Sound? A Review

In the past three decades, snoring (affecting more than 30 % adults of the UK population) has been increasingly studied in the transdisciplinary research community involving medicine and engineering. Early work demonstrated that, the snore sound can carry important information about the status of the upper airway, which facilitates the development of non-invasive acoustic based approaches for diagnosing and screening of obstructive sleep apnoea and other sleep disorders. Nonetheless, there are more demands from clinical practice on finding methods to localise the snore sound's excitation rather than only detecting sleep disorders. In order to further the relevant studies and attract more attention, we provide a comprehensive review on the state-of-the-art techniques from machine learning to automatically classify snore sounds. First, we introduce the background and definition of the problem. Second, we illustrate the current work in detail and explain potential applications. Finally, we discuss the limitations and challenges in the snore sound classification task. Overall, our review provides a comprehensive guidance for researchers to contribute to this area.

COVID-19 and Computer Audition: An Overview on What Speech & Sound Analysis Could Contribute in the SARS-CoV-2 Corona Crisis

At the time of writing this article, the world population is suffering from more than 2 million registered COVID-19 disease epidemic-induced deaths since the outbreak of the corona virus, which is now officially known as SARS-CoV-2. However, tremendous efforts have been made worldwide to counter-steer and control the epidemic by now labelled as pandemic. In this contribution, we provide an overview on the potential for computer audition (CA), i.e., the usage of speech and sound analysis by artificial intelligence to help in this scenario. We first survey which types of related or contextually significant phenomena can be automatically assessed from speech or sound. These include the automatic recognition and monitoring of COVID-19 directly or its symptoms such as breathing, dry, and wet coughing or sneezing sounds, speech under cold, eating behaviour, sleepiness, or pain to name but a few. Then, we consider potential use-cases for exploitation. These include risk assessment and diagnosis based on symptom histograms and their development over time, as well as monitoring of spread, social distancing and its effects, treatment and recovery, and patient well-being. We quickly guide further through challenges that need to be faced for real-life usage and limitations also in comparison with non-audio solutions. We come to the conclusion that CA appears ready for implementation of (pre-)diagnosis and monitoring tools, and more generally provides rich and significant, yet so far untapped potential in the fight against COVID-19 spread.

An automated snoring sound classification method based on local dual octal pattern and iterative hybrid feature selector

In this research, a novel snoring sound classification (SSC) method is presented by proposing a new feature generation function to yield a high classification rate. The proposed feature extractor is named as Local Dual Octal Pattern (LDOP). A novel LDOP based SSC method is presented to solve the low success rate problems for Munich-Passau Snore Sound Corpus (MPSSC) dataset. Multilevel discrete wavelet transform (DWT) decomposition and the LDOP based feature generation, informative features selection with ReliefF and iterative neighborhood component analysis (RFINCA), and classification using k nearest neighbors (kNN) are fundamental phases of the proposed SSC method. Seven leveled DWT transform, and LDOP are used together to generate low, medium, and high levels features. This feature generation network extracts 4096 features in total. RFINCA selects 95 the most discriminative and informative ones of these 4096 features. In the classification phase, kNN with leave one out cross-validation (LOOCV) is used. 95.53% classification accuracy and 94.65% unweighted average recall (UAR) have been achieved using this method. The proposed LDOP based SSC method reaches 22% better result than the best of the other state-of-the-art machine learning and deep learning-based methods. These results clearly denote the success of the proposed SSC method.

Amplitude spectrum trend-based feature for excitation location classification from snore sounds

OBJECTIVE

Successful surgical treatment of obstructive sleep apnea (OSA) depends on the precise location of the vibrating tissue. Snoring is the main symptom of OSA and can be utilized to detect the active location of tissues. However, existing approaches are limited, owing to their inability to capture the characteristics of snoring produced from the upper airway. This paper proposes a new approach to better distinguish different snoring sounds that are generated from four different excitation locations.

APPROACH

First, we propose a robust null space pursuit algorithm for extracting the trend from the amplitude spectrum of snoring. Second, a new feature from this extracted amplitude spectrum trend, which outperforms the Mel-frequency cepstral coefficient (MFCC) feature, is designed. Subsequently, the newly proposed feature, namely the trend-based MFCC (TCC), is reduced in dimensionality by using principal component analysis. Finally, a support vector machine is employed for the classification task.

MAIN RESULTS

By using the TCC, the proposed approach achieves an unweighted average recall of 87.5% on the classification of four excitation locations on the public dataset Munich Passau Snore Sound Corpus.

SIGNIFICANCE

The TCC is a promising feature for capturing the characteristics of snoring. The proposed method can effectively perform snore classification and assist in accurate OSA diagnosis.

Snoring patterns during home polysomnography. A proposal for a new classification

PURPOSE

Snoring is a very common disorder, but, at present, there is no universally accepted classification for the condition. The main aim of this paper is to introduce a home sleep monitoring-based classification of common snoring patterns in simple snorers and in patients with obstructive sleep apnea-hypopnea syndrome (OSAHS).

MATERIALS AND METHODS

In total, 561 consecutive patients with a history of snoring, either simple or associated with apnea, were enrolled in this home sleep monitoring study. Analysis of the polysomnographic traces and the snoring sensor allowed the main patterns of snoring and their characteristics to be determined.

RESULTS

Four patterns of snoring were identified. In a spectrum of increasing severity (mild, moderate or severe), snoring can be episodic, positional, continuous, or alternating, whereas in obstructive sleep apnea syndrome, the snoring events only occur between successive respiratory obstructive events. In mild snoring, the episodic pattern is the most frequent, whereas in moderate and severe snoring, the continuous snoring pattern occurs in most cases.

CONCLUSIONS

The proposed classification of snoring patterns would be beneficial for providing a realistic disturbance index, for the selection and evaluation of the outcomes of surgical techniques.

Snoring - An Acoustic Definition

Objective- The distinction of snoring and loud breathing is often subjective and lies in the ear of the beholder. The aim of this study is to identify and assess acoustic features with a high suitability to distinguish these two classes of sound, in order to facilitate an objective definition of snoring based on acoustic parameters. Methods- A corpus of snore and breath sounds from 23 subjects has been used that were classified by 25 human raters. Using the openSMILE feature extractor, 6 373 acoustic features have been evaluated for their selectivity comparing SVM classification, logistic regression, and the recall of each single feature. Results- Most selective single features were several statistical functionals of the first and second mel frequency spectrum-generated perceptual linear predictive (PLP) cepstral coefficient with an unweighted average recall (UAR) of up to 93.8%. The best performing feature sets were low level descriptors (LLDs), derivatives and statistical functionals based on fast Fourier transformation (FFT), with a UAR of 93.0%, and on the summed mel frequency spectrum-generated PLP cepstral coefficients, with a UAR of 92.2% using SVM classification. Compared to SVM classification, logistic regression did not show considerable differences in classification performance. Conclusion- It could be shown that snoring and loud breathing can be distinguished by robust acoustic features. The findings might serve as a guidance to find a consensus for an objective definition of snoring compared to loud breathing.

Prediction of Obstructive Sleep Apnea Based on Respiratory Sounds Recorded Between Sleep Onset and Sleep Offset

OBJECTIVES

To develop a simple algorithm for prescreening of obstructive sleep apnea (OSA) on the basis of respiratory sounds recorded during polysomnography during all sleep stages between sleep onset and offset.

METHODS

Patients who underwent attended, in-laboratory, full-night polysomnography were included. For all patients, audio recordings were performed with an air-conduction microphone during polysomnography. Analyses included all sleep stages (i.e., N1, N2, N3, rapid eye movement, and waking). After noise reduction preprocessing, data were segmented into 5-s windows and sound features were extracted. Prediction models were established and validated with 10-fold cross-validation by using simple logistic regression. Binary classifications were separately conducted for three different threshold criteria at apnea hypopnea index (AHI) of 5, 15, or 30. Prediction model characteristics, including accuracy, sensitivity, specificity, positive predictive value (precision), negative predictive value, and area under the curve (AUC) of the receiver operating characteristic were computed.

RESULTS

A total of 116 subjects were included; their mean age, body mass index, and AHI were 50.4 years, 25.5 kg/m2 , and 23.0/hr, respectively. A total of 508 sound features were extracted from respiratory sounds recorded throughout sleep. Accuracies of binary classifiers at AHIs of 5, 15, and 30 were 82.7%, 84.4%, and 85.3%, respectively. Prediction performances for the classifiers at AHIs of 5, 15, and 30 were AUC, 0.83, 0.901, and 0.91; sensitivity, 87.5%, 81.6%, and 60%; and specificity, 67.8%, 87.5%, and 94.1%. Respective precision values of the classifiers were 89.5%, 87.5%, and 78.2% for AHIs of 5, 15, and 30.

CONCLUSION

This study showed that our binary classifier predicted patients with AHI of ≥15 with sensitivity and specificity of >80% by using respiratory sounds during sleep. Since our prediction model included all sleep stage data, algorithms based on respiratory sounds may have a high value for prescreening OSA with mobile devices.