Audio-based snore detection using deep neural networks

A multi-branch convolutional neural network for snoring detection based on audio

Obstructive sleep apnea (OSA) is associated with various health complications, and snoring is a prominent characteristic of this disorder. Therefore, the exploration of a concise and effective method for detecting snoring has consistently been a crucial aspect of sleep medicine. As the easily accessible data, the identification of snoring through sound analysis offers a more convenient and straightforward method. The objective of this study was to develop a convolutional neural network (CNN) for classifying snoring and non-snoring events based on audio. This study utilized Mel-frequency cepstral coefficients (MFCCs) as a method for extracting features during the preprocessing of raw data. In order to extract multi-scale features from the frequency domain of sound sources, this study proposes the utilization of a multi-branch convolutional neural network (MBCNN) for the purpose of classification. The network utilized asymmetric convolutional kernels to acquire additional information, while the adoption of one-hot encoding labels aimed to mitigate the impact of labels. The experiment tested the network's performance by utilizing a publicly available dataset consisting of 1,000 sound samples. The test results indicate that the MBCNN achieved a snoring detection accuracy of 99.5%. The integration of multi-scale features and the implementation of MBCNN, based on audio data, have demonstrated a substantial improvement in the performance of snoring classification.

Auditory chaos classification in real-world environments

Background & motivation

Household chaos is an established risk factor for child development. However, current methods for measuring household chaos rely on parent surveys, meaning existing research efforts cannot disentangle potentially dynamic bidirectional relations between high chaos environments and child behavior problems.

Proposed approach

We train and make publicly available a classifier to provide objective, high-resolution predictions of household chaos from real-world child-worn audio recordings. To do so, we collect and annotate a novel dataset of ground-truth auditory chaos labels compiled from over 411 h of daylong recordings collected via audio recorders worn by N = 22 infants in their homes. We leverage an existing sound event classifier to identify candidate high chaos segments, increasing annotation efficiency 8.32× relative to random sampling.

Result

Our best-performing model successfully classifies four levels of real-world household auditory chaos with a macro F1 score of 0.701 (Precision: 0.705, Recall: 0.702) and a weighted F1 score of 0.679 (Precision: 0.685, Recall: 0.680).

Significance

In future work, high-resolution objective chaos predictions from our model can be leveraged for basic science and intervention, including testing theorized mechanisms by which chaos affects children's cognition and behavior. Additionally, to facilitate further model development we make publicly available the first and largest balanced annotated audio dataset of real-world household chaos.

Automatic snoring detection using a hybrid 1D-2D convolutional neural network

Snoring, as a prevalent symptom, seriously interferes with life quality of patients with sleep disordered breathing only (simple snorers), patients with obstructive sleep apnea (OSA) and their bed partners. Researches have shown that snoring could be used for screening and diagnosis of OSA. Therefore, accurate detection of snoring sounds from sleep respiratory audio at night has been one of the most important parts. Considered that the snoring is somewhat dangerously overlooked around the world, an automatic and high-precision snoring detection algorithm is required. In this work, we designed a non-contact data acquire equipment to record nocturnal sleep respiratory audio of subjects in their private bedrooms, and proposed a hybrid convolutional neural network (CNN) model for the automatic snore detection. This model consists of a one-dimensional (1D) CNN processing the original signal and a two-dimensional (2D) CNN representing images mapped by the visibility graph method. In our experiment, our algorithm achieves an average classification accuracy of 89.3%, an average sensitivity of 89.7%, an average specificity of 88.5%, and an average AUC of 0.947, which surpasses some state-of-the-art models trained on our data. In conclusion, our results indicate that the proposed method in this study could be effective and significance for massive screening of OSA patients in daily life. And our work provides an alternative framework for time series analysis.

Automatic stridor detection using small training set via patch-wise few-shot learning for diagnosis of multiple system atrophy

Stridor is a rare but important non-motor symptom that can support the diagnosis and prediction of worse prognosis in multiple system atrophy. Recording sounds generated during sleep by video-polysomnography is recommended for detecting stridor, but the analysis is labor intensive and time consuming. A method for automatic stridor detection should be developed using technologies such as artificial intelligence (AI) or machine learning. However, the rarity of stridor hinders the collection of sufficient data from diverse patients. Therefore, an AI method with high diagnostic performance should be devised to address this limitation. We propose an AI method for detecting patients with stridor by combining audio splitting and reintegration with few-shot learning for diagnosis. We used video-polysomnography data from patients with stridor (19 patients with multiple system atrophy) and without stridor (28 patients with parkinsonism and 18 patients with sleep disorders). To the best of our knowledge, this is the first study to propose a method for stridor detection and attempt the validation of few-shot learning to process medical audio signals. Even with a small training set, a substantial improvement was achieved for stridor detection, confirming the clinical utility of our method compared with similar developments. The proposed method achieved a detection accuracy above 96% using data from only eight patients with stridor for training. Performance improvements of 4%-13% were achieved compared with a state-of-the-art AI baseline. Moreover, our method determined whether a patient had stridor and performed real-time localization of the corresponding audio patches, thus providing physicians with support for interpreting and efficiently employing the results of this method.

Application of artificial intelligence in the diagnosis of sleep apnea

STUDY OBJECTIVES

Machine learning (ML) models have been employed in the setting of sleep disorders. This review aims to summarize the existing data about the role of ML techniques in the diagnosis, classification, and treatment of sleep-related breathing disorders.

METHODS

A systematic search in Medline, EMBASE, and Cochrane databases through January 2022 was performed.

RESULTS

Our search strategy revealed 132 studies that were included in the systematic review. Existing data show that ML models have been successfully used for diagnostic purposes. Specifically, ML models showed good performance in diagnosing sleep apnea using easily obtained features from the electrocardiogram, pulse oximetry, and sound signals. Similarly, ML showed good performance for the classification of sleep apnea into obstructive and central categories, as well as predicting apnea severity. Existing data show promising results for the ML-based guided treatment of sleep apnea. Specifically, the prediction of outcomes following surgical treatment and optimization of continuous positive airway pressure therapy can be guided by ML models.

CONCLUSIONS

The adoption and implementation of ML in the field of sleep-related breathing disorders is promising. Advancements in wearable sensor technology and ML models can help clinicians predict, diagnose, and classify sleep apnea more accurately and efficiently.

CITATION

Bazoukis G, Bollepalli SC, Chung CT, et al. Application of artificial intelligence in the diagnosis of sleep apnea. J Clin Sleep Med. 2023;19(7):1337-1363.

A wearable device for at-home obstructive sleep apnea assessment: State-of-the-art and research challenges

In the last 3 years, almost all medical resources have been reserved for the screening and treatment of patients with coronavirus disease (COVID-19). Due to a shortage of medical staff and equipment, diagnosing sleep disorders, such as obstructive sleep apnea (OSA), has become more difficult than ever. In addition to being diagnosed using polysomnography at a hospital, people seem to pay more attention to alternative at-home OSA detection solutions. This study aims to review state-of-the-art assessment techniques for out-of-center detection of the main characteristics of OSA, such as sleep, cardiovascular function, oxygen balance and consumption, sleep position, breathing effort, respiratory function, and audio, as well as recent progress in the implementation of data acquisition and processing and machine learning techniques that support early detection of severe OSA levels.

Audio deepfakes: A survey

A deepfake is content or material that is synthetically generated or manipulated using artificial intelligence (AI) methods, to be passed off as real and can include audio, video, image, and text synthesis. The key difference between manual editing and deepfakes is that deepfakes are AI generated or AI manipulated and closely resemble authentic artifacts. In some cases, deepfakes can be fabricated using AI-generated content in its entirety. Deepfakes have started to have a major impact on society with more generation mechanisms emerging everyday. This article makes a contribution in understanding the landscape of deepfakes, and their detection and generation methods. We evaluate various categories of deepfakes especially in audio. The purpose of this survey is to provide readers with a deeper understanding of (1) different deepfake categories; (2) how they could be created and detected; (3) more specifically, how audio deepfakes are created and detected in more detail, which is the main focus of this paper. We found that generative adversarial networks (GANs), convolutional neural networks (CNNs), and deep neural networks (DNNs) are common ways of creating and detecting deepfakes. In our evaluation of over 150 methods, we found that the majority of the focus is on video deepfakes, and, in particular, the generation of video deepfakes. We found that for text deepfakes, there are more generation methods but very few robust methods for detection, including fake news detection, which has become a controversial area of research because of the potential heavy overlaps with human generation of fake content. Our study reveals a clear need to research audio deepfakes and particularly detection of audio deepfakes. This survey has been conducted with a different perspective, compared to existing survey papers that mostly focus on just video and image deepfakes. This survey mainly focuses on audio deepfakes that are overlooked in most of the existing surveys. This article's most important contribution is to critically analyze and provide a unique source of audio deepfake research, mostly ranging from 2016 to 2021. To the best of our knowledge, this is the first survey focusing on audio deepfakes generation and detection in English.

Monitoring of Sleep Breathing States Based on Audio Sensor Utilizing Mel-Scale Features in Home Healthcare

Sleep-related breathing disorders (SBDs) will lead to poor sleep quality and increase the risk of cardiovascular and cerebrovascular diseases which may cause death in serious cases. This paper aims to detect breathing states related to SBDs by breathing sound signals. A moment waveform analysis is applied to locate and segment the breathing cycles. As the core of our study, a set of useful features of breathing signal is proposed based on Mel frequency cepstrum analysis. Finally, the normal and abnormal sleep breathing states can be distinguished by the extracted Mel-scale indexes. Young healthy testers and patients who suffered from obstructive sleep apnea are tested utilizing the proposed method. The average accuracy for detecting abnormal breathing states can reach 93.1%. It will be helpful to prevent SBDs and improve the sleep quality of home healthcare.

Large-scale audio dataset for emergency vehicle sirens and road noises

Traffic congestion, accidents, and pollution are becoming a challenge for researchers. It is essential to develop new ideas to solve these problems, either by improving the infrastructure or applying the latest technology to use the existing infrastructure better. This research paper presents a high-resolution dataset that will help the research community to apply AI techniques to classify any emergency vehicle from traffic and road noises. Demand for such datasets is high as they can control traffic flow and reduce traffic congestion. It also improves emergency response time, especially for fire and health events. This work collects audio data using different methods, and pre-processed them  to develop a high-quality and clean dataset. The dataset is divided into two labelled classes one for emergency vehicle sirens and one for traffic noises. The developed dataset offers high quality and range of real-world traffic sounds and emergency vehicle sirens. The technical validity of the dataset is also established.

Automatic Identification of Snoring and Groaning Segments in Acoustic Recordings

Sleep-related breathing disorders have severe impact on the quality of lives of those suffering from them. These disorders present with a variety of symptoms, out of which snoring and groaning are very common. This paper presents an algorithm to identify and classify segments of acoustic respiratory sound recordings that contain both groaning and snoring events. The recordings were obtained from a database containing 20 subjects from which features based on the Mel-frequency cepstral coefficients (MFCC) were extracted. In the first stage of the algorithm, segments of recordings consisting of either snoring or groaning episodes - without classifying them - were identified. In the second stage, these segments were further differentiated into individual groaning or snoring events. The algorithm in the first stage achieved a sensitivity and specificity of 90.5% ±2.9% and 90.0% ±1.6% respectively, using a RUSBoost model. In the second stage, a random forest classifier was used, and the accuracies for groan and snore events were 78.1% ±4.7% and 78.4% ±4.7% respectively.