Post-stroke respiratory complications using machine learning with voice features from mobile devices

Automatic Clinical Assessment of Swallowing Behavior and Diagnosis of Silent Aspiration Using Wireless Multimodal Wearable Electronics

Dysphagia is more common in conditions such as stroke, Parkinson's disease, and head and neck cancer. This can lead to pneumonia, choking, malnutrition, and dehydration. Currently, the diagnostic gold standard uses radiologic imaging, the videofluoroscopic swallow study (VFSS); however, it is expensive and necessitates specialized facilities and trained personnel. Although several devices attempt to address the limitations, none offer the clinical-grade quality and accuracy of the VFSS. Here, this study reports a wireless multimodal wearable system with machine learning for automatic, accurate clinical assessment of swallowing behavior and diagnosis of silent aspirations from dysphagia patients. The device includes a kirigami-structured electrode that suppresses changes in skin contact impedance caused by movements and a microphone with a gel layer that effectively blocks external noise for measuring high-quality electromyograms and swallowing sounds. The deep learning algorithm offers the classification of swallowing patterns while diagnosing silent aspirations, with an accuracy of 89.47%. The demonstration with post-stroke patients captures the system's significance in measuring multiple physiological signals in real-time for detecting swallowing disorders, validated by comparing them with the VFSS. The multimodal electronics can ensure a promising future for dysphagia healthcare and rehabilitation therapy, providing an accurate, non-invasive alternative for monitoring swallowing and aspiration events.

Deep learning approach for dysphagia detection by syllable-based speech analysis with daily conversations

Dysphagia, a disorder affecting the ability to swallow, has a high prevalence among the older adults and can lead to serious health complications. Therefore, early detection of dysphagia is important. This study evaluated the effectiveness of a newly developed deep learning model that analyzes syllable-segmented data for diagnosing dysphagia, an aspect not addressed in prior studies. The audio data of daily conversations were collected from 16 patients with dysphagia and 24 controls. The presence of dysphagia was determined by videofluoroscopic swallowing study. The data were segmented into syllables using a speech-to-text model and analyzed with a convolutional neural network to perform binary classification between the dysphagia patients and control group. The proposed model in this study was assessed in two different aspects. Firstly, with syllable-segmented analysis, it demonstrated a diagnostic accuracy of 0.794 for dysphagia, a sensitivity of 0.901, a specificity of 0.687, a positive predictive value of 0.742, and a negative predictive value of 0.874. Secondly, at the individual level, it achieved an overall accuracy of 0.900 and area under the curve of 0.953. This research highlights the potential of deep learning modal as an early, non-invasive, and simple method for detecting dysphagia in everyday environments.

A deep learning approach to dysphagia-aspiration detecting algorithm through pre- and post-swallowing voice changes

Introduction

This study aimed to identify differences in voice characteristics and changes between patients with dysphagia-aspiration and healthy individuals using a deep learning model, with a focus on under-researched areas of pre- and post-swallowing voice changes in patients with dysphagia. We hypothesized that these variations may be due to weakened muscles and blocked airways in patients with dysphagia.

Methods

A prospective cohort study was conducted on 198 participants aged >40 years at the Seoul National University Bundang Hospital from October 2021 to February 2023. Pre- and post-swallowing voice data of the participants were converted to a 64-kbps mp3 format, and all voice data were trimmed to a length of 2 s. The data were divided for 10-fold cross-validation and stored in HDF5 format with anonymized IDs and labels for the normal and aspiration groups. During preprocessing, the data were converted to Mel spectrograms, and the EfficientAT model was modified using the final layer of MobileNetV3 to effectively detect voice changes and analyze pre- and post-swallowing voices. This enabled the model to probabilistically categorize new patient voices as normal or aspirated.

Results

In a study of the machine-learning model for aspiration detection, area under the receiver operating characteristic curve (AUC) values were analyzed across sexes under different configurations. The average AUC values for males ranged from 0.8117 to 0.8319, with the best performance achieved at a learning rate of 3.00e-5 and a batch size of 16. The average AUC values for females improved from 0.6975 to 0.7331, with the best performance observed at a learning rate of 5.00e-5 and a batch size of 32. As there were fewer female participants, a combined model was developed to maintain the sex balance. In the combined model, the average AUC values ranged from 0.7746 to 0.7997, and optimal performance was achieved at a learning rate of 3.00e-5 and a batch size of 16.

Conclusion

This study evaluated a voice analysis-based program to detect pre- and post-swallowing changes in patients with dysphagia, potentially aiding in real-time monitoring. Such a system can provide healthcare professionals with daily insights into the conditions of patients, allowing for personalized interventions.

Clinical Trial Registration

ClinicalTrials.gov, identifier NCT05149976.

Prediction of dysphagia aspiration through machine learning-based analysis of patients' postprandial voices

BACKGROUND

Conventional diagnostic methods for dysphagia have limitations such as long wait times, radiation risks, and restricted evaluation. Therefore, voice-based diagnostic and monitoring technologies are required to overcome these limitations. Based on our hypothesis regarding the impact of weakened muscle strength and the presence of aspiration on vocal characteristics, this single-center, prospective study aimed to develop a machine-learning algorithm for predicting dysphagia status (normal, and aspiration) by analyzing postprandial voice limiting intake to 3 cc.

METHODS

Conducted from September 2021 to February 2023 at Seoul National University Bundang Hospital, this single center, prospective cohort study included 198 participants aged 40 or older, with 128 without suspected dysphagia and 70 with dysphagia-aspiration. Voice data from participants were collected and used to develop dysphagia prediction models using the Multi-Layer Perceptron (MLP) with MobileNet V3. Male-only, female-only, and combined models were constructed using 10-fold cross-validation. Through the inference process, we established a model capable of probabilistically categorizing a new patient's voice as either normal or indicating the possibility of aspiration.

RESULTS

The pre-trained models (mn40_as and mn30_as) exhibited superior performance compared to the non-pre-trained models (mn4.0 and mn3.0). Overall, the best-performing model, mn30_as, which is a pre-trained model, demonstrated an average AUC across 10 folds as follows: combined model 0.8361 (95% CI 0.7667-0.9056; max 0.9541), male model 0.8010 (95% CI 0.6589-0.9432; max 1.000), and female model 0.7572 (95% CI 0.6578-0.8567; max 0.9779). However, for the female model, a slightly higher result was observed with the mn4.0, which scored 0.7679 (95% CI 0.6426-0.8931; max 0.9722). Additionally, the other models (pre-trained; mn40_as, non-pre-trained; mn4.0 and mn3.0) also achieved performance above 0.7 in most cases, and the highest fold-level performance for most models was approximately around 0.9. The 'mn' in model names refers to MobileNet and the following number indicates the 'width_mult' parameter.

CONCLUSIONS

In this study, we used mel-spectrogram analysis and a MobileNetV3 model for predicting dysphagia aspiration. Our research highlights voice analysis potential in dysphagia screening, diagnosis, and monitoring, aiming for non-invasive safer, and more effective interventions.

TRIAL REGISTRATION

This study was approved by the IRB (No. B-2109-707-303) and registered on clinicaltrials.gov (ID: NCT05149976).

Machine-learning assisted swallowing assessment: a deep learning-based quality improvement tool to screen for post-stroke dysphagia

Introduction

Post-stroke dysphagia is common and associated with significant morbidity and mortality, rendering bedside screening of significant clinical importance. Using voice as a biomarker coupled with deep learning has the potential to improve patient access to screening and mitigate the subjectivity associated with detecting voice change, a component of several validated screening protocols.

Methods

In this single-center study, we developed a proof-of-concept model for automated dysphagia screening and evaluated the performance of this model on training and testing cohorts. Patients were admitted to a comprehensive stroke center, where primary English speakers could follow commands without significant aphasia and participated on a rolling basis. The primary outcome was classification either as a pass or fail equivalent using a dysphagia screening test as a label. Voice data was recorded from patients who spoke a standardized set of vowels, words, and sentences from the National Institute of Health Stroke Scale. Seventy patients were recruited and 68 were included in the analysis, with 40 in training and 28 in testing cohorts, respectively. Speech from patients was segmented into 1,579 audio clips, from which 6,655 Mel-spectrogram images were computed and used as inputs for deep-learning models (DenseNet and ConvNext, separately and together). Clip-level and participant-level swallowing status predictions were obtained through a voting method.

Results

The models demonstrated clip-level dysphagia screening sensitivity of 71% and specificity of 77% (F1 = 0.73, AUC = 0.80 [95% CI: 0.78-0.82]). At the participant level, the sensitivity and specificity were 89 and 79%, respectively (F1 = 0.81, AUC = 0.91 [95% CI: 0.77-1.05]).

Discussion

This study is the first to demonstrate the feasibility of applying deep learning to classify vocalizations to detect post-stroke dysphagia. Our findings suggest potential for enhancing dysphagia screening in clinical settings. https://github.com/UofTNeurology/masa-open-source.

Computer-aided screening of aspiration risks in dysphagia with wearable technology: a Systematic Review and meta-analysis on test accuracy

Aspiration caused by dysphagia is a prevalent problem that causes serious health consequences and even death. Traditional diagnostic instruments could induce pain, discomfort, nausea, and radiation exposure. The emergence of wearable technology with computer-aided screening might facilitate continuous or frequent assessments to prompt early and effective management. The objectives of this review are to summarize these systems to identify aspiration risks in dysphagic individuals and inquire about their accuracy. Two authors independently searched electronic databases, including CINAHL, Embase, IEEE Xplore^® Digital Library, PubMed, Scopus, and Web of Science (PROSPERO reference number: CRD42023408960). The risk of bias and applicability were assessed using QUADAS-2. Nine (n = 9) articles applied accelerometers and/or acoustic devices to identify aspiration risks in patients with neurodegenerative problems (e.g., dementia, Alzheimer's disease), neurogenic problems (e.g., stroke, brain injury), in addition to some children with congenital abnormalities, using videofluoroscopic swallowing study (VFSS) or fiberoptic endoscopic evaluation of swallowing (FEES) as the reference standard. All studies employed a traditional machine learning approach with a feature extraction process. Support vector machine (SVM) was the most famous machine learning model used. A meta-analysis was conducted to evaluate the classification accuracy and identify risky swallows. Nevertheless, we decided not to conclude the meta-analysis findings (pooled diagnostic odds ratio: 21.5, 95% CI, 2.7-173.6) because studies had unique methodological characteristics and major differences in the set of parameters/thresholds, in addition to the substantial heterogeneity and variations, with sensitivity levels ranging from 21.7% to 90.0% between studies. Small sample sizes could be a critical problem in existing studies (median = 34.5, range 18-449), especially for machine learning models. Only two out of the nine studies had an optimized model with sensitivity over 90%. There is a need to enlarge the sample size for better generalizability and optimize signal processing, segmentation, feature extraction, classifiers, and their combinations to improve the assessment performance. Systematic Review Registration: (https://www.crd.york.ac.uk/prospero/), identifier (CRD42023408960).

Investigating the Impact of Voice Impairment on Quality of Life in Stroke Patients: The Voice Handicap Index (VHI) Questionnaire Study

The Voice Handicap Index (VHI) is a patient-centered evaluation tool specifically designed for assessing voice-related quality of life. Although the VHI has been extensively used in patients with voice disorders, its applicability in stroke patients has not been fully established. This prospective cross-sectional study aimed to investigate the feasibility of using the VHI questionnaire in identifying stroke patients with voice problems. The study included a cohort of acute to subacute first-ever stroke patients (n = 48), with or without voice problems, as well as other non-stroke patients (n = 31) who agreed to complete the VHI questionnaire. Stroke patients with self-reported voice problems demonstrated significantly higher VHI scores and poorer life quality scores compared to the control groups. These patients also had lower Mini-Mental State Examination (MMSE), Modified Barthel Index (MBI), and Euro-QoL-5D-5L (EQ-5D-5L) scores. Spearman correlation analysis revealed an inverse association between VHI scores and EQ-5D-5L (rho = -0.77, p < 0.001), Korean Mann Assessment of Swallowing Ability (rho = -0.51, p < 0.001), and other functional parameters, including the National Institutes of Health Stroke Scale, MMSE, and MBI scores. Multiple regression analysis indicated that the VHI score was the biggest contributing factor to EQ scores. This is the first study to demonstrate that stroke patients with voice problems may experience reduced quality of life, even after controlling for other confounding factors such as dysphagia or neurological deficits. Future studies are needed whether addressing these issues by implementing the VHI may facilitate the improvement of patients' quality of life.

Research on a real-time dynamic monitoring method for silent aspiration after stroke based on semisupervised deep learning: A protocol study

Objective

This study aims to establish a real-time dynamic monitoring system for silent aspiration (SA) to provide evidence for the early diagnosis of and precise intervention for SA after stroke.

Methods

Multisource signals, including sound, nasal airflow, electromyographic, pressure and acceleration signals, will be obtained by multisource sensors during swallowing events. The extracted signals will be labeled according to videofluoroscopic swallowing studies (VFSSs) and input into a special dataset. Then, a real-time dynamic monitoring model for SA will be built and trained based on semisupervised deep learning. Model optimization will be performed based on the mapping relationship between multisource signals and insula-centered cerebral cortex-brainstem functional connectivity through resting-state functional magnetic resonance imaging. Finally, a real-time dynamic monitoring system for SA will be established, of which the sensitivity and specificity will be improved by clinical application.

Results

Multisource signals will be stably extracted by multisource sensors. Data from a total of 3200 swallows will be obtained from patients with SA, including 1200 labeled swallows from the nonaspiration category from VFSSs and 2000 unlabeled swallows. A significant difference in the multisource signals is expected to be found between the SA and nonaspiration groups. The features of labeled and pseudolabeled multisource signals will be extracted through semisupervised deep learning to establish a dynamic monitoring model for SA. Moreover, strong correlations are expected to be found between the Granger causality analysis (GCA) value (from the left middle frontal gyrus to the right anterior insula) and the laryngeal rise time (LRT). Finally, a dynamic monitoring system will be established based on the former model, by which SA can be identified precisely.

Conclusion

The study will establish a real-time dynamic monitoring system for SA with high sensitivity, specificity, accuracy and F1 score.