A Deep Learning Algorithm for Objective Assessment of Hypernasality in Children With Cleft Palate

Computing nasalance with MFCCs and Convolutional Neural Networks

Nasalance is a valuable clinical biomarker for hypernasality. It is computed as the ratio of acoustic energy emitted through the nose to the total energy emitted through the mouth and nose (eNasalance). A new approach is proposed to compute nasalance using Convolutional Neural Networks (CNNs) trained with Mel-Frequency Cepstrum Coefficients (mfccNasalance). mfccNasalance is evaluated by examining its accuracy: 1) when the train and test data are from the same or from different dialects; 2) with test data that differs in dynamicity (e.g. rapidly produced diadochokinetic syllables versus short words); and 3) using multiple CNN configurations (i.e. kernel shape and use of 1 × 1 pointwise convolution). Dual-channel Nasometer speech data from healthy speakers from different dialects: Costa Rica, more(+) nasal, Spain and Chile, less(-) nasal, are recorded. The input to the CNN models were sequences of 39 MFCC vectors computed from 250 ms moving windows. The test data were recorded in Spain and included short words (-dynamic), sentences (+dynamic), and diadochokinetic syllables (+dynamic). The accuracy of a CNN model was defined as the Spearman correlation between the mfccNasalance for that model and the perceptual nasality scores of human experts. In the same-dialect condition, mfccNasalance was more accurate than eNasalance independently of the CNN configuration; using a 1 × 1 kernel resulted in increased accuracy for +dynamic utterances (p < .000), though not for -dynamic utterances. The kernel shape had a significant impact for -dynamic utterances (p < .000) exclusively. In the different-dialect condition, the scores were significantly less accurate than in the same-dialect condition, particularly for Costa Rica trained models. We conclude that mfccNasalance is a flexible and useful alternative to eNasalance. Future studies should explore how to optimize mfccNasalance by selecting the most adequate CNN model as a function of the dynamicity of the target speech data.

Using Artificial Intelligence for Assessment of Velopharyngeal Competence in Children Born With Cleft Palate With or Without Cleft Lip

OBJECTIVE

Development of an AI tool to assess velopharyngeal competence (VPC) in children with cleft palate, with/without cleft lip.

DESIGN

Innovation of an AI tool using retrospective audio recordings and assessments of VPC.

SETTING

Two datasets were used. The first, named the SR dataset, included data from follow-up visits to Skåne University Hospital, Sweden. The second, named the SC + IC dataset, was a combined dataset (SC + IC dataset) with data from the Scandcleft randomized trials across five countries and an intercenter study performed at six Swedish CL/P centers.

PARTICIPANTS

SR dataset included 153 recordings from 162 children, and SC + IC dataset included 308 recordings from 399 children. All recordings were from ages 5 or 10, with corresponding VPC assessments.

INTERVENTIONS

Development of two networks, a convolutional neural network (CNN) and a pre-trained CNN (VGGish). After initial testing using the SR dataset, the networks were re-tested using the SC + IC dataset and modified to improve performance.

MAIN OUTCOME MEASURES

Accuracy of the networks' VPC scores, with speech and language pathologistś scores seen as the true values. A three-point scale was used for VPC assessments.

RESULTS

VGGish outperformed CNN, achieving 57.1% accuracy compared to 39.8%. Minor adjustments in data pre-processing and network characteristics improved accuracies.

CONCLUSIONS

Network accuracies were too low for the networks to be useful alternatives for VPC assessment in clinical practice. Suggestions for future research with regards to study design and dataset optimization were discussed.

Responsible development of clinical speech AI: Bridging the gap between clinical research and technology

This perspective article explores the challenges and potential of using speech as a biomarker in clinical settings, particularly when constrained by the small clinical datasets typically available in such contexts. We contend that by integrating insights from speech science and clinical research, we can reduce sample complexity in clinical speech AI models with the potential to decrease timelines to translation. Most existing models are based on high-dimensional feature representations trained with limited sample sizes and often do not leverage insights from speech science and clinical research. This approach can lead to overfitting, where the models perform exceptionally well on training data but fail to generalize to new, unseen data. Additionally, without incorporating theoretical knowledge, these models may lack interpretability and robustness, making them challenging to troubleshoot or improve post-deployment. We propose a framework for organizing health conditions based on their impact on speech and promote the use of speech analytics in diverse clinical contexts beyond cross-sectional classification. For high-stakes clinical use cases, we advocate for a focus on explainable and individually-validated measures and stress the importance of rigorous validation frameworks and ethical considerations for responsible deployment. Bridging the gap between AI research and clinical speech research presents new opportunities for more efficient translation of speech-based AI tools and advancement of scientific discoveries in this interdisciplinary space, particularly if limited to small or retrospective datasets.

Deep Learning-Based Diagnostic System for Velopharyngeal Insufficiency Based on Videofluoroscopy in Patients With Repaired Cleft Palates

Velopharyngeal insufficiency (VPI), which is the incomplete closure of the velopharyngeal valve during speech, is a typical poor outcome that should be evaluated after cleft palate repair. The interpretation of VPI considering both imaging analysis and perceptual evaluation is essential for further management. The authors retrospectively reviewed patients with repaired cleft palates who underwent assessment for velopharyngeal function, including both videofluoroscopic imaging and perceptual speech evaluation. The final diagnosis of VPI was made by plastic surgeons based on both assessment modalities. Deep learning techniques were applied for the diagnosis of VPI and compared with the human experts' diagnostic results of videofluoroscopic imaging. In addition, the results of the deep learning techniques were compared with a speech pathologist's diagnosis of perceptual evaluation to assess consistency with clinical symptoms. A total of 714 cases from January 2010 to June 2019 were reviewed. Six deep learning algorithms (VGGNet, ResNet, Xception, ResNext, DenseNet, and SENet) were trained using the obtained dataset. The area under the receiver operating characteristic curve of the algorithms ranged between 0.8758 and 0.9468 in the hold-out method and between 0.7992 and 0.8574 in the 5-fold cross-validation. Our findings demonstrated the deep learning algorithms performed comparable to experienced plastic surgeons in the diagnosis of VPI based on videofluoroscopic velopharyngeal imaging.

Automatic Detection System for Velopharyngeal Insufficiency Based on Acoustic Signals from Nasal and Oral Channels

Velopharyngeal insufficiency (VPI) is a type of pharyngeal function dysfunction that causes speech impairment and swallowing disorder. Speech therapists play a key role on the diagnosis and treatment of speech disorders. However, there is a worldwide shortage of experienced speech therapists. Artificial intelligence-based computer-aided diagnosing technology could be a solution for this. This paper proposes an automatic system for VPI detection at the subject level. It is a non-invasive and convenient approach for VPI diagnosis. Based on the principle of impaired articulation of VPI patients, nasal- and oral-channel acoustic signals are collected as raw data. The system integrates the symptom discriminant results at the phoneme level. For consonants, relative prominent frequency description and relative frequency distribution features are proposed to discriminate nasal air emission caused by VPI. For hypernasality-sensitive vowels, a cross-attention residual Siamese network (CARS-Net) is proposed to perform automatic VPI/non-VPI classification at the phoneme level. CARS-Net embeds a cross-attention module between the two branches to improve the VPI/non-VPI classification model for vowels. We validate the proposed system on a self-built dataset, and the accuracy reaches 98.52%. This provides possibilities for implementing automatic VPI diagnosis.

Dysarthria detection based on a deep learning model with a clinically-interpretable layer

Studies have shown deep neural networks (DNN) as a potential tool for classifying dysarthric speakers and controls. However, representations used to train DNNs are largely not clinically interpretable, which limits clinical value. Here, a model with a bottleneck layer is trained to jointly learn a classification label and four clinically-interpretable features. Evaluation of two dysarthria subtypes shows that the proposed method can flexibly trade-off between improved classification accuracy and discovery of clinically-interpretable deficit patterns. The analysis using Shapley additive explanation shows the model learns a representation consistent with the disturbances that define the two dysarthria subtypes considered in this work.

The Feasibility of Cross-Linguistic Speech Evaluation in the Care of International Cleft Palate Patients

ABSTRACT

Many patients with cleft palate in developing countries never receive postoperative speech assessment or therapy. The use of audiovisual recordings could improve access to post-repair speech care. The present study evaluated whether English-speaking speech-language pathologists (SLPs) could assess cleft palate patients speaking an unfamiliar language (Tamil) using recorded media. Recordings obtained from Tamil-speaking participants were rated by 1 Tamil-speaking SLP and 3 English-speaking SLPs. Ratings were analyzed for inter-rater reliability and scored for percent correct. Accuracy of the English SLPs was compared with independent t tests and Analysis of Variance. Sixteen participants (mean age 14.5 years, standard deviation [SD] 7.4 years; mean age of surgery of 2.7 years, SD 3.7 years; time since surgery: 10.8 years, SD 5.7 years) were evaluated. Across the 4 SLPs, 5 speech elements were found to have moderate agreement, and the mean kappa was 0.145 (slight agreement). Amongst the English-speaking SLPs, 10 speech elements were found to have substantial or moderate agreement, and the mean kappa was 0.333 (fair agreement). Speech measures with the highest inter-rater reliability were hypernasality and consonant production errors. The average percent correct of the English SLPs was 60.7% (SD 20.2%). English SLPs were more accurate if the participant was female, under eighteen, bilingual, or had speech therapy. The results demonstrate that English SLPs without training in a specific language (Tamil) have limited potential to assess speech elements accurately. This research could guide training interventions to augment the ability of SLPs to conduct cross-linguistic evaluations and improve international cleft care by global health teams.