A noninvasive estimation of hypernasality using a linear predictive model

Subjective and Objective Evaluation of Speech in Adult Patients With Repaired Cleft Palate

OBJECTIVE

To explore the speech outcomes of adult patients with repaired cleft palate through subjective perception evaluation and objective acoustic analysis, and to compare the differences in pronunciation characteristics between speakers with complete velopharyngeal closure (VPC) and velopharyngeal insufficiency (VPI) patients.

PARTICIPANTS AND INTERVENTION

Subjective evaluation indicators included speech intelligibility, nasality and consonant missing rate, for objective acoustic analysis, we used speech sample normalization and objective acoustic parameters included normalized vowel formants, voice onset time and the analysis of 3-dimensional spectrogram and spectrum, were carried out on speech samples produced by 3 groups of speakers: (a) speakers with velopharyngeal competence after palatorrhaphy (n=38); (b) speakers with velopharyngeal incompetence after palatorrhaphy (n=70), (c) adult patients with cleft palate (n=65) and (d) typical speakers (n=30).

RESULTS

There was a highly negative correlation between VPC grade and speech intelligibility (ρ=-0.933), and a highly positive correlation between VPC and nasality (ρ=0.813). In subjective evaluation, the speech level of VPI patients was significantly lower than that of VPC patients and normal adults. Although the nasality and consonant loss rate of VPC patients were significantly higher than that of normal adults, the speech intelligibility of VPC patients was not significantly different from that of normal adults. In acoustic analysis, patients with VPI still performed poorly compared with patients with VPC.

CONCLUSIONS

The speech function of adult cleft palate patients is affected by abnormal palatal structure and bad pronunciation habits. In subjective evaluation, there was no significant difference in speech level between VPC patients and normal adults, whereas there was significant difference between VPI patients and normal adults. The acoustic parameters were different between the 2 groups after cleft palate repair. The condition of palatopharyngeal closure after cleft palate can affect the patient's speech.

Evaluation of noise excitation as a method for detection of hypernasality

Hypernasality is a disorder where excess nasal resonance is perceived during speech, often as a result of abnormal coupling between the oral and nasal tracts known as velopharyngeal insufficiency (VPI). The most common cause of VPI is a cleft palate, which affects around 1 in 1650 babies, around ⅓ of whom have persistent speech problems after surgery. Current equipment-based assessment methods are invasive and require expert knowledge, and perceptual assessment methods are limited by the availability of expert listeners and differing interpretations of assessment scales. Spectral analysis of hypernasality within the academic community has resulted in potentially useful spectral indicators, but these are highly variable, vowel specific, and not commonly used within clinical practice. Previous works by others have developed noise excitation technologies for the measurement of oral tract transfer functions using resonance measurement devices (RMD). These techniques provide an opportunity to investigate the structural system abnormalities which lead to hypernasality, without the need for invasive measurement equipment. Thus, the work presented in this study adapts these techniques for the detection of hypernasality. These adaptations include augmentation of the hardware and development of the software, so as to be suitable for transfer function measurement at the nostrils rather than the mouth (nRMD). The new method was tested with a single participant trained in hypernasal production, producing 'normal' and hypernasal vowels, and the recordings validated through a listening test by an expert listener and calculation of nasalance values using a nasality microphone. These validation stages indicated the reliability of the captured data, and analysis of the nRMD measurements indicated the presence of a systematic difference in the frequency range 2 to 2.5 kHz between normal and hypernasal speech. Further investigation is warranted to determine the generalisability of these findings across speakers, and to investigate the origins of differences manifesting in the transfer functions between conditions. This will provide new insights into the effects of nasal tract coupling on voice acoustics, which could in turn lead to the development of useful new tools to support clinicians in their work with hypernasality.

Sch-net: a deep learning architecture for automatic detection of schizophrenia

BACKGROUND

Schizophrenia is a chronic and severe mental disease, which largely influences the daily life and work of patients. Clinically, schizophrenia with negative symptoms is usually misdiagnosed. The diagnosis is also dependent on the experience of clinicians. It is urgent to develop an objective and effective method to diagnose schizophrenia with negative symptoms. Recent studies had shown that impaired speech could be considered as an indicator to diagnose schizophrenia. The literature about schizophrenic speech detection was mainly based on feature engineering, in which effective feature extraction is difficult because of the variability of speech signals.

METHODS

This work designs a novel Sch-net neural network based on a convolutional neural network, which is the first work for end-to-end schizophrenic speech detection using deep learning techniques. The Sch-net adds two components, skip connections and convolutional block attention module (CBAM), to the convolutional backbone architecture. The skip connections enrich the information used for the classification by emerging low- and high-level features. The CBAM highlights the effective features by giving learnable weights. The proposed Sch-net combines the advantages of the two components, which can avoid the procedure of manual feature extraction and selection.

RESULTS

We validate our Sch-net through ablation experiments on a schizophrenic speech data set that contains 28 patients with schizophrenia and 28 healthy controls. The comparisons with the models based on feature engineering and deep neural networks are also conducted. The experimental results show that the Sch-net has a great performance on the schizophrenic speech detection task, which can achieve 97.68% accuracy on the schizophrenic speech data set. To further verify the generalization of our model, the Sch-net is tested on open access LANNA children speech database for specific language impairment detection. The results show that our model achieves 99.52% accuracy in classifying patients with SLI and healthy controls. Our code will be available at https://github.com/Scu-sen/Sch-net .

CONCLUSIONS

Extensive experiments show that the proposed Sch-net can provide aided information for the diagnosis of schizophrenia and specific language impairment.

Speech Outcomes Comparison Between Adult Velopharyngeal Insufficiency and Patients With Unrepaired Cleft Palate

OBJECTIVE

This study compared the speech outcomes of adult velopharyngeal insufficiency patients and adult cleft palate (ACP) patients, and explored whether there was any difference in the phonological level of these 2 types of patients.

METHODS

Perceptual evaluation was used to assess speech intelligibility, hypernasality and compensatory articulation in 89 adult patients with velopharyngeal insufficiency and 35 adult patients with unrepaired cleft palate. Each group was divided into complete cleft palate and incomplete cleft palate (including submucous cleft palate). The phonological differences were compared between the 2 groups of patients and 2 types of cleft palate.

RESULTS

The mean speech intelligibility was 43.04% in velopharyngeal insufficiency group and 32.87% in ACP group. There was a significant difference in speech intelligibility between the 2 groups by T test, t = 2.916 (P < 0.01), speech intelligibility between 2 types of cleft palate was no significant difference. Also, there was a significant difference between the 2 groups in the constitution of hypernasality degree by Chi-Square test, x2 = 31.650 (P < 0.01), compensatory articulation were present in 74.3% ACP patients (26/35) and 47.2% velopharyngeal insufficiency patients (42/89), x2 = 7.446 (P < 0.01), there was a significant difference in incidence of compensatory articulation between the 2 groups.

CONCLUSIONS

Adult patients with unpaired cleft palate present an even worse speech intelligibility and hypernasality degree than velopharyngeal insufficiency patients after cleft palate repair, regardless of the cleft type. Additionally, patients in ACP group have a higher incidence of compensatory articulation than that in incomplete cleft palate group. In sequenced treatments of cleft lip and palate, evaluation and treatment of speech disorders cannot be ignored.

Robust Estimation of Hypernasality in Dysarthria with Acoustic Model Likelihood Features

Hypernasality is a common characteristic symptom across many motor-speech disorders. For voiced sounds, hypernasality introduces an additional resonance in the lower frequencies and, for unvoiced sounds, there is reduced articulatory precision due to air escaping through the nasal cavity. However, the acoustic manifestation of these symptoms is highly variable, making hypernasality estimation very challenging, both for human specialists and automated systems. Previous work in this area relies on either engineered features based on statistical signal processing or machine learning models trained on clinical ratings. Engineered features often fail to capture the complex acoustic patterns associated with hypernasality, whereas metrics based on machine learning are prone to overfitting to the small disease-specific speech datasets on which they are trained. Here we propose a new set of acoustic features that capture these complementary dimensions. The features are based on two acoustic models trained on a large corpus of healthy speech. The first acoustic model aims to measure nasal resonance from voiced sounds, whereas the second acoustic model aims to measure articulatory imprecision from unvoiced sounds. To demonstrate that the features derived from these acoustic models are specific to hypernasal speech, we evaluate them across different dysarthria corpora. Our results show that the features generalize even when training on hypernasal speech from one disease and evaluating on hypernasal speech from another disease (e.g., training on Parkinson's disease, evaluation on Huntington's disease), and when training on neurologically disordered speech but evaluating on cleft palate speech.

Automatic hypernasality grade assessment in cleft palate speech based on the spectral envelope method

Due to velopharyngeal incompetence, airflow overflows from the oral cavity to the nasal cavity, which results in hypernasality. Hypernasality greatly reduces speech intelligibility and affects the daily communication of patients with cleft palate. Accurate assessment of hypernasality grades can provide assisted diagnosis for speech-language pathologists (SLPs) in clinical settings. Utilizing a support vector machine (SVM), this paper classifies speech recordings into four grades (normal, mild, moderate and severe hypernasality) based on vocal tract characteristics. Linear prediction (LP) analysis is widely used to model the vocal tract. Glottal source information may be included in the LP-based spectrum. The stabilized weighted linear prediction (SWLP) method, which imposes the temporal weights on the closed-phase interval of the glottal cycle, is a more robust approach for modeling the vocal tract. The extended weighted linear prediction (XLP) method weights each lagged speech signal separately, which achieves a finer time scale on the spectral envelope than the SWLP method. Tested speech recordings were collected from 60 subjects with cleft palate and 20 control subjects, and included a total of 4640 Mandarin syllables. The experimental results showed that the spectral envelope of normal speech decreases faster than that of hypernasal speech in the high-frequency part. The experimental results also indicate that the SWLP- and XLP-based methods have smaller correlation coefficients between normal and hypernasal speech than the LP method. Thus, the SWLP and XLP methods have better ability to distinguish hypernasal from normal speech than the LP method. The classification accuracies of the four hypernasality grades using the SWLP and XLP methods range from 83.86% to 97.47%. The selection of the model order and the size of the weight function are also discussed in this paper.

Detection and assessment of hypernasality in repaired cleft palate speech using vocal tract and residual features

The presence of hypernasality in repaired cleft palate (CP) speech is a consequence of velopharyngeal insufficiency. The coupling of the nasal tract with the oral tract adds nasal formant and antiformant pairs in the hypernasal speech spectrum. This addition deviates the spectral and linear prediction (LP) residual characteristics of hypernasal speech compared to normal speech. In this work, the vocal tract constriction feature, peak to side-lobe ratio feature, and spectral moment features augmented by low-order cepstral coefficients are used to capture the spectral and residual deviations for hypernasality detection. The first feature captures the lower-frequencies prominence in speech due to the presence of nasal formants, the second feature captures the undesirable signal components in the residual signal due to the nasal antiformants, and the third feature captures the information about formants and antiformants in the spectrum along with the spectral envelope. The combination of three features gives normal versus hypernasal speech detection accuracies of 87.76%, 91.13%, and 93.70% for /a/, /i/, and /u/ vowels, respectively, and hypernasality severity detection accuracies of 80.13% and 81.25% for /i/ and /u/ vowels, respectively. The speech data are collected from 30 control normal and 30 repaired CP children between the ages of 7 and 12.

HypernasalityNet: Deep recurrent neural network for automatic hypernasality detection

BACKGROUND

Cleft palate patients have inability to produce adequate velopharyngeal closure, which results in hypernasal speech. In clinic, hypernasal speech is assessed through subject assessment by speech language pathologists. Automatic hypernasal speech detection can provide aided diagnoses for speech language pathologists and clinicians.

OBJECTIVES

This study aims to develop Long Short-Term Memory (LSTM) based Deep Recurrent Neural Network (DRNN) system to detect hypernasal speech from cleft palate patients, thus to provide aided diagnoses for clinical operation and speech therapy. Meanwhile, the feature mining and classification abilities of LSTM-DRNN system are explored.

METHODS

The utilized speech recordings are 14,544 vowels in Mandarin. Speech data is collected from 144 children (72 children with hypernasality and 72 controls) with the age of 5-12 years old. This work proposes a LSTM based DRNN system to achieve automatic hypernasal speech detection, since LSTM-DRNN can learn short-time dependences of hypernasal speech. The vocal tract based features are fed into LSTM-DRNN to achieve deep mining of features. To verify the feature mining ability of LSTM-DRNN, features projected by LSTM-DRNN are fed into shallow classifiers instead of the following two fully connected layers and a softmax layer. And the features without the projecting process of LSTM-DRNN are directly fed into shallow classifiers as a comparison. Hypernasality-sensitive vowels (/a/, /i/, and /u/) are analyzed for the first time.

RESULTS

This LSTM-DRNN based hypernasal speech detection method reaches higher detection accuracy than that using shallow classifiers, since LSTM-DRNN mines features through time axis and network depth simultaneously. The proposed LSTM-DRNN based hypernasality detection system reaches the highest accuracy of 93.35%. According to the analysis of hypernasality-sensitive vowels, the experimental result concludes that vowels /i/ and /u/ are the most sensitive vowels to hypernasal speech.

CONCLUSIONS

The results show that LSTM-DRNN has robust feature mining ability and classification ability. This is the first work that applies the LSTM-DRNN technique to automatically detect hypernasality in cleft palate speech. The experimental results demonstrate the potential of deep learning on pathologist speech detection.

Automatic initial and final segmentation in cleft palate speech of Mandarin speakers

The speech unit segmentation is an important pre-processing step in the analysis of cleft palate speech. In Mandarin, one syllable is composed of two parts: initial and final. In cleft palate speech, the resonance disorders occur at the finals and the voiced initials, while the articulation disorders occur at the unvoiced initials. Thus, the initials and finals are the minimum speech units, which could reflect the characteristics of cleft palate speech disorders. In this work, an automatic initial/final segmentation method is proposed. It is an important preprocessing step in cleft palate speech signal processing. The tested cleft palate speech utterances are collected from the Cleft Palate Speech Treatment Center in the Hospital of Stomatology, Sichuan University, which has the largest cleft palate patients in China. The cleft palate speech data includes 824 speech segments, and the control samples contain 228 speech segments. The syllables are extracted from the speech utterances firstly. The proposed syllable extraction method avoids the training stage, and achieves a good performance for both voiced and unvoiced speech. Then, the syllables are classified into with "quasi-unvoiced" or with "quasi-voiced" initials. Respective initial/final segmentation methods are proposed to these two types of syllables. Moreover, a two-step segmentation method is proposed. The rough locations of syllable and initial/final boundaries are refined in the second segmentation step, in order to improve the robustness of segmentation accuracy. The experiments show that the initial/final segmentation accuracies for syllables with quasi-unvoiced initials are higher than quasi-voiced initials. For the cleft palate speech, the mean time error is 4.4ms for syllables with quasi-unvoiced initials, and 25.7ms for syllables with quasi-voiced initials, and the correct segmentation accuracy P30 for all the syllables is 91.69%. For the control samples, P30 for all the syllables is 91.24%.

Influence of gender and age on the Nasality Severity Index 2.0 in Dutch-speaking Flemish children and adults

This study aimed to explore the influence of gender and age on the Nasality Severity Index 2.0 (NSI 2.0), an instrumental multiparametric index to determine hypernasality. Additionally, reference values will be established for this new index. Influence of gender and age on the NSI 2.0 was explored in 80 Flemish-speaking children (4-12 years; 40 boys, 40 girls) and 60 Flemish-speaking adults (18-60 years, 30 men, 30 women) without resonance disorders by determining its incorporated acoustic parameters: nasalance of the vowel/u/and an oral text, determined by a Nasometer, and voice low tone to high tone ratio (VLHR) of the vowel/i/. The equation yields NSI 2.0 = 13.20 - (0.0824 × nasalance/u/(%)) - (0.26 × nasalance oral text (%)) - (0.242 × VLHR/i/4.47*F0 Hz (dB)). No effect of gender or age was found on the NSI 2.0 in children. However, significant differences were found for the NSI 2.0, nasalance of/u/and an oral text between adult men and women. Additionally, an interaction effect between gender and age was found for these parameters. Consequently, separate reference values for the NSI 2.0 in children, adult men and adult women were established. Based on these reference scores, deviation of the NSI 2.0 score in patients with resonance disorders can be defined, which can determine the need for (additional) intervention. Further research can explore the possible influence of language on the index.

Short-term and long-term test-retest reliability of the Nasality Severity Index 2.0

PURPOSE

The Nasality Severity Index 2.0 (NSI 2.0) forms a new, multiparametric approach in the assessment of hypernasality. To enable clinical implementation of this index, the short- and long-term test-retest reliability of this index was explored.

METHODS

In 40 normal-speaking adults (mean age 32y, SD 11, 18-56y) and 29 normal-speaking children (mean age 8y, SD 2, 4-12y), the acoustic parameters included in the NSI 2.0 (i.e. nasalance of the vowel /u/ and an oral text, and the voice low tone to high tone ratio (VLHR) of the vowel /i/) were obtained twice at the same test moment and during a second assessment two weeks later. After determination of the NSI 2.0, a comprehensive set of statistical measures was applied to determine its reliability.

RESULTS

Long-term variability of the NSI 2.0 and its parameters was slightly higher compared to the short-term variability, both in adults and in children. Overall, a difference of 2.82 for adults and 2.68 for children between the results of two consecutive measurements can be interpreted as a genuine change. With an ICC of 0.84 in adults and 0.77 in children, the NSI 2.0 additionally shows an excellent relative consistency. No statistically significant difference was withheld in the reliability of test-retest measurements between adults and children.

CONCLUSION

Reliable test-retest measurements of the NSI 2.0 can be performed. Consequently, the NSI 2.0 can be applied in clinical practice, in which successive NSI 2.0 scores can be reliably compared and interpreted.

LEARNING OUTCOMES

The reader will be able to describe and discuss both the short-term and long-term test-retest reliability of the Nasality Severity Index 2.0, a new multiparametric approach to hypernasality, and its parameters. Based on this information, the NSI 2.0 can be applied in clinical practice, in which successive NSI 2.0 scores, e.g. before and after surgery or speech therapy, can be compared and interpreted.

Instrumental assessment of velopharyngeal function and resonance: a review

The purpose of this literature review is to describe and discuss instrumental assessment techniques of the velopharyngeal function in order to diagnose velopharyngeal disorders and resonance characteristics. Both direct and indirect assessment techniques are addressed, in which successively nasopharyngoscopy, videofluoroscopy, magnetic resonance imaging (MRI), cephalometric radiographic analysis, computed tomography (CT), ultrasound, acoustic and aerodynamic measurements are considered. Despite the multiple instrumental assessments available to detect and define velopharyngeal dysfunction, the ideal technique is not yet accessible. Therefore, a combination of different quantitative parameters can possibly form a solution for a more reliable determination of resonance disorders. These multi-dimensional approaches will be described and discussed. The combination of quantitative measurement techniques and perceptual evaluation of nasality will probably remain necessary to provide sufficient information to make appropriate decisions concerning the diagnosis and treatment of resonance disorders.

LEARNING OUTCOMES

The reader will be able to describe and discuss currently available instrumental techniques to assess the velopharyngeal mechanism and its functioning in order to diagnose velopharyngeal disorders. Additionally, he will be able to explain the possible advantages of the combination of several types of complementary measurement techniques.

Acoustic analysis and detection of hypernasality using a group delay function

In this paper, we describe a group delay-based signal processing technique for the analysis and detection of hypernasal speech. Our preliminary acoustic analysis on nasalized vowels shows that, even though additional resonances are introduced at various frequency locations, the introduction of a new resonance in the low-frequency region (around 250 Hz) is found to be consistent. This observation is further confirmed by a perceptual analysis carried out on vowel sounds that are modified by introducing different nasal resonances, and an acoustic analysis on hypernasal speech. Based on this, for subsequent experiments the focus is given only to the low-frequency region. The additive property of the group delay function can be exploited to resolve two closely spaced formants. However, when the formants are very close with considerably wider bandwidths as in hypernasal speech, the group delay function also fails to resolve. To overcome this, we suggest a band-limited approach to estimate the locations of the formants. Using the band-limited group delay spectrum, we define a new acoustic measure for the detection of hypernasality. Experiments are carried out on the phonemes /a/, /i/, and /u/ uttered by 33 hypernasal speakers and 30 normal speakers. Using the group delay-based acoustic measure, the performance on a hypernasality detection task is found to be 100% for /a/, 88.78% for /i/ and 86.66% for /u/. The effectiveness of this acoustic measure is further cross-verified on a speech data collected in an entirely different recording environment.