Can Natural Speech Prosody Distinguish Autism Spectrum Disorders? A Meta-Analysis

Natural speech plays a pivotal role in communication and interactions between human beings. The prosody of natural speech, due to its high ecological validity and sensitivity, has been acoustically analyzed and more recently utilized in machine learning to identify individuals with autism spectrum disorders (ASDs). In this meta-analysis, we evaluated the findings of empirical studies on acoustic analysis and machine learning techniques to provide statistically supporting evidence for adopting natural speech prosody for ASD detection. Using a random-effects model, the results observed moderate-to-large pooled effect sizes for pitch-related parameters in distinguishing individuals with ASD from their typically developing (TD) counterparts. Specifically, the standardized mean difference (SMD) values for pitch mean, pitch range, pitch standard deviation, and pitch variability were 0.3528, 0.6744, 0.5735, and 0.5137, respectively. However, the differences between the two groups in temporal features could be unreliable, as the SMD values for duration and speech rate were only 0.0738 and -0.0547. Moderator analysis indicated task types were unlikely to influence the final results, whereas age groups showed a moderating role in pooling pitch range differences. Furthermore, promising accuracy rates on ASD identification were shown in our analysis of multivariate machine learning studies, indicating averaged sensitivity and specificity of 75.51% and 80.31%, respectively. In conclusion, these findings shed light on the efficacy of natural prosody in identifying ASD and offer insights for future investigations in this line of research.

Oromotor skills in autism spectrum disorder: A scoping review

Oromotor functioning plays a foundational role in spoken communication and feeding, two areas of significant difficulty for many autistic individuals. However, despite years of research and established differences in gross and fine motor skills in this population, there is currently no clear consensus regarding the presence or nature of oral motor control deficits in autistic individuals. In this scoping review, we summarize research published between 1994 and 2022 to answer the following research questions: (1) What methods have been used to investigate oromotor functioning in autistic individuals? (2) Which oromotor behaviors have been investigated in this population? and (3) What conclusions can be drawn regarding oromotor skills in this population? Seven online databases were searched resulting in 107 studies meeting our inclusion criteria. Included studies varied widely in sample characteristics, behaviors analyzed, and research methodology. The large majority (81%) of included studies report a significant oromotor abnormality related to speech production, nonspeech oromotor skills, or feeding within a sample of autistic individuals based on age norms or in comparison to a control group. We examine these findings to identify trends, address methodological aspects hindering cross-study synthesis and generalization, and provide suggestions for future research.

Language in autism: domains, profiles and co-occurring conditions

This article reviews the current knowledge state on pragmatic and structural language abilities in autism and their potential relation to extralinguistic abilities and autistic traits. The focus is on questions regarding autism language profiles with varying degrees of (selective) impairment and with respect to potential comorbidity of autism and language impairment: Is language impairment in autism the co-occurrence of two distinct conditions (comorbidity), a consequence of autism itself (no comorbidity), or one possible combination from a series of neurodevelopmental properties (dimensional approach)? As for language profiles in autism, three main groups are identified, namely, (i) verbal autistic individuals without structural language impairment, (ii) verbal autistic individuals with structural language impairment, and (iii) minimally verbal autistic individuals. However, this tripartite distinction hides enormous linguistic heterogeneity. Regarding the nature of language impairment in autism, there is currently no model of how language difficulties may interact with autism characteristics and with various extralinguistic cognitive abilities. Building such a model requires carefully designed explorations that address specific aspects of language and extralinguistic cognition. This should lead to a fundamental increase in our understanding of language impairment in autism, thereby paving the way for a substantial contribution to the question of how to best characterize neurodevelopmental disorders.

Individuals with autism spectrum disorder are impaired in absolute but not relative pitch and duration matching in speech and song imitation

Individuals with autism spectrum disorder (ASD) often exhibit atypical imitation. However, few studies have identified clear quantitative characteristics of vocal imitation in ASD. This study investigated imitation of speech and song in English-speaking individuals with and without ASD and its modulation by age. Participants consisted of 25 autistic children and 19 autistic adults, who were compared to 25 children and 19 adults with typical development matched on age, gender, musical training, and cognitive abilities. The task required participants to imitate speech and song stimuli with varying pitch and duration patterns. Acoustic analyses of the imitation performance suggested that individuals with ASD were worse than controls on absolute pitch and duration matching for both speech and song imitation, although they performed as well as controls on relative pitch and duration matching. Furthermore, the two groups produced similar numbers of pitch contour, pitch interval-, and time errors. Across both groups, sung pitch was imitated more accurately than spoken pitch, whereas spoken duration was imitated more accurately than sung duration. Children imitated spoken pitch more accurately than adults when it came to speech stimuli, whereas age showed no significant relationship to song imitation. These results reveal a vocal imitation deficit across speech and music domains in ASD that is specific to absolute pitch and duration matching. This finding provides evidence for shared mechanisms between speech and song imitation, which involves independent implementation of relative versus absolute features. LAY SUMMARY: Individuals with autism spectrum disorder (ASD) often exhibit atypical imitation of actions and gestures. Characteristics of vocal imitation in ASD remain unclear. By comparing speech and song imitation, this study shows that individuals with ASD have a vocal imitative deficit that is specific to absolute pitch and duration matching, while performing as well as controls on relative pitch and duration matching, across speech and music domains.

An Acoustic Characterization of Prosodic Differences in Autism Spectrum Disorder and First-Degree Relatives

This study examined prosody through characterization of acoustic properties of the speech of individuals with ASD and their parents, during narration. A subset of utterances were low-pass filtered and rated for differences in intonation, speech rate, and rhythm. Listener ratings were minimally related to acoustic measures, underscoring the complexity of atypical prosody in ASD. Acoustic analyses revealed greater utterance-final fundamental frequency excursion size and slower speech rate in the ASD group. Slower speech rate was also evident in the ASD parent group, particularly parents with the broad autism phenotype. Overlapping prosodic differences in ASD and ASD Parent groups suggest that prosodic differences may constitute an important phenotype contributing to ASD features and index genetic liability to ASD among first-degree relatives.

Lexical stress contrastivity in Italian children with autism spectrum disorders: an exploratory acoustic study

We investigated production of lexical stress in children with and without autism spectrum disorders (ASD), all monolingual Italian speakers. The mean age of the 16 autistic children was 5.73 years and the mean age of the 16 typically developing children was 4.65 years. Picture-naming targets were five trisyllabic words that began with a weak-strong pattern of lexical stress across the initial two syllables (WS: matita) and five trisyllabic words beginning with a strong-weak pattern (SW: gomito). Acoustic measures of the duration, fundamental frequency, and intensity of the first two vowels for correct word productions were used to calculate a normalised Pairwise Variability Index (PVI) for WS and SW words. Results of acoustic analyses indicated no statistically significant group differences in PVIs. Results should be interpreted in line with the exploratory nature of this study. We hope this study will encourage additional cross-linguistic studies of prosody in children's speech production.

Cross-Modal Coordination of Face-Directed Gaze and Emotional Speech Production in School-Aged Children and Adolescents with ASD

Autism spectrum disorder involves persistent difficulties in social communication. Although these difficulties affect both verbal and nonverbal communication, there are no quantitative behavioral studies to date investigating the cross-modal coordination of verbal and nonverbal communication in autism. The objective of the present study was to characterize the dynamic relation between speech production and facial expression in children with autism and to establish how face-directed gaze modulates this cross-modal coordination. In a dynamic mimicry task, experiment participants watched and repeated neutral and emotional spoken sentences with accompanying facial expressions. Analysis of audio and motion capture data quantified cross-modal coordination between simultaneous speech production and facial expression. Whereas neurotypical children produced emotional sentences with strong cross-modal coordination and produced neutral sentences with weak cross-modal coordination, autistic children produced similar levels of cross-modal coordination for both neutral and emotional sentences. An eyetracking analysis revealed that cross-modal coordination of speech production and facial expression was greater when the neurotypical child spent more time looking at the face, but weaker when the autistic child spent more time looking at the face. In sum, social communication difficulties in autism spectrum disorder may involve deficits in cross-modal coordination. This finding may inform how autistic individuals are perceived in their daily conversations.

An acoustic study of lexical stress contrastivity in children with and without autism spectrum disorders

In this exploratory study, we examined stress contrastivity within real word productions elicited via picture naming in 20 children with autism spectrum disorders (ASD) and 20 typical peers group-wise matched on age and vocabulary. Targets had a dominant pattern of lexical stress beginning with a strong-weak pattern (SW: 'caterpillar', 'butterfly') or a non-dominant pattern of lexical stress beginning with a weak-strong pattern (WS: 'tomato', 'potato'). Children produced each target twice (n = 320 productions). Acoustic measures were made for the duration, fundamental frequency, and intensity of the first two vowels for each word production. For vowel duration and fundamental frequency, children with ASD and typical peers produced a similar magnitude of stress contrastivity for SW and WS words. However, there was a significant group difference in the way contrastivity in intensity was realised for WS words whereby children with ASD produced less stress contrastivity than typical peers. Bayesian analyses were in line with our interpretation of our frequentist analyses.

A Systematic Review of Speech Assessments for Children With Autism Spectrum Disorder: Recommendations for Best Practice

PURPOSE

The purpose of this systematic review was to provide a summary and evaluation of speech assessments used with children with autism spectrum disorders (ASD). A subsequent narrative review was completed to ascertain the core components of an evidence-based pediatric speech assessment, which, together with the results of the systematic review, provide clinical and research guidelines for best practice.

METHOD

A systematic search of eight databases was used to find peer-reviewed research articles published between 1990 and 2014 assessing the speech of children with ASD. Eligible articles were categorized according to the assessment methods used and the speech characteristics described.

RESULTS

The review identified 21 articles that met the inclusion criteria, search criteria, and confidence in ASD diagnosis. The speech of prelinguistic participants was assessed in seven articles. Speech assessments with verbal participants were completed in 15 articles with segmental and suprasegmental aspects of speech analyzed. Assessment methods included connected speech samples, single-word naming tasks, speech imitation tasks, and analysis of the production of words and sentences.

CONCLUSIONS

Clinical and research guidelines for speech assessment of children with ASD are outlined. Future comparisons will be facilitated by the use of consistent reporting methods in research focusing on children with ASD.

Detecting Abnormal Word Utterances in Children With Autism Spectrum Disorders: Machine-Learning-Based Voice Analysis Versus Speech Therapists

Abnormal prosody is often evident in the voice intonations of individuals with autism spectrum disorders. We compared a machine-learning-based voice analysis with human hearing judgments made by 10 speech therapists for classifying children with autism spectrum disorders ( n = 30) and typical development ( n = 51). Using stimuli limited to single-word utterances, machine-learning-based voice analysis was superior to speech therapist judgments. There was a significantly higher true-positive than false-negative rate for machine-learning-based voice analysis but not for speech therapists. Results are discussed in terms of some artificiality of clinician judgments based on single-word utterances, and the objectivity machine-learning-based voice analysis adds to judging abnormal prosody.

Uh and um in children with autism spectrum disorders or language impairment

Atypical pragmatic language is often present in individuals with autism spectrum disorders (ASD), along with delays or deficits in structural language. This study investigated the use of the "fillers" uh and um by children ages 4-8 during the autism diagnostic observation schedule. Fillers reflect speakers' difficulties with planning and delivering speech, but they also serve communicative purposes, such as negotiating control of the floor or conveying uncertainty. We hypothesized that children with ASD would use different patterns of fillers compared to peers with typical development or with specific language impairment (SLI), reflecting differences in social ability and communicative intent. Regression analyses revealed that children in the ASD group were much less likely to use um than children in the other two groups. Filler use is an easy-to-quantify feature of behavior that, in concert with other observations, may help to distinguish ASD from SLI. Autism Res 2016, 9: 854-865. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.

The development of the Athens Emotional States Inventory (AESI): collection, validation and automatic processing of emotionally loaded sentences

OBJECTIVES

The development of ecologically valid procedures for collecting reliable and unbiased emotional data towards computer interfaces with social and affective intelligence targeting patients with mental disorders.

METHODS

Following its development, presented with, the Athens Emotional States Inventory (AESI) proposes the design, recording and validation of an audiovisual database for five emotional states: anger, fear, joy, sadness and neutral. The items of the AESI consist of sentences each having content indicative of the corresponding emotion. Emotional content was assessed through a survey of 40 young participants with a questionnaire following the Latin square design. The emotional sentences that were correctly identified by 85% of the participants were recorded in a soundproof room with microphones and cameras. A preliminary validation of AESI is performed through automatic emotion recognition experiments from speech.

RESULTS

The resulting database contains 696 recorded utterances in Greek language by 20 native speakers and has a total duration of approximately 28 min. Speech classification results yield accuracy up to 75.15% for automatically recognizing the emotions in AESI.

CONCLUSIONS

These results indicate the usefulness of our approach for collecting emotional data with reliable content, balanced across classes and with reduced environmental variability.

The psychologist as an interlocutor in autism spectrum disorder assessment: insights from a study of spontaneous prosody

PURPOSE

The purpose of this study was to examine relationships between prosodic speech cues and autism spectrum disorder (ASD) severity, hypothesizing a mutually interactive relationship between the speech characteristics of the psychologist and the child. The authors objectively quantified acoustic-prosodic cues of the psychologist and of the child with ASD during spontaneous interaction, establishing a methodology for future large-sample analysis.

METHOD

Speech acoustic-prosodic features were semiautomatically derived from segments of semistructured interviews (Autism Diagnostic Observation Schedule, ADOS; Lord, Rutter, DiLavore, & Risi, 1999; Lord et al., 2012) with 28 children who had previously been diagnosed with ASD. Prosody was quantified in terms of intonation, volume, rate, and voice quality. Research hypotheses were tested via correlation as well as hierarchical and predictive regression between ADOS severity and prosodic cues.

RESULTS

Automatically extracted speech features demonstrated prosodic characteristics of dyadic interactions. As rated ASD severity increased, both the psychologist and the child demonstrated effects for turn-end pitch slope, and both spoke with atypical voice quality. The psychologist's acoustic cues predicted the child's symptom severity better than did the child's acoustic cues.

CONCLUSION

The psychologist, acting as evaluator and interlocutor, was shown to adjust his or her behavior in predictable ways based on the child's social-communicative impairments. The results support future study of speech prosody of both interaction partners during spontaneous conversation, while using automatic computational methods that allow for scalable analysis on much larger corpora.

ECOLOGICALLY VALID LONG-TERM MOOD MONITORING OF INDIVIDUALS WITH BIPOLAR DISORDER USING SPEECH

Speech patterns are modulated by the emotional and neurophysiological state of the speaker. There exists a growing body of work that computationally examines this modulation in patients suffering from depression, autism, and post-traumatic stress disorder. However, the majority of the work in this area focuses on the analysis of structured speech collected in controlled environments. Here we expand on the existing literature by examining bipolar disorder (BP). BP is characterized by mood transitions, varying from a healthy euthymic state to states characterized by mania or depression. The speech patterns associated with these mood states provide a unique opportunity to study the modulations characteristic of mood variation. We describe methodology to collect unstructured speech continuously and unobtrusively via the recording of day-to-day cellular phone conversations. Our pilot investigation suggests that manic and depressive mood states can be recognized from this speech data, providing new insight into the feasibility of unobtrusive, unstructured, and continuous speech-based wellness monitoring for individuals with BP.

QUANTIFYING ATYPICALITY IN AFFECTIVE FACIAL EXPRESSIONS OF CHILDREN WITH AUTISM SPECTRUM DISORDERS

We focus on the analysis, quantification and visualization of atypicality in affective facial expressions of children with High Functioning Autism (HFA). We examine facial Motion Capture data from typically developing (TD) children and children with HFA, using various statistical methods, including Functional Data Analysis, in order to quantify atypical expression characteristics and uncover patterns of expression evolution in the two populations. Our results show that children with HFA display higher asynchrony of motion between facial regions, more rough facial and head motion, and a larger range of facial region motion. Overall, subjects with HFA consistently display a wider variability in the expressive facial gestures that they employ. Our analysis demonstrates the utility of computational approaches for understanding behavioral data and brings new insights into the autism domain regarding the atypicality that is often associated with facial expressions of subjects with HFA.

Emotional facial and vocal expressions during story retelling by children and adolescents with high-functioning autism

PURPOSE

People with high-functioning autism (HFA) have qualitative differences in facial expression and prosody production, which are rarely systematically quantified. The authors' goals were to qualitatively and quantitatively analyze prosody and facial expression productions in children and adolescents with HFA.

METHOD

Participants were 22 male children and adolescents with HFA and 18 typically developing (TD) controls (17 males, 1 female). The authors used a story retelling task to elicit emotionally laden narratives, which were analyzed through the use of acoustic measures and perceptual codes. Naïve listeners coded all productions for emotion type, degree of expressiveness, and awkwardness.

RESULTS

The group with HFA was not significantly different in accuracy or expressiveness of facial productions, but was significantly more awkward than the TD group. Participants with HFA were significantly more expressive in their vocal productions, with a trend for greater awkwardness. Severity of social communication impairment, as captured by the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 1999), was correlated with greater vocal and facial awkwardness.

CONCLUSIONS

Facial and vocal expressions of participants with HFA were as recognizable as those of their TD peers but were qualitatively different, particularly when listeners coded samples with intact dynamic properties. These preliminary data show qualitative differences in nonverbal communication that may have significant negative impact on the social communication success of children and adolescents with HFA.

Quantifying repetitive speech in autism spectrum disorders and language impairment

We report on an automatic technique for quantifying two types of repetitive speech: repetitions of what the child says him/herself (self-repeats) and of what is uttered by an interlocutor (echolalia). We apply this technique to a sample of 111 children between the ages of four and eight: 42 typically developing children (TD), 19 children with specific language impairment (SLI), 25 children with autism spectrum disorders (ASD) plus language impairment (ALI), and 25 children with ASD with normal, non-impaired language (ALN). The results indicate robust differences in echolalia between the TD and ASD groups as a whole (ALN + ALI), and between TD and ALN children. There were no significant differences between ALI and SLI children for echolalia or self-repetitions. The results confirm previous findings that children with ASD repeat the language of others more than other populations of children. On the other hand, self-repetition does not appear to be significantly more frequent in ASD, nor does it matter whether the child's echolalia occurred within one (immediate) or two turns (near-immediate) of the adult's original utterance. Furthermore, non-significant differences between ALN and SLI, between TD and SLI, and between ALI and TD are suggestive that echolalia may not be specific to ALN or to ASD in general. One important innovation of this work is an objective fully automatic technique for assessing the amount of repetition in a transcript of a child's utterances.