Computational prosodic markers for autism

Successful lexical tone production of Mandarin Chinese autistic children with intellectual impairment

BACKGROUND

Atypical speech prosody has been commonly found among autistic children. Yet it remains unknown whether prosody impairment originates from poor pitch ability in general or whether it is the result of the difficulty in understanding and using prosody for communicative purposes.

AIMS

To investigate whether native Mandarin Chinese-speaking autistic children with intellectual impairment were able to accurately produce native lexical tones, which are pitch patterns that distinguish word meaning lexically and serve little social purpose.

METHODS & PROCEDURES

Using a picture-naming task, thirteen 8-13-year-old Mandarin Chinese-speaking autistic children with intellectual impairment were tested on their production of Chinese lexical tones. Chronical age-matched typically developing (TD) children were included as the control group. Perceptual assessment and phonetic analyses were conducted with the produced lexical tones.

OUTCOMES & RESULTS

The majority of the lexical tones produced by the autistic children were perceived as accurate by adult judges. Phonetic analysis of the pitch contours found no significant difference between the two groups, and the autistic children and TD children used the phonetic features in comparable ways when differentiating the lexical tones. However, the lexical tone accuracy rate was lower among the autistic children than among the TDs, and the larger individual difference was observed among the autistic children than the TD children.

CONCLUSIONS & IMPLICATIONS

These results indicate that autistic children are able to produce the global contours of the lexical tones, and pitch deficits do not seem to qualify as a core feature of autism.

WHAT THIS PAPER ADDS

What is already known on the subject Atypical prosody has been considered a maker of the speech of autistic children, and meta-analysis found a significant difference in mean pitch and pitch range between TD children and autistic children. Yet it remains unknown whether the pitch deficits are the result of impaired perceptual-motoric ability or if they reflect failure in learning sentential prosody, which requires an understanding of the interlocutors' mind. In addition, research on pitch ability of autistic children with intellectual disabilities has been scarce, and whether these children are able to produce pitch variation is largely unknown. What this paper adds to existing knowledge We tested native Mandarin Chinese autistic children with intellectual impairment on their production of native lexical tones. The lexical tones in Chinese are pitch variations realized on individual syllables that distinguish lexical meaning, but they do not serve social pragmatic purposes. We found that although these autistic children had only developed limited spoken language, the majority of their lexical tones were perceived as accurate. They were able to use the phonetic features in comparable ways with the TD children when distinguishing the lexical tones. What are the potential or actual clinical implications of this work? It seems unlikely that pitch processing at the lexical level is fundamentally impaired in autistic children, and pitch deficits do not seem to qualify for a core feature of their speech. Practitioners should be cautious when using pitch production as a clinical marker for autistic children.

Oromotor skills in autism spectrum disorder: A scoping review

Oromotor functioning plays a foundational role in spoken communication and feeding, two areas of significant difficulty for many autistic individuals. However, despite years of research and established differences in gross and fine motor skills in this population, there is currently no clear consensus regarding the presence or nature of oral motor control deficits in autistic individuals. In this scoping review, we summarize research published between 1994 and 2022 to answer the following research questions: (1) What methods have been used to investigate oromotor functioning in autistic individuals? (2) Which oromotor behaviors have been investigated in this population? and (3) What conclusions can be drawn regarding oromotor skills in this population? Seven online databases were searched resulting in 107 studies meeting our inclusion criteria. Included studies varied widely in sample characteristics, behaviors analyzed, and research methodology. The large majority (81%) of included studies report a significant oromotor abnormality related to speech production, nonspeech oromotor skills, or feeding within a sample of autistic individuals based on age norms or in comparison to a control group. We examine these findings to identify trends, address methodological aspects hindering cross-study synthesis and generalization, and provide suggestions for future research.

Distinctive prosodic features of people with autism spectrum disorder: a systematic review and meta-analysis study

In this systematic review, we analyzed and evaluated the findings of studies on prosodic features of vocal productions of people with autism spectrum disorder (ASD) in order to recognize the statistically significant, most confirmed and reliable prosodic differences distinguishing people with ASD from typically developing individuals. Using suitable keywords, three major databases including Web of Science, PubMed and Scopus, were searched. The results for prosodic features such as mean pitch, pitch range and variability, speech rate, intensity and voice duration were extracted from eligible studies. The pooled standard mean difference between ASD and control groups was extracted or calculated. Using I2 statistic and Cochrane Q-test, between-study heterogeneity was evaluated. Furthermore, publication bias was assessed using funnel plot and its significance was evaluated using Egger's and Begg's tests. Thirty-nine eligible studies were retrieved (including 910 and 850 participants for ASD and control groups, respectively). This systematic review and meta-analysis showed that ASD group members had a significantly larger mean pitch (SMD =  - 0.4, 95% CI [- 0.70, - 0.10]), larger pitch range (SMD =  - 0.78, 95% CI [- 1.34, - 0.21]), longer voice duration (SMD =  - 0.43, 95% CI [- 0.72, - 0.15]), and larger pitch variability (SMD = - 0.46, 95% CI [- 0.84, - 0.08]), compared with typically developing control group. However, no significant differences in pitch standard deviation, voice intensity and speech rate were found between groups. Chronological age of participants and voice elicitation tasks were two sources of between-study heterogeneity. Furthermore, no publication bias was observed during analyses (p > 0.05). Mean pitch, pitch range, pitch variability and voice duration were recognized as the prosodic features reliably distinguishing people with ASD from TD individuals.

An Acoustic Characterization of Prosodic Differences in Autism Spectrum Disorder and First-Degree Relatives

This study examined prosody through characterization of acoustic properties of the speech of individuals with ASD and their parents, during narration. A subset of utterances were low-pass filtered and rated for differences in intonation, speech rate, and rhythm. Listener ratings were minimally related to acoustic measures, underscoring the complexity of atypical prosody in ASD. Acoustic analyses revealed greater utterance-final fundamental frequency excursion size and slower speech rate in the ASD group. Slower speech rate was also evident in the ASD parent group, particularly parents with the broad autism phenotype. Overlapping prosodic differences in ASD and ASD Parent groups suggest that prosodic differences may constitute an important phenotype contributing to ASD features and index genetic liability to ASD among first-degree relatives.

Cross-Modal Coordination of Face-Directed Gaze and Emotional Speech Production in School-Aged Children and Adolescents with ASD

Autism spectrum disorder involves persistent difficulties in social communication. Although these difficulties affect both verbal and nonverbal communication, there are no quantitative behavioral studies to date investigating the cross-modal coordination of verbal and nonverbal communication in autism. The objective of the present study was to characterize the dynamic relation between speech production and facial expression in children with autism and to establish how face-directed gaze modulates this cross-modal coordination. In a dynamic mimicry task, experiment participants watched and repeated neutral and emotional spoken sentences with accompanying facial expressions. Analysis of audio and motion capture data quantified cross-modal coordination between simultaneous speech production and facial expression. Whereas neurotypical children produced emotional sentences with strong cross-modal coordination and produced neutral sentences with weak cross-modal coordination, autistic children produced similar levels of cross-modal coordination for both neutral and emotional sentences. An eyetracking analysis revealed that cross-modal coordination of speech production and facial expression was greater when the neurotypical child spent more time looking at the face, but weaker when the autistic child spent more time looking at the face. In sum, social communication difficulties in autism spectrum disorder may involve deficits in cross-modal coordination. This finding may inform how autistic individuals are perceived in their daily conversations.

Oral-Motor and Lexical Diversity During Naturalistic Conversations in Adults with Autism Spectrum Disorder

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by impaired social communication and the presence of restricted, repetitive patterns of behaviors and interests. Prior research suggests that restricted patterns of behavior in ASD may be cross-domain phenomena that are evident in a variety of modalities. Computational studies of language in ASD provide support for the existence of an underlying dimension of restriction that emerges during a conversation. Similar evidence exists for restricted patterns of facial movement. Using tools from computational linguistics, computer vision, and information theory, this study tests whether cognitive-motor restriction can be detected across multiple behavioral domains in adults with ASD during a naturalistic conversation. Our methods identify restricted behavioral patterns, as measured by entropy in word use and mouth movement. Results suggest that adults with ASD produce significantly less diverse mouth movements and words than neurotypical adults, with an increased reliance on repeated patterns in both domains. The diversity values of the two domains are not significantly correlated, suggesting that they provide complementary information.

Uh and um in children with autism spectrum disorders or language impairment

Atypical pragmatic language is often present in individuals with autism spectrum disorders (ASD), along with delays or deficits in structural language. This study investigated the use of the "fillers" uh and um by children ages 4-8 during the autism diagnostic observation schedule. Fillers reflect speakers' difficulties with planning and delivering speech, but they also serve communicative purposes, such as negotiating control of the floor or conveying uncertainty. We hypothesized that children with ASD would use different patterns of fillers compared to peers with typical development or with specific language impairment (SLI), reflecting differences in social ability and communicative intent. Regression analyses revealed that children in the ASD group were much less likely to use um than children in the other two groups. Filler use is an easy-to-quantify feature of behavior that, in concert with other observations, may help to distinguish ASD from SLI. Autism Res 2016, 9: 854-865. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.

The psychologist as an interlocutor in autism spectrum disorder assessment: insights from a study of spontaneous prosody

PURPOSE

The purpose of this study was to examine relationships between prosodic speech cues and autism spectrum disorder (ASD) severity, hypothesizing a mutually interactive relationship between the speech characteristics of the psychologist and the child. The authors objectively quantified acoustic-prosodic cues of the psychologist and of the child with ASD during spontaneous interaction, establishing a methodology for future large-sample analysis.

METHOD

Speech acoustic-prosodic features were semiautomatically derived from segments of semistructured interviews (Autism Diagnostic Observation Schedule, ADOS; Lord, Rutter, DiLavore, & Risi, 1999; Lord et al., 2012) with 28 children who had previously been diagnosed with ASD. Prosody was quantified in terms of intonation, volume, rate, and voice quality. Research hypotheses were tested via correlation as well as hierarchical and predictive regression between ADOS severity and prosodic cues.

RESULTS

Automatically extracted speech features demonstrated prosodic characteristics of dyadic interactions. As rated ASD severity increased, both the psychologist and the child demonstrated effects for turn-end pitch slope, and both spoke with atypical voice quality. The psychologist's acoustic cues predicted the child's symptom severity better than did the child's acoustic cues.

CONCLUSION

The psychologist, acting as evaluator and interlocutor, was shown to adjust his or her behavior in predictable ways based on the child's social-communicative impairments. The results support future study of speech prosody of both interaction partners during spontaneous conversation, while using automatic computational methods that allow for scalable analysis on much larger corpora.

ECOLOGICALLY VALID LONG-TERM MOOD MONITORING OF INDIVIDUALS WITH BIPOLAR DISORDER USING SPEECH

Speech patterns are modulated by the emotional and neurophysiological state of the speaker. There exists a growing body of work that computationally examines this modulation in patients suffering from depression, autism, and post-traumatic stress disorder. However, the majority of the work in this area focuses on the analysis of structured speech collected in controlled environments. Here we expand on the existing literature by examining bipolar disorder (BP). BP is characterized by mood transitions, varying from a healthy euthymic state to states characterized by mania or depression. The speech patterns associated with these mood states provide a unique opportunity to study the modulations characteristic of mood variation. We describe methodology to collect unstructured speech continuously and unobtrusively via the recording of day-to-day cellular phone conversations. Our pilot investigation suggests that manic and depressive mood states can be recognized from this speech data, providing new insight into the feasibility of unobtrusive, unstructured, and continuous speech-based wellness monitoring for individuals with BP.

QUANTIFYING ATYPICALITY IN AFFECTIVE FACIAL EXPRESSIONS OF CHILDREN WITH AUTISM SPECTRUM DISORDERS

We focus on the analysis, quantification and visualization of atypicality in affective facial expressions of children with High Functioning Autism (HFA). We examine facial Motion Capture data from typically developing (TD) children and children with HFA, using various statistical methods, including Functional Data Analysis, in order to quantify atypical expression characteristics and uncover patterns of expression evolution in the two populations. Our results show that children with HFA display higher asynchrony of motion between facial regions, more rough facial and head motion, and a larger range of facial region motion. Overall, subjects with HFA consistently display a wider variability in the expressive facial gestures that they employ. Our analysis demonstrates the utility of computational approaches for understanding behavioral data and brings new insights into the autism domain regarding the atypicality that is often associated with facial expressions of subjects with HFA.

Emotional facial and vocal expressions during story retelling by children and adolescents with high-functioning autism

PURPOSE

People with high-functioning autism (HFA) have qualitative differences in facial expression and prosody production, which are rarely systematically quantified. The authors' goals were to qualitatively and quantitatively analyze prosody and facial expression productions in children and adolescents with HFA.

METHOD

Participants were 22 male children and adolescents with HFA and 18 typically developing (TD) controls (17 males, 1 female). The authors used a story retelling task to elicit emotionally laden narratives, which were analyzed through the use of acoustic measures and perceptual codes. Naïve listeners coded all productions for emotion type, degree of expressiveness, and awkwardness.

RESULTS

The group with HFA was not significantly different in accuracy or expressiveness of facial productions, but was significantly more awkward than the TD group. Participants with HFA were significantly more expressive in their vocal productions, with a trend for greater awkwardness. Severity of social communication impairment, as captured by the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 1999), was correlated with greater vocal and facial awkwardness.

CONCLUSIONS

Facial and vocal expressions of participants with HFA were as recognizable as those of their TD peers but were qualitatively different, particularly when listeners coded samples with intact dynamic properties. These preliminary data show qualitative differences in nonverbal communication that may have significant negative impact on the social communication success of children and adolescents with HFA.

Quantifying repetitive speech in autism spectrum disorders and language impairment

We report on an automatic technique for quantifying two types of repetitive speech: repetitions of what the child says him/herself (self-repeats) and of what is uttered by an interlocutor (echolalia). We apply this technique to a sample of 111 children between the ages of four and eight: 42 typically developing children (TD), 19 children with specific language impairment (SLI), 25 children with autism spectrum disorders (ASD) plus language impairment (ALI), and 25 children with ASD with normal, non-impaired language (ALN). The results indicate robust differences in echolalia between the TD and ASD groups as a whole (ALN + ALI), and between TD and ALN children. There were no significant differences between ALI and SLI children for echolalia or self-repetitions. The results confirm previous findings that children with ASD repeat the language of others more than other populations of children. On the other hand, self-repetition does not appear to be significantly more frequent in ASD, nor does it matter whether the child's echolalia occurred within one (immediate) or two turns (near-immediate) of the adult's original utterance. Furthermore, non-significant differences between ALN and SLI, between TD and SLI, and between ALI and TD are suggestive that echolalia may not be specific to ALN or to ASD in general. One important innovation of this work is an objective fully automatic technique for assessing the amount of repetition in a transcript of a child's utterances.

Acoustic Differences In The Imitation Of Prosodic Patterns In Children With Autism Spectrum Disorders

In research, it has been difficult to characterize the prosodic production differences that have been observed clinically in Autism Spectrum Disorders (ASD). Moreover, the nature of these differences has been particularly hard to identify. This study examined one possible contributor to these perceived differences: motor planning. We examined the ability of children and adolescents with ASD to imitate prosodic patterns in comparison to a group with learning disabilities (LD) and a typically-developing (TD) comparison group. Overall, we found that both the ASD and LD groups were significantly worse at perceiving and imitating prosodic patterns than the TD comparison group. Similar to previous studies using non-imitative speech, participants with ASD showed a significantly longer duration of utterances than the two comparison groups when attempting to imitate an intonation pattern. The implications of differences in duration of utterances are discussed. This study also highlights the importance of using clinical comparison groups in studies of language performance in individuals with ASD.

The hypothesis of apraxia of speech in children with autism spectrum disorder

In a sample of 46 children aged 4-7 years with Autism Spectrum Disorder (ASD) and intelligible speech, there was no statistical support for the hypothesis of concomitant Childhood Apraxia of Speech (CAS). Perceptual and acoustic measures of participants' speech, prosody, and voice were compared with data from 40 typically-developing children, 13 preschool children with Speech Delay, and 15 participants aged 5-49 years with CAS in neurogenetic disorders. Speech Delay and Speech Errors, respectively, were modestly and substantially more prevalent in participants with ASD than reported population estimates. Double dissociations in speech, prosody, and voice impairments in ASD were interpreted as consistent with a speech attunement framework, rather than with the motor speech impairments that define CAS.