Speech-and-gesture integration in high functioning autism

Examining Phenotypical Heterogeneity and its Underlying Factors in Gesture Skills of Chinese Autistic Children: Clustering Analysis

PURPOSE

The heterogeneity of autism is well documented, but few studies have studied the heterogeneity of gesture production ability in autistic children. The present study aimed to identify subgroups of autistic children who displayed heterogeneous gesture production abilities and explore the underlying factors, including autism characteristics, intellectual ability, and language ability, that were associated with the heterogeneity.

METHODS

A total of 65 Chinese autistic children (mean age = 5;3) participated. Their autism characteristics and intellectual ability were assessed by standardized measurements. Language output and gesture production were captured from a parent-child interaction task.

RESULTS

We conducted a hierarchical cluster analysis and identified four distinct clusters. Cluster 1 and Cluster 2 both had low gesture production whereas Cluster 3 and Cluster 4 had high gesture production. Both Clusters 1 and 2 had relatively strong autism characteristics, in comparison to Clusters 3 and 4.

CONCLUSIONS

Our findings revealed that children with stronger autism characteristics may gesture less often than those with weaker characteristics. However, the relationship between language ability and intellectual ability and gesture production was not clear. These findings shed light on the directions of intervention on gesture production for autistic children, especially those with stronger autism characteristics.

Resolution of impaired multisensory processing in autism and the cost of switching sensory modality

Children with autism spectrum disorders (ASD) exhibit alterations in multisensory processing, which may contribute to the prevalence of social and communicative deficits in this population. Resolution of multisensory deficits has been observed in teenagers with ASD for complex, social speech stimuli; however, whether this resolution extends to more basic multisensory processing deficits remains unclear. Here, in a cohort of 364 participants we show using simple, non-social audiovisual stimuli that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Computational modelling indicated that multisensory processing transitions from a default state of competition to one of facilitation, and that this transition is delayed in ASD. Further analysis revealed group differences in how sensory channels are weighted, and how this is impacted by preceding cross-sensory inputs. Our findings indicate that there is a complex and dynamic interplay among the sensory systems that differs considerably in individuals with ASD.

Crosse et al. study a cohort of 364 participants with autism spectrum disorders (ASD) and matched controls, and show that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Using computational modelling they go on to demonstrate that there is a delayed transition of multisensory processing from a default state of competition to one of facilitation in ASD, as well as differences in sensory weighting and the ability to switch between sensory modalities, which sheds light on the interplay among sensory systems that differ in ASD individuals.

Differences in functional brain organization during gesture recognition between autistic and neurotypical individuals

Persons with and without autism process sensory information differently. Differences in sensory processing are directly relevant to social functioning and communicative abilities, which are known to be hampered in persons with autism. We collected functional magnetic resonance imaging data from 25 autistic individuals and 25 neurotypical individuals while they performed a silent gesture recognition task. We exploited brain network topology, a holistic quantification of how networks within the brain are organized to provide new insights into how visual communicative signals are processed in autistic and neurotypical individuals. Performing graph theoretical analysis, we calculated two network properties of the action observation network: 'local efficiency', as a measure of network segregation, and 'global efficiency', as a measure of network integration. We found that persons with autism and neurotypical persons differ in how the action observation network is organized. Persons with autism utilize a more clustered, local-processing-oriented network configuration (i.e. higher local efficiency) rather than the more integrative network organization seen in neurotypicals (i.e. higher global efficiency). These results shed new light on the complex interplay between social and sensory processing in autism.

Identifying Patterns of Similarities and Differences between Gesture Production and Comprehension in Autism and Typical Development

Production and comprehension of gesture emerge early and are key to subsequent language development in typical development. Compared to typically developing (TD) children, children with autism spectrum disorders (ASD) exhibit difficulties and/or differences in gesture production. However, we do not yet know if gesture production either shows similar patterns to gesture comprehension across different ages and learners, or alternatively, lags behind gesture comprehension, thus mimicking a pattern akin to speech comprehension and production. In this study, we focus on the gestures produced and comprehended by a group of young TD children and children with ASD—comparable in language ability—with the goal to identify whether gesture production and comprehension follow similar patterns between ages and between learners. We elicited production of gesture in a semi-structured parent–child play and comprehension of gesture in a structured experimenter-child play across two studies. We tested whether young TD children (ages 2–4) follow a similar trajectory in their production and comprehension of gesture (Study 1) across ages, and if so, whether this alignment remains similar for verbal children with ASD (M_age = 5 years), comparable to TD children in language ability (Study 2). Our results provided evidence for similarities between gesture production and comprehension across ages and across learners, suggesting that comprehension and production of gesture form a largely integrated system of communication.

Manual Gestures Modulate Early Neural Responses in Loudness Perception

How different sensory modalities interact to shape perception is a fundamental question in cognitive neuroscience. Previous studies in audiovisual interaction have focused on abstract levels such as categorical representation (e.g., McGurk effect). It is unclear whether the cross-modal modulation can extend to low-level perceptual attributes. This study used motional manual gestures to test whether and how the loudness perception can be modulated by visual-motion information. Specifically, we implemented a novel paradigm in which participants compared the loudness of two consecutive sounds whose intensity changes around the just noticeable difference (JND), with manual gestures concurrently presented with the second sound. In two behavioral experiments and two EEG experiments, we investigated our hypothesis that the visual-motor information in gestures would modulate loudness perception. Behavioral results showed that the gestural information biased the judgment of loudness. More importantly, the EEG results demonstrated that early auditory responses around 100 ms after sound onset (N100) were modulated by the gestures. These consistent results in four behavioral and EEG experiments suggest that visual-motor processing can integrate with auditory processing at an early perceptual stage to shape the perception of a low-level perceptual attribute such as loudness, at least under challenging listening conditions.

Contrast Is in the Eye of the Beholder: Infelicitous Beat Gesture Increases Cognitive Load During Online Spoken Discourse Comprehension

We investigated how two cues to contrast-beat gesture and contrastive pitch accenting-affect comprehenders' cognitive load during processing of spoken referring expressions. In two visual-world experiments, we orthogonally manipulated the presence of these cues and their felicity, or fit, with the local (sentence-level) referential context in critical referring expressions while comprehenders' task-evoked pupillary responses (TEPRs) were examined. In Experiment 1, beat gesture and contrastive accenting always matched the referential context of filler referring expressions and were therefore relatively felicitous on the global (experiment) level, whereas in Experiment 2, beat gesture and contrastive accenting never fit the referential context of filler referring expressions and were therefore infelicitous on the global level. The results revealed that both beat gesture and contrastive accenting increased comprehenders' cognitive load. For beat gesture, this increase in cognitive load was driven by both local and global infelicity. For contrastive accenting, this increase in cognitive load was unaffected when cues were globally felicitous but exacerbated when cues were globally infelicitous. Together, these results suggest that comprehenders' cognitive resources are taxed by processing infelicitous use of beat gesture and contrastive accenting to convey contrast on both the local and global levels.

Short report: Learning through iconic gesture in autism spectrum disorder

BACKGROUND

Understanding and therefore recalling spoken messages, including narratives, can be challenging for children with autism. While observing gesture can benefit narrative recall in typically developing children, whether observing gesture facilitates narrative recall in children with autism is unclear.

AIMS

This paper examines whether observing iconic gestures affects narrative recall in children with a diagnosis of autism.

METHODS AND PROCEDURES

We first identified iconic gestures to be observed by participants in the main study. Once appropriate iconic gestures had been identified, children with autism watched one video narrative with iconic gestures and one without gestures. While watching the video narratives, children wore Tobii Pro Glasses-2 to track their eye-movements. After watching each narrative, children were asked questions about the narratives to assess recall.

OUTCOMES

Iconic gestures significantly benefitted narrative recall in children with autism beyond watching no gestures, and eye-tracking results suggested gestures helped children focus on the narrator.

CONCLUSIONS AND IMPLICATIONS

Through identifying appropriate iconic gestures and producing them alongside a verbal narrative, gestures may successfully enhance learning in children with autism.

The interaction of fine motor, gesture, and structural language skills: The case of autism spectrum disorder

Motor skill differences have been consistently reported in individuals with ASD. Associations between motor skill and social communication skills have been reported in both typical development (TD) and autism spectrum disorder (ASD). The current study extends these findings to characterize performance on a fine motor imitation task, probing skills as a predictor of social and communicative functioning, and co-speech gesture use. These research questions were addressed by a secondary analysis of data collected during a previous study characterizing a cohort of individuals who were diagnosed with ASD in early childhood but lost the autism diagnosis (LAD) by the time of adolescence. Fine motor imitation skills were compared between 14 individuals with LAD, 15 individuals with autism spectrum disorder (ASD), and 12 typically developing (TD) individuals. LAD and TD groups had more advanced fine motor imitation skills than the ASD group, and abilities were significantly associated with ASD symptoms and amount of gesture use (though there was a counterintuitive interaction between group and fine motor skill in the LAD and TD groups only, in which lower motor skills predicted more ASD symptoms; this relationship was of a small effect size and is likely driven by the compressed range of fine motor skills in these two groups). Findings suggest that fine motor skills normalize along with social communication skills and restricted and repetitive behaviors and interests in individuals who lose the ASD diagnosis, and that individuals with better fine motor abilities produce more co-speech gesture.

PROTOCOL: Video-based interventions for promoting positive social behaviour in children with autism spectrum disorders: a systematic review and meta-analysis

This is the protocol for a Campbell review. The primary objective for this review is summarising the effectiveness of video-based interventions (VBI) in promoting prosocial behaviours in a population of young people with autism spectrum disorders (ASD). The research questions employed to fulfil this objective include: (1) Do VBI improve prosocial behaviours in children with ASD? (2) Which social skills and interactive behaviours are most successful? (3) Do VBI generally have successful rates of skill generalisation and response maintenance? (4) Do demographic characteristics (age, gender) of participants influence the effectiveness of VBI's?

Processing of pragmatic communication in ASD: a video-based brain imaging study

Social and pragmatic difficulties in autism spectrum disorder (ASD) are widely recognized, although their underlying neural level processing is not well understood. The aim of this study was to examine the activity of the brain network components linked to social and pragmatic understanding in order to reveal whether complex socio-pragmatic events evoke differences in brain activity between the ASD and control groups. Nineteen young adults (mean age 23.6 years) with ASD and 19 controls (mean age 22.7 years) were recruited for the study. The stimulus data consisted of video clips showing complex social events that demanded processing of pragmatic communication. In the analysis, the functional magnetic resonance imaging signal responses of the selected brain network components linked to social and pragmatic information processing were compared. Although the processing of the young adults with ASD was similar to that of the control group during the majority of the social scenes, differences between the groups were found in the activity of the social brain network components when the participants were observing situations with concurrent verbal and non-verbal communication events. The results suggest that the ASD group had challenges in processing concurrent multimodal cues in complex pragmatic communication situations.

N400 amplitude, latency, and variability reflect temporal integration of beat gesture and pitch accent during language processing

This study examines how across-trial (average) and trial-by-trial (variability in) amplitude and latency of the N400 event-related potential (ERP) reflect temporal integration of pitch accent and beat gesture. Thirty native English speakers viewed videos of a talker producing sentences with beat gesture co-occurring with a pitch accented focus word (synchronous), beat gesture co-occurring with the onset of a subsequent non-focused word (asynchronous), or the absence of beat gesture (no beat). Across trials, increased amplitude and earlier latency were observed when beat gesture was temporally asynchronous with pitch accenting than when it was temporally synchronous with pitch accenting or absent. Moreover, temporal asynchrony of beat gesture relative to pitch accent increased trial-by-trial variability of N400 amplitude and latency and influenced the relationship between across-trial and trial-by-trial N400 latency. These results indicate that across-trial and trial-by-trial amplitude and latency of the N400 ERP reflect temporal integration of beat gesture and pitch accent during language comprehension, supporting extension of the integrated systems hypothesis of gesture-speech processing and neural noise theories to focus processing in typical adult populations.

Language and perception: Introduction to the Special Issue “Speakers and Listeners in the Visual World”

Language and perception are two central cognitive systems. Until relatively recently, however, the interaction between them has been examined only partially and not from an over-arching theoretical perspective. Yet it has become clear that linguistic and perceptual interactions are essential to understanding both typical and atypical human behaviour. In this editorial, we examine the link between language and perception across three domains. First, we present a brief review of work investigating the importance of perceptual features, particularly shape bias, when learning names for novel objects—a critical skill acquired during language development. Second, we describe the Visual World Paradigm, an experimental method uniquely suited to investigate the language-perception relationship. Studies using the Visual World Paradigm demonstrate that the relationship between linguistic and perceptual information during processing is both intricate and bi-directional: linguistic cues guide interpretation of visual scenes, while perceptual information shapes interpretation of linguistic input. Finally, we turn to a discussion of co-speech gesture focusing on iconic gestures which depict aspects of the visual world (e.g., motion, shape). The relationship between language and these semantically-meaningful gestures is likewise complex and bi-directional. However, more research is needed to illuminate the exact circumstances under which iconic gestures shape language production and comprehension. In conclusion, although strong evidence exists supporting a critical relationship between linguistic and perceptual systems, the exact levels at which these two systems interact, the time-course of the interaction, and what is driving the interaction, remain largely open questions in need of future research.

Gaze and Motor Traces of Language Processing: Evidence from Autism Spectrum Disorders in Comparison to Typical Controls

We investigated what strategies underlie figurative language processing in two groups of participants distinguished by the presence of a developmental deficit, highly-verbal participants with autism, and control participants without autism in two age ranges each. Individuals with autism spectrum disorder are characterised by impaired social interaction and communication. Even at the high end of the spectrum, where structural language is adequate, difficulties in comprehending non-literal aspects of language are widely attested. The exact causes of these problems are, however, still open to debate. In an interactive sentence-picture matching task participants selected the most suitable image representation of a non-literal figurative expression that matched the target meaning, while their eye-movements and hand movements were being tracked. Our results suggest that individuals with ASD have different processing patterns than typically developing peers when interpreting figurative language, even when they provide the correct answers. Both children with and without autism, and participants with autism display greater uncertainty and competition between alternatives when providing the answer, often reflected in also considering the literal interpretation of the expression against its target figurative meaning. We provide evidence that expression transparency and decomposability play a central role in figurative language processing across all groups.

Reduced multisensory facilitation in adolescents and adults on the Autism Spectrum

Individuals with autism are reported to integrate information from visual and auditory channels in an idiosyncratic way. Multisensory integration (MSI) of simple, non-social stimuli (i.e., flashes and beeps) was evaluated in adolescents and adults with (n = 20) and without autism (n = 19) using a reaction time (RT) paradigm using audio, visual, and audiovisual stimuli. For each participant, the race model analysis compares the RTs on the audiovisual condition to a bound value computed from the unimodal RTs that reflects the effect of redundancy. If the actual audiovisual RTs are significantly faster than this bound, the race model is violated, indicating evidence of MSI. Our results show that the race model violation occurred only for the typically-developing (TD) group. While the TD group shows evidence of MSI, the autism group does not. These results suggest that multisensory integration of simple information, void of social content or complexity, is altered in autism. Individuals with autism may not benefit from the advantage conferred by multisensory stimulation to the same extent as TD individuals. Altered MSI for simple, non-social information may have cascading effects on more complex perceptual processes related to language and behaviour in autism.

Precursors of language development in ASC: A longitudinal single-subject study of gestures in relation to phonetic prosody

We longitudinally investigated the development of deictic and general accompanying gestures in one 5-year-old boy with autism spectrum condition (ASC) and intellectual disability. The investigation was performed during an intervention focusing on phonological segments. The purpose was to see whether gestures developed in conjunction with language. We found that deictic and general accompanying gestures increased during the study, despite not being trained. There was also a positive temporal correlation between deictic gestures and speech. Contrary to the expectations in this case, declarative gestures were frequent and increased, while imperative decreased during the period of intervention. Furthermore, we detected a specific kind of general accompanying non-context-sensitive gestures that may be involved in the retrieval of phonemes and syllables. Speech prosody was also analysed in relation to these gestures.

Communicative misalignment in Autism Spectrum Disorder

Communication deficits are a defining feature of Autism Spectrum Disorder (ASD), manifest during social interactions. Previous studies investigating communicative deficits have largely focused on the perceptual biases, social motivation, cognitive flexibility, or mentalizing abilities of isolated individuals. By embedding autistic individuals in live non-verbal interactions, we characterized a novel cause for their communication deficits. Adults with ASD matched neurotypical individuals in their ability and propensity to generate and modify intelligible behaviors for a communicative partner. However, they struggled to align the meaning of those behaviors with their partner when meaning required referencing their recent communicative history. This communicative misalignment explains why autistic individuals are vulnerable in everyday interactions, which entail fleeting ambiguities, but succeed in social cognition tests involving stereotyped contextual cues. These findings illustrate the cognitive and clinical importance of considering social interaction as a communicative alignment challenge, and how ineffective human communication is without this key interactional ingredient.

Auditory perception is associated with implicit language learning and receptive language ability in autism spectrum disorder

BACKGROUND

Autism spectrum disorder (ASD) is associated with language impairment as well as atypical auditory sensory processing. The current study investigated associations among auditory perception, implicit language learning and receptive language ability in youth with ASD.

METHODS

We measured auditory event related potentials (ERP) during an artificial language statistical learning task in 76 youth with ASD and 27 neurotypical (NT) controls. Participants with ASD had a broad range of cognitive and language abilities.

RESULTS

NT youth showed evidence of implicit learning via attenuated P1 amplitude in the left hemisphere. In contrast, among youth with ASD, implicit learning elicited bilateral attenuation that was increasingly evident with greater receptive language skill.

CONCLUSIONS

Efficient early auditory perception reflects language learning and is a marker of language ability among youth with ASD. Atypical lateralization of word learning is evident in ASD across a broad range of receptive language abilities.

The connectivity signature of co-speech gesture integration: The superior temporal sulcus modulates connectivity between areas related to visual gesture and auditory speech processing

Humans integrate information communicated by speech and gestures. Functional magnetic resonance imaging (fMRI) studies suggest that the posterior superior temporal sulcus (STS) and adjacent gyri are relevant for multisensory integration. However, a connectivity model representing this essential combinatory process is still missing. Here, we used dynamic causal modeling for fMRI to analyze the effective connectivity pattern between middle temporal gyrus (MTG), occipital cortex (OC) and STS associated with auditory verbal, visual gesture-related, and integrative processing, respectively, to unveil the neural mechanisms underlying integration of intrinsically meaningful gestures (e.g., "Thumbs-up gesture") and corresponding speech. 20 participants were presented videos of an actor either performing intrinsic meaningful gestures in the context of German or Russian sentences, or speaking a German sentence without gesture, while performing a content judgment task. The connectivity analyses resulted in a winning model that included bidirectional intrinsic connectivity between all areas. Furthermore, the model included modulations of both connections to the STS (OC→STS; MTG→STS), and non-linear modulatory effects of the STS on bidirectional connections between MTG and OC. Coupling strength in the occipital pathway (OC→STS) correlated with gesture related advantages in task performance, whereas the temporal pathway (MTG→STS) correlated with performance in the speech only condition. Coupling between MTG and OC correlated negatively with subsequent memory performance for sentences of the Gesture-German condition. Our model provides a first step towards a better understanding of speech-gesture integration on network level. It corroborates the importance of the STS during audio-visual integration by showing that this region inhibits direct auditory-visual coupling.

Do Verbal Children with Autism Comprehend Gesture as Readily as Typically Developing Children?

Gesture comprehension remains understudied, particularly in children with autism spectrum disorder (ASD) who have difficulties in gesture production. Using a novel gesture comprehension task, Study 1 examined how 2- to 4-year-old typically-developing (TD) children comprehend types of gestures and gesture-speech combinations, and showed better comprehension of deictic gestures and reinforcing gesture-speech combinations than iconic/conventional gestures and supplementary gesture-speech combinations at each age. Study 2 compared verbal children with ASD to TD children, comparable in receptive language ability, and showed similar patterns of comprehension in each group. Our results suggest that children comprehend deictic gestures and reinforcing gesture-speech combinations better than iconic/conventional gestures and supplementary combinations-a pattern that remains robust across different ages within TD children and children with ASD.

Absence of delay in spontaneous use of gestures in spoken narratives among children with Autism Spectrum Disorders

BACKGROUND

Gestures are spontaneous hand movements produced when speaking. Despite gestures being of communicative significance, little is known about the gestural production in spoken narratives in six- to 12-year-old children with Autism Spectrum Disorders (ASD).

AIMS

The present study examined whether six- to 12-year-old children with ASD have a delay in gestural production in a spoken narrative task, in comparison to their typically-developing (TD) peers.

METHODS AND PROCEDURES

Six- to-12-year-old children with ASD (N=14) and their age- and IQ-matched TD peers (N=12) narrated a story, which could elicit spontaneous speech and gestures. Their speech and gestures were then transcribed and coded.

OUTCOMES AND RESULTS

Both groups of children had comparable expressive language skills. Children with ASD produced a similar number of pointing and marker gestures to TD children and significantly more iconic gestures in their spoken narratives. While children with ASD produced more reinforcing gestures than their TD counterparts, both groups of children produced comparable numbers of disambiguating and supplementary gestures.

CONCLUSIONS

Our findings indicate that children with ASD may be as capable as TD children in gestural production when they engage in spoken narratives, which gives them spontaneity in producing gestures.

Ripe for solution: Delayed development of multisensory processing in autism and its remediation

Difficulty integrating inputs from different sensory sources is commonly reported in individuals with Autism Spectrum Disorder (ASD). Accumulating evidence consistently points to altered patterns of behavioral reactions and neural activity when individuals with ASD observe or act upon information arriving through multiple sensory systems. For example, impairments in the integration of seen and heard speech appear to be particularly acute, with obvious implications for interpersonal communication. Here, we explore the literature on multisensory processing in autism with a focus on developmental trajectories. While much remains to be understood, some consistent observations emerge. Broadly, sensory integration deficits are found in children with an ASD whereas these appear to be much ameliorated, or even fully recovered, in older teenagers and adults on the spectrum. This protracted delay in the development of multisensory processing raises the possibility of applying early intervention strategies focused on multisensory integration, to accelerate resolution of these functions. We also consider how dysfunctional cross-sensory oscillatory neural communication may be one key pathway to impaired multisensory processing in ASD.

The impact of atypical sensory processing on social impairments in autism spectrum disorder

Altered sensory processing has been an important feature of the clinical descriptions of autism spectrum disorder (ASD). There is evidence that sensory dysregulation arises early in the progression of ASD and impacts social functioning. This paper reviews behavioral and neurobiological evidence that describes how sensory deficits across multiple modalities (vision, hearing, touch, olfaction, gustation, and multisensory integration) could impact social functions in ASD. Theoretical models of ASD and their implications for the relationship between sensory and social functioning are discussed. Furthermore, neural differences in anatomy, function, and connectivity of different regions underlying sensory and social processing are also discussed. We conclude that there are multiple mechanisms through which early sensory dysregulation in ASD could cascade into social deficits across development. Future research is needed to clarify these mechanisms, and specific focus should be given to distinguish between deficits in primary sensory processing and altered top-down attentional and cognitive processes.

'Emotiplay': a serious game for learning about emotions in children with autism: results of a cross-cultural evaluation

Children with autism spectrum conditions (ASC) experience difficulties recognizing others' emotions and mental states. It has been shown that serious games (SG) can produce simplified versions of the socio-emotional world. The current study performed a cross-cultural evaluation (in the UK, Israel and Sweden) of Emotiplay's SG, a system aimed to teach emotion recognition (ER) to children with ASC in an entertaining, and intrinsically motivating way. Participants were 6-9 year olds with high functioning ASC who used the SG for 8-12 weeks. Measures included face, voice, body, and integrative ER tasks, as well as parent-reported level of autism symptoms, and adaptive socialization. In the UK, 15 children were tested before and after using the SG. In Israel (n = 38) and Sweden (n = 36), children were randomized into a SG or a waiting list control group. In the UK, results revealed that 8 weeks of SG use significantly improved participants' performance on ER body language and integrative tasks. Parents also reported their children improved their adaptive socialization. In Israel and Sweden, participants using the SG improved significantly more than controls on all ER measures. In addition, parents in the Israeli SG group reported their children showed reduced autism symptoms after using the SG. In conclusion, Emotiplay's SG is an effective and motivating psycho-educational intervention, cross-culturally teaching ER from faces, voices, body language, and their integration in context to children with high functioning ASC. Local evidence was found for more generalized gains to socialization and reduced autism symptoms.

I tawt i taw a puddy tat: Gestures in canary row narrations by high-functioning youth with autism spectrum disorder

This study examined whether individuals with autism spectrum disorder (ASD) produce co-speech gestures similarly to typically developing (TD) peers. Participants were 20 youth ages 10-18 years with high-functioning ASD and 21 TD controls matched on age, gender, verbal IQ, and handedness. Gestures were elicited using a classic narrative-retelling task, in which participants watched a Tweety and Sylvester cartoon and retold the cartoon to a confederate. Analyses compared gesture rate, type, and viewpoint (character, observer, dual) across groups. Communicative utility of gestures was measured via naïve coder ratings of whether a movement was a gesture, and the clarity of a gesture's meaning. The ASD group produced shorter narratives and fewer total gestures than the TD group. Accounting for narrative length, the ASD group produced fewer gestures per clause than the TD group; however, proportions of gesture types (iconic, deictic, beat, metaphoric, emblems) did not differ. Most notably, the ASD group's gestures were rated as less clearly gestures in terms of timing and well formedness, with lower certainty ratings for gesture meaning. Gesture clarity and gesture meaning scores were related to diagnostic measures of gesture competence in ASD. Findings suggest that although fluent children and adolescents with ASD use the same type of gestures as controls, their gestures are more difficult to understand, which has significant implications for their communicative abilities more broadly. Autism Res 2017, 10: 1353-1363. © 2017 International Society for Autism Research, Wiley Periodicals, Inc.

Altered Gesture and Speech Production in ASD Detract from In-Person Communicative Quality

This study disentangled the influences of language and social processing on communication in autism spectrum disorder (ASD) by examining whether gesture and speech production differs as a function of social context. The results indicate that, unlike other adolescents, adolescents with ASD did not increase their coherency and engagement in the presence of a visible listener, and that greater coherency and engagement were related to lesser social and communicative impairments. Additionally, the results indicated that adolescents with ASD produced sparser speech and fewer gestures conveying supplementary information, and that both of these effects increased in the presence of a visible listener. Together, these findings suggest that interpersonal communication deficits in ASD are driven more strongly by social processing than language processing.

Dyspraxia in ASD: Impaired coordination of movement elements

Children with autism spectrum disorders (ASD) have long been known to have deficits in the performance of praxis gestures; these motor deficits also correlate with social and communicative deficits. To date, the precise nature of the errors involved in praxis has not been clearly mapped out. Based on observations of individuals with ASD performing gestures, we hypothesized that the simultaneous execution of multiple movement elements is especially impaired in affected children. We examined 25 school-aged participants with ASD and 25 age-matched controls performing seven simultaneous gestures that required the concurrent performance of movement elements and nine serial gestures, in which all elements were performed serially. There was indeed a group × gesture-type interaction (P < 0.001). Whereas both groups had greater difficulty performing simultaneous than serial gestures, children with ASD had a 2.6-times greater performance decrement with simultaneous (vs. serial) gestures than controls. These results point to a potential deficit in the simultaneous processing of multiple inputs and outputs in ASD. Such deficits could relate to models of social interaction that highlight the parallel-processing nature of social communication. Autism Res 2016,. © 2016 International Society for Autism Research, Wiley Periodicals, Inc. Autism Res 2017, 10: 648-652. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.

The use of hand gestures to communicate about nonpresent objects in mind among children with autism spectrum disorder

PURPOSE

The current study examined whether children with autism spectrum disorder (ASD), in comparison with typically developing children, perceive and produce gestures to identify nonpresent objects (i.e., referent-identifying gestures), which is crucial for communicating ideas in a discourse.

METHOD

An experimenter described the uses of daily-life objects to 6- to 12-year-old children both orally and with gestures. The children were then asked to describe how they performed daily activities using those objects.

RESULTS

All children gestured. A gesture identified a nonpresent referent if it was produced in the same location that had previously been established by the experimenter. Children with ASD gestured at the specific locations less often than typically developing children. Verbal and spatial memory were positively correlated with the ability to produce referent-identifying gestures for all children. However, the positive correlation between Raven's Children Progressive Matrices score and the production of referent-identifying gestures was found only in children with ASD.

CONCLUSIONS

Children with ASD might be less able to perceive and produce referent-identifying gestures and may rely more heavily on visual-spatial skills in producing referent-identifying gestures. The results have clinical implications for designing an intervention program to enhance the ability of children with ASD to communicate about nonpresent objects with gestures.

Responses of prefrontal multisensory neurons to mismatching faces and vocalizations

Social communication relies on the integration of auditory and visual information, which are present in faces and vocalizations. Evidence suggests that the integration of information from multiple sources enhances perception compared with the processing of a unimodal stimulus. Our previous studies demonstrated that single neurons in the ventrolateral prefrontal cortex (VLPFC) of the rhesus monkey (Macaca mulatta) respond to and integrate conspecific vocalizations and their accompanying facial gestures. We were therefore interested in how VLPFC neurons respond differentially to matching (congruent) and mismatching (incongruent) faces and vocalizations. We recorded VLPFC neurons during the presentation of movies with congruent or incongruent species-specific facial gestures and vocalizations as well as their unimodal components. Recordings showed that while many VLPFC units are multisensory and respond to faces, vocalizations, or their combination, a subset of neurons showed a significant change in neuronal activity in response to incongruent versus congruent vocalization movies. Among these neurons, we typically observed incongruent suppression during the early stimulus period and incongruent enhancement during the late stimulus period. Incongruent-responsive VLPFC neurons were both bimodal and nonlinear multisensory, fostering their ability to respond to changes in either modality of a face-vocalization stimulus. These results demonstrate that ventral prefrontal neurons respond to changes in either modality of an audiovisual stimulus, which is important in identity processing and for the integration of multisensory communication information.

Influences of semantic and prosodic cues on word repetition and categorization in autism

PURPOSE

To investigate sensitivity to prosodic and semantic cues to emotion in individuals with high-functioning autism (HFA).

METHOD

Emotional prosody and semantics were independently manipulated to assess the relative influence of prosody versus semantics on speech processing. A sample of 10-year-old typically developing children (n = 10) and children with HFA (n = 10) were asked to repeat words that were either emotionally congruent or incongruent in form and content (Experiment 1A). In a second task (Experiment 1B), the same participants were asked to classify stimuli on the basis of emotional prosody. A final experiment (Experiment 2) focused on sensitivity to congruence in a non-emotional source of variation: talker gender.

RESULTS

The results revealed a selective impairment in spontaneous integration of prosodic and semantic cues to emotion in HFA; however, the same participants were able to categorize emotions on the basis of prosody under reduced task demands. Individuals with HFA were highly sensitive to another surface characteristic in speech: talker gender.

CONCLUSIONS

The study reveals impairment in the spontaneous integration of prosodic and semantic cues to emotion in HFA; however, insensitivity to surface detail, such as prosody, in HFA appears to be highly task dependent and selective to the domain of emotion.

Sources of variation in developmental language disorders: evidence from eye-tracking studies of sentence production

Skilled sentence production involves distinct stages of message conceptualization (deciding what to talk about) and message formulation (deciding how to talk about it). Eye-movement paradigms provide a mechanism for observing how speakers accomplish these aspects of production in real time. These methods have recently been applied to children with autism spectrum disorder (ASD) and specific language impairment (LI) in an effort to reveal qualitative differences between groups in sentence production processes. Findings support a multiple-deficit account in which language production is influenced not only by lexical and syntactic constraints, but also by variation in attention control, inhibition and social competence. Thus, children with ASD are especially vulnerable to atypical patterns of visual inspection and verbal utterance. The potential to influence attentional focus and prime appropriate language structures are considered as a mechanism for facilitating language adaptation and learning.

The development of co-speech gesture in the communication of children with autism spectrum disorders

Co-speech gestures have a close semantic relationship to speech in adult conversation. In typically developing children co-speech gestures which give additional information to speech facilitate the emergence of multi-word speech. A difficulty with integrating audio-visual information is known to exist for individuals with Autism Spectrum Disorder (ASD), which may affect development of the speech-gesture system. A longitudinal observational study was conducted with four children with ASD, aged 2;4 to 3;5 years. Participants were video-recorded for 20 min every 2 weeks during their attendance on an intervention programme. Recording continued for up to 8 months, thus affording a rich analysis of gestural practices from pre-verbal to multi-word speech across the group. All participants combined gesture with either speech or vocalisations. Co-speech gestures providing additional information to speech were observed to be either absent or rare. Findings suggest that children with ASD do not make use of the facilitating communicative effects of gesture in the same way as typically developing children.

Altered integration of speech and gesture in children with autism spectrum disorders

The presence of gesture during speech has been shown to impact perception, comprehension, learning, and memory in normal adults and typically developing children. In neurotypical individuals, the impact of viewing co-speech gestures representing an object and/or action (i.e., iconic gesture) or speech rhythm (i.e., beat gesture) has also been observed at the neural level. Yet, despite growing evidence of delayed gesture development in children with autism spectrum disorders (ASD), few studies have examined how the brain processes multimodal communicative cues occurring during everyday communication in individuals with ASD. Here, we used a previously validated functional magnetic resonance imaging (fMRI) paradigm to examine the neural processing of co-speech beat gesture in children with ASD and matched controls. Consistent with prior observations in adults, typically developing children showed increased responses in right superior temporal gyrus and sulcus while listening to speech accompanied by beat gesture. Children with ASD, however, exhibited no significant modulatory effects in secondary auditory cortices for the presence of co-speech beat gesture. Rather, relative to their typically developing counterparts, children with ASD showed significantly greater activity in visual cortex while listening to speech accompanied by beat gesture. Importantly, the severity of their socio-communicative impairments correlated with activity in this region, such that the more impaired children demonstrated the greatest activity in visual areas while viewing co-speech beat gesture. These findings suggest that although the typically developing brain recognizes beat gesture as communicative and successfully integrates it with co-occurring speech, information from multiple sensory modalities is not effectively integrated during social communication in the autistic brain.

A combination of thematic and similarity-based semantic processes confers resistance to deficit following left hemisphere stroke

Semantic knowledge may be organized in terms of similarity relations based on shared features and/or complementary relations based on co-occurrence in events. Thus, relationships between manipulable objects such as tools may be defined by their functional properties (what the objects are used for) or thematic properties (e.g., what the objects are used with or on). A recent study from our laboratory used eye-tracking to examine incidental activation of semantic relations in a word–picture matching task and found relatively early activation of thematic relations (e.g., broom–dustpan), later activation of general functional relations (e.g., broom–sponge), and an intermediate pattern for specific functional relations (e.g., broom–vacuum cleaner). Combined with other recent studies, these results suggest that there are distinct semantic systems for thematic and similarity-based knowledge and that the “specific function” condition drew on both systems. This predicts that left hemisphere stroke that damages either system (but not both) may spare specific function processing. The present experiment tested these hypotheses using the same experimental paradigm with participants with left hemisphere lesions (N = 17). The results revealed that, compared to neurologically intact controls (N = 12), stroke participants showed later activation of thematic and general function relations, but activation of specific function relations was spared and was significantly earlier for stroke participants than controls. Across the stroke participants, activation of thematic and general function relations was negatively correlated, further suggesting that damage tended to affect either one semantic system or the other. These results support the distinction between similarity-based and complementarity-based semantic relations and suggest that relations that draw on both systems are relatively more robust to damage.

Conversational gestures in autism spectrum disorders: asynchrony but not decreased frequency

Conversational or "co-speech" gestures play an important role in communication, facilitating turntaking, providing visuospatial information, clarifying subtleties of emphasis, and other pragmatic cues. Consistent with other pragmatic language deficits, individuals with autism spectrum disorders (ASD) are said to produce fewer conversational gestures, as specified in many diagnostic measures. Surprisingly, while research shows fewer deictic gestures in young children with ASD, there is a little empirical evidence addressing other forms of gesture. The discrepancy between clinical and empirical observations may reflect impairments unrelated to frequency, such as gesture quality or integration with speech. Adolescents with high-functioning ASD (n = 15), matched on age, gender, and IQ to 15 typically developing (TD) adolescents, completed a narrative task to assess the spontaneous production of speech and gesture. Naïve observers rated the stories for communicative quality. Overall, the ASD group's stories were rated as less clear and engaging. Although utterance and gesture rates were comparable, the ASD group's gestures were less closely synchronized with the co-occurring speech, relative to control participants. This gesture-speech synchrony specifically impacted communicative quality across participants. Furthermore, while story ratings were associated with gesture count in TD adolescents, no such relationship was observed in adolescents with ASD, suggesting that gestures do not amplify communication in this population. Quality ratings were, however, correlated with ASD symptom severity scores, such that participants with fewer ASD symptoms were rated as telling higher quality stories. Implications of these findings are discussed in terms of communication and neuropsychological functioning in ASD.

Sound before meaning: word learning in autistic disorders

Successful word learning depends on the integration of phonological and semantic information with social cues provided by interlocutors. How then, do children with autism spectrum disorders (ASDs) learn new words when social impairments pervade? We recorded the eye-movements of verbally-able children with ASD and their typical peers while completing a word learning task in a social context. We assessed learning of semantic and phonological features immediately after learning and again four weeks later. Eye-movement data revealed that both groups could follow social cues, but that typically developing children were more sensitive to the social informativeness of gaze cues. In contrast, children with ASD were more successful than peers at mapping phonological forms to novel referents; however, this advantage was not maintained over time. Typical children showed clear consolidation of learning both semantic and phonological information, children with ASD did not. These results provide unique evidence of qualitative differences in word learning and consolidation and elucidate the different mechanisms underlying the unusual nature of autistic language.

Embodied communication: speakers' gestures affect listeners' actions

We explored how speakers and listeners use hand gestures as a source of perceptual-motor information during naturalistic communication. After solving the Tower of Hanoi task either with real objects or on a computer, speakers explained the task to listeners. Speakers' hand gestures, but not their speech, reflected properties of the particular objects and the actions that they had previously used to solve the task. Speakers who solved the problem with real objects used more grasping handshapes and produced more curved trajectories during the explanation. Listeners who observed explanations from speakers who had previously solved the problem with real objects subsequently treated computer objects more like real objects; their mouse trajectories revealed that they lifted the objects in conjunction with moving them sideways, and this behavior was related to the particular gestures that were observed. These findings demonstrate that hand gestures are a reliable source of perceptual-motor information during human communication.