The limited role of hippocampal declarative memory in transient semantic activation during online language processing

Reduced on-line speech gesture integration during multimodal language processing in adults with moderate-severe traumatic brain injury: Evidence from eye-tracking

BACKGROUND

Language is multimodal and situated in rich visual contexts. Language is also incremental, unfolding moment-to-moment in real time, yet few studies have examined how spoken language interacts with gesture and visual context during multimodal language processing. Gesture is a rich communication cue that is integrally related to speech and often depicts concrete referents from the visual world. Using eye-tracking in an adapted visual world paradigm, we examined how participants with and without moderate-severe traumatic brain injury (TBI) use gesture to resolve temporary referential ambiguity.

METHODS

Participants viewed a screen with four objects and one video. The speaker in the video produced sentences (e.g., "The girl will eat the very good sandwich"), paired with either a meaningful gesture (e.g., sandwich-holding gesture) or a meaningless grooming movement (e.g., arm scratch) at the verb "will eat." We measured participants' gaze to the target object (e.g., sandwich), a semantic competitor (e.g., apple), and two unrelated distractors (e.g., piano, guitar) during the critical window between movement onset in the gesture modality and onset of the spoken referent in speech.

RESULTS

Both participants with and without TBI were more likely to fixate the target when the speaker produced a gesture compared to a grooming movement; however, relative to non-injured participants, the effect was significantly attenuated in the TBI group.

DISCUSSION

We demonstrated evidence of reduced speech-gesture integration in participants with TBI relative to non-injured peers. This study advances our understanding of the communicative abilities of adults with TBI and could lead to a more mechanistic account of the communication difficulties adults with TBI experience in rich communication contexts that require the processing and integration of multiple co-occurring cues. This work has the potential to increase the ecological validity of language assessment and provide insights into the cognitive and neural mechanisms that support multimodal language processing.

Spoken language comprehension in children and adults with Angelman Syndrome

INTRODUCTION

Objective evaluation of receptive communication abilities in nonspeaking individuals using standardized behavioral measures can be complicated by co-occurring intellectual disabilities and motor difficulties. Eye tracking during listening may offer an informative complementary approach to directly evaluate receptive language skills.

METHOD

This study examined feasibility of eye gaze measures as an index of spoken language comprehension in nonspeaking children and adults with Angelman syndrome (AS; n = 23) using a looking-while-listening procedure. Typically developing children (n = 34) provided a reference data set. Primary caregivers of participants with AS completed standardized informant reports (MacArthur-Bates Communicative Development Inventory: Words and Gestures; Vineland Adaptive Behavior Scales-3; Aberrant Behavior Checklist-2) to characterize communicative skills and general adaptive functioning.

RESULTS

Gaze data in participants with AS, particularly in the individuals reported by caregivers to have larger receptive vocabularies and stronger adaptive communicative functioning, demonstrated the expected pattern of comprehension reflected by the increased probability of looks to the target images after vs. before they were named in a spoken sentence. However, processing speed (gaze reaction time) was significantly slower in participants with AS than in the typically developing group.

CONCLUSIONS

Gaze-based paradigms could be an informative measure of receptive communication processes in participants who are unable to complete traditional standardized behavioral assessments.