The relationship between co-speech gesture production and macrolinguistic discourse abilities in people with focal brain injury

Language and gesture neural correlates: A meta-analysis of functional magnetic resonance imaging studies

BACKGROUND

Humans often use co-speech gestures to promote effective communication. Attention has been paid to the cortical areas engaged in the processing of co-speech gestures.

AIMS

To investigate the neural network underpinned in the processing of co-speech gestures and to observe whether there is a relationship between areas involved in language and gesture processing.

METHODS & PROCEDURES

We planned to include studies with neurotypical and/or stroke participants who underwent a bimodal task (i.e., processing of co-speech gestures with relative speech) and a unimodal task (i.e., speech or gesture alone) during a functional magnetic resonance imaging (fMRI) session. After a database search, abstract and full-text screening were conducted. Qualitative and quantitative data were extracted, and a meta-analysis was performed with the software GingerALE 3.0.2, performing contrast analyses of uni- and bimodal tasks.

MAIN CONTRIBUTION

The database search produced 1024 records. After the screening process, 27 studies were included in the review. Data from 15 studies were quantitatively analysed through meta-analysis. Meta-analysis found three clusters with a significant activation of the left middle frontal gyrus and inferior frontal gyrus, and bilateral middle occipital gyrus and inferior temporal gyrus.

CONCLUSIONS

There is a close link at the neural level for the semantic processing of auditory and visual information during communication. These findings encourage the integration of the use of co-speech gestures during aphasia treatment as a strategy to foster the possibility to communicate effectively for people with aphasia.

WHAT THIS PAPER ADDS

What is already known on this subject Gestures are an integral part of human communication, and they may have a relationship at neural level with speech processing. What this paper adds to the existing knowledge During processing of bi- and unimodal communication, areas related to semantic processing and multimodal processing are activated, suggesting that there is a close link between co-speech gestures and spoken language at a neural level. What are the potential or actual clinical implications of this work? Knowledge of the functions related to gesture and speech processing neural networks will allow for the adoption of model-based neurorehabilitation programs to foster recovery from aphasia by strengthening the specific functions of these brain networks.

Looking at gesture: The reciprocal influence between gesture and conversation

INTRODUCTION

There is limited research in group communication treatment for people with aphasia but existing studies report benefits of gesture to support conversation. Gesture supports conversation through recipient design features and reducing linguistic demands of lexical retrieval and formulation. Additionally, gesture serves an affiliative function. However, the relationship between gesture use and gestural capacity has not been widely examined. As part of a larger study on group cohesiveness and conversation, this investigation examined the patterns of co-speech gesture within authentic conversations among persons with aphasia to discern the functions of gesture use for the participants, changes in the use of gesture over time, and the relationship between gesture use and gesture ability.

METHODS

Conversation Analysis (CA) was applied in an embedded case-study design. Three participants received an academic semester of group and individual conversation-based treatment according to Facilitating Authentic Conversation (Damico et al., 2015). Four conversations from the treatment were selected and transcribed for multi-modality communication with CA conventions applied, and then cyclically analysed for patterns of gesture.

RESULTS

Participants demonstrated gesture that served social and linguistic functions: ratifying clinicians' proxy turns, turn-allocation, turn repair, relaying novel visual information, emphasizing content, demonstrating affiliation with the prior speaker, demonstrating their assessment others' talk, and demonstrating humor. All three participants showed an increased rate of gesture per turn and increasingly used gesture to repair conversation breakdown. Increased gesture use over the course of the semester coincided with increased scores for pantomime on the Porch Index of Communicative Ability (Porch, 1981, PICA).

CONCLUSION

Individuals with aphasia demonstrated increased use of gesture for varied purposes and improved gestural processing following a semester of conversation-based treatment. This is significant because gesture is an effective support for the repair of conversation breakdown typical of persons with aphasia.

Production of co-speech gestures in the right hemisphere: Evidence from individuals with complete or anterior callosotomy

INTRODUCTION

A right-hand preference for co-speech gestures in right-handed neurotypical individuals as well as the co-occurrence of speech and gesture has induced neuropsychological research to primarily target the left hemisphere when investigating co-speech gesture production. However, the substantial number of spontaneous left-hand gestures in right-handed individuals has, thus far, been unexplained. Recent studies in individuals with complete callosotomy and exclusive left hemisphere speech production show a reliable left-hand preference for co-speech gestures, indicating a right hemispheric generation. However, the findings raise the issue if the separate right hemisphere is able to also generate representational gestures. The present study challenges the proposition of a specific right hemispheric contribution to gesture production by differentiating gesture types including representational ones in individuals with complete callosotomy and by including individuals with anterior callosotomy in whom neural reorganization is less extensive.

METHODS

Three right-handed individuals with complete commissurotomy (A.A., N.G., G.C.) and three right-handed individuals with anterior callosotomy (C.E., S.R., L. D), all with left hemisphere language dominance, and a matched right-handed neurotypical control group (n = 10) were examined in an experimental setting, including re-narration of a nonverbal animated cartoon and responding to intelligence questions. The participants' video-taped hand movement behavior was analyzed by two independent certified raters with the NEUROGES-ELAN system for nonverbal behavior and gesture. Unimanual right-hand and left-hand gestures were classified into eight gesture types.

RESULTS

The individuals with complete and anterior callosotomy performed unimanual co-speech gestures with the left as well as the right hand, with no significant preference of one hand for gestures overall. Concerning the specific gesture types, the group with complete callosotomy showed a significant right-hand preference for pantomime gestures, which also applied to the callosotomy total group. The group with anterior callosotomy displayed a significant left-hand preference for form presentation gestures. As a trend, the callosotomy total group differed from the neurotypical group as they performed more left-hand egocentric deictic and left-hand form presentation gestures.

DISCUSSION

The present study replicates the finding of a substantial left-hand use for unimanual co-speech gestures in individuals with complete callosotomy. The proposition of a right hemispheric contribution to gesture production independent from left hemispheric language production is corroborated by the finding that individuals with anterior callosotomy show a similar pattern of hand use for gestures. Representational gestures were displayed with either hand, suggesting that in particular right hemispheric spatial cognition can be directly expressed in gesture. The significant right-hand preference for pantomime gesture was outstanding and compatible with the established left hemispheric specialization for tool use praxis. The findings shed a new light on the left-hand gestures in neurotypical individuals, suggesting that these can be generated in the right hemisphere.

Pragmatic deficits in patients with schizophrenia and right hemisphere damage: A pilot study

BACKGROUND

While pragmatic deficits are well documented in patients with schizophrenia (SCZ) and right hemisphere damage (RHD), there is a paucity of research comparing the pragmatic deficits of these two groups. Do they experience similar cognitive dysfunction or is there a dissociation between the two patient groups?

AIMS

To investigate the nature of pragmatic deficits in these two groups and to gain an understanding of the underlying cognitive mechanisms that might be associated with these deficits to further future investigations.

METHODS & PROCEDURES

A total of 60 participants (15 patients with SCZ; 15 with RHD; 30 (15 + 15) healthy controls (HC) were administered the Bengali Audio-Visual Test-Battery for Assessment of Pragmatic Skills.

OUTCOMES & RESULTS

Both SCZ and RHD patients were found to have significant pragmatic deficits compared with their matched controls. SCZ patients were found to score significantly better than the RHD group in six out of the 10 pragmatic skills when controlled for age and education. Discriminant function analysis was performed and 86.7% of the cases (HC = 100%, SCZ = 73.3% and RHD = 86.7%) were correctly reclassified into their original categories using the test scores.

CONCLUSIONS & IMPLICATIONS

The study suggests that there is heterogeneity in the nature of the pragmatic breakdown within and across patient groups. Therefore, individualized restorative measures targeting the disrupted cognitive mechanism(s) might help elevate pragmatic competence and enhance the social functioning of patients with pragmatic deficits.

WHAT THIS PAPER ADDS

What is already known on the subject Pragmatic deficits are common in adults with cognitive impairments of different etiologies. However, few studies have explored pragmatic deficits across clinical populations. Consequently, very little is known about the nature of pragmatic deficits in patients with schizophrenia and right hemisphere damage. What this paper adds to existing knowledge This work offers preliminary data on pragmatic difficulties in patients with schizophrenia and right hemisphere damage. This study overrides the boundaries of traditional classifications and evaluates pragmatic difficulties in these two clinical populations with reference to the underlying cognitive mechanisms, which might be disrupted. What are the potential or actual clinical implications of this work? The study adds a transdiagnostic perspective suggesting that there might be heterogeneity in pragmatic deficits, both within and across patient groups, and stresses the need for individualized therapy.

Task and Material Properties Interactively Affect Softness Explorations Along Different Dimensions

Haptic research has frequently equated softness with the compliance of elastic objects. However, in a recent study we have suggested that compliance is not the only perceived material dimension underlying what is commonly called softness [1 ]. Here, we investigate, whether the different perceptual dimensions of softness affect how materials are haptically explored. Specifically, we tested whether also the task, i.e., the attribute that a material is being judged on, might affect how a material is explored. To this end we selected 15 adjectives and 19 materials that each associate with different softness dimensions for the study. In the experiment, while participants freely explored and rated the materials, we recorded their hand movements. These movements were subsequently categorized into distinct exploratory procedures (EPs) and analyzed in a multivariate analysis of variance (MANOVA). The results of this analysis suggest that the pattern of EPs depended not only on the material's softness dimension and the task (i.e., what attributes were rated), but also on an interaction between the two factors. Taken together, our findings support the notion of multiple perceptual dimensions of softness and suggest that participants actively adapt their EPs in a nuanced way when judging a particular softness dimensions for a given material.

Nonverbal Semantics Test (NVST)-A Novel Diagnostic Tool to Assess Semantic Processing Deficits: Application to Persons with Aphasia after Cerebrovascular Accident

Assessment of semantic processing capacities often relies on verbal tasks which are, however, sensitive to impairments at several language processing levels. Especially for persons with aphasia there is a strong need for a tool that measures semantic processing skills independent of verbal abilities. Furthermore, in order to assess a patient’s potential for using alternative means of communication in cases of severe aphasia, semantic processing should be assessed in different nonverbal conditions. The Nonverbal Semantics Test (NVST) is a tool that captures semantic processing capacities through three tasks—Semantic Sorting, Drawing, and Pantomime. The main aim of the current study was to investigate the relationship between the NVST and measures of standard neurolinguistic assessment. Fifty-one persons with aphasia caused by left hemisphere brain damage were administered the NVST as well as the Aachen Aphasia Test (AAT). A principal component analysis (PCA) was conducted across all AAT and NVST subtests. The analysis resulted in a two-factor model that captured 69% of the variance of the original data, with all linguistic tasks loading high on one factor and the NVST subtests loading high on the other. These findings suggest that nonverbal tasks assessing semantic processing capacities should be administered alongside standard neurolinguistic aphasia tests.

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Speakers use spontaneous hand gestures as they speak and think. These gestures serve many functions for speakers who produce them as well as for listeners who observe them. To date, studies in the gesture literature mostly focused on group-comparisons or the external sources of variation to examine when people use, process, and benefit from using and observing gestures. However, there are also internal sources of variation in gesture use and processing. People differ in how frequently they use gestures, how salient their gestures are, for what purposes they produce gestures, and how much they benefit from using and seeing gestures during comprehension and learning depending on their cognitive dispositions. This review addresses how individual differences in different cognitive skills relate to how people employ gestures in production and comprehension across different ages (from infancy through adulthood to healthy aging) from a functionalist perspective. We conclude that speakers and listeners can use gestures as a compensation tool during communication and thinking that interacts with individuals' cognitive dispositions.

The Role of Gesture in Communication and Cognition: Implications for Understanding and Treating Neurogenic Communication Disorders

When people talk, they gesture. Gesture is a fundamental component of language that contributes meaningful and unique information to a spoken message and reflects the speaker's underlying knowledge and experiences. Theoretical perspectives of speech and gesture propose that they share a common conceptual origin and have a tightly integrated relationship, overlapping in time, meaning, and function to enrich the communicative context. We review a robust literature from the field of psychology documenting the benefits of gesture for communication for both speakers and listeners, as well as its important cognitive functions for organizing spoken language, and facilitating problem-solving, learning, and memory. Despite this evidence, gesture has been relatively understudied in populations with neurogenic communication disorders. While few studies have examined the rehabilitative potential of gesture in these populations, others have ignored gesture entirely or even discouraged its use. We review the literature characterizing gesture production and its role in intervention for people with aphasia, as well as describe the much sparser literature on gesture in cognitive communication disorders including right hemisphere damage, traumatic brain injury, and Alzheimer's disease. The neuroanatomical and behavioral profiles of these patient populations provide a unique opportunity to test theories of the relationship of speech and gesture and advance our understanding of their neural correlates. This review highlights several gaps in the field of communication disorders which may serve as a bridge for applying the psychological literature of gesture to the study of language disorders. Such future work would benefit from considering theoretical perspectives of gesture and using more rigorous and quantitative empirical methods in its approaches. We discuss implications for leveraging gesture to explore its untapped potential in understanding and rehabilitating neurogenic communication disorders.

Differential roles of gestures on spatial language in neurotypical elderly adults and individuals with focal brain injury

Gestures might serve communicative functions by supplementing spoken expressions or restorative functions by facilitating speech production. Also, speakers with speech deficits use gestures to compensate for their speech impairments. In this study, we examined gesture use in speakers with and without speech impairments and how spoken spatial expressions changed when gestures were restrained. Six patients with speech problems and with left frontal and/or temporal lesions and 20 neurotypical controls described motion events in 3 different conditions (spontaneous gesture, only speech, and only gesture). In addition to the group analyses, we ran case analyses. Results showed that patients used more gestures compared to controls. Gestures served both communicative and restorative functions for patients whereas controls only used gestures for communicative purposes. Case analyses revealed that there were differential patterns among patients. Overall, gesture production is multifunctional and gestures serve different functions for different populations as well as within a population.