Naming and gesturing spatial relations: evidence from focal brain-injured individuals

Apragmatism: The renewal of a label for communication disorders associated with right hemisphere brain damage

BACKGROUND

Right hemisphere communication disorders are neither consistently labelled nor adequately defined. Labels associated with right hemisphere brain damage (RHD) are broad and fail to capture the essence of communication challenges needed for stroke-related service provisions. Determination of rehabilitation needs and best-practice guidelines for the education, management and functional improvement of communication disorders after RHD are all predicated on an apt diagnostic label and disorder characteristics.

AIMS

In this paper apragmatism is proposed as a potential communication-specific diagnostic label for the impairments in communication that occur after RHD. In particular, the researchers aimed: (1) to establish an operational definition of apragmatism; and (2) to describe the linguistic, paralinguistic and extralinguistic communication deficits under the umbrella term apragmatism.

METHODS & PROCEDURES

An international collaborative of researchers with expertise in RHD followed a multilevel approach to consider the utility of apragmatism as a diagnostic label. Adopting the relational approach to concept mapping, the researchers engaged in a series of group meetings to complete four levels of mapping: (1) identify and review, (2) define, (3) expert discussion and (4) label determination.

MAIN CONTRIBUTION

Apragmatism was established as a suitable diagnostic label for the impairments in communication associated with RHD. The paper offers an operational definition and description of the linguistic, paralinguistic and extralinguistic features of apragmatism through evidence summaries and examples from people with RHD retrieved from the RHDBank.

CONCLUSIONS & IMPLICATIONS

The adoption of the term apragmatism offers an opportunity to capture the hallmark of RHD communication deficits. The use of the term is recommended when referencing the pragmatic language impairments in this population. Apragmatism, which may co-occur with or be exacerbated by cognitive impairments, can interfere with the ability to interpret and convey intended meaning and impact the lives of right hemisphere stroke survivors and their families.

WHAT THIS PAPER ADDS

What is already known on the subject RHD results in a heterogeneous group of deficits that range in cognitive-communicative complexity. Many of the deficits are subsumed under pragmatics. For example, adults with RHD may demonstrate tangential or verbose communication, insensitivity to others' needs and feelings, prosodic changes, minimal gesture use and facial expression, and more. While descriptions of pragmatic impairments pervade the literature, there is no consistently used diagnostic label. The clinical consequences of this absence include difficulty with inter- and intra-disciplinary communication about these patients, difficulty consolidating findings across research studies, and challenges in communicating about these pragmatic changes with patients, families and other stakeholders. What this paper adds to existing knowledge The term apragmatism is proposed as a diagnostic label to consistently describe pragmatic communication changes after RHD. Apragmatism is characterized using three components of pragmatics: linguistic, paralinguistic and extralinguistic. Descriptions and examples of these three components are provided with supplemental transcripts retrieved from the RHDBank. What are the potential or actual clinical implications of this work? Adoption of the term apragmatism by speech and language therapists and other medical and rehabilitation professionals has the potential to provide consistency in describing the abilities and challenges experienced by people following a right hemisphere stroke. Such improvements may help drive the development of evidence-based assessments and treatments for this population.

Aging, Gesture Production, and Disfluency in Speech: A Comparison of Younger and Older Adults

Age-related changes are observed in the speech and gestures of neurotypical individuals. Older adults are more disfluent in speech and use fewer representational gestures (e.g., holding two hands close to each other to mean small), compared to younger adults. Using gestures, especially representational gestures, is common in difficult tasks to aid the conceptualization process and to facilitate lexical access. This study investigates how aging can affect gesture production and the co-occurrence between gesture and speech disfluency. We elicited speech and gesture samples from younger and older adults (N = 60) by using a painting description task that provided concrete and abstract contexts. Results indicated that albeit the two age groups revealed comparable overall speech disfluency and gesture rates, they differed in terms of how their disfluencies and gestures were distributed across specific categories. Moreover, the proportion of speech disfluencies that occur with a gesture was significantly higher for younger than older adults. However, the two age groups were comparable in terms of the proportion of gestures that were accompanied by a speech disfluency. These findings suggest that younger adults' language production system might be better at benefiting from other modalities, that is, gesture, to resolve temporary problems in speech planning. However, from a gesture perspective, it might be difficult to differentiate between gestures' self-oriented and communicative functions and understand their role in speech facilitation. Focusing on specific cases where speech disfluency and gestures co-occur and considering individual differences might bring insight into multimodal communication.

Suggestions for Improving the Investigation of Gesture in Aphasia

Purpose When we speak, we gesture, and indeed, persons with aphasia gesture more frequently. The reason(s) for this is still being investigated, spurring an increase in the number of studies of gesture in persons with aphasia. As the number of studies increases, so too does the need for a shared set of best practices for gesture research in aphasia. After briefly reviewing the importance and use of gesture in persons with aphasia, this viewpoint puts forth methodological and design considerations when evaluating gesture in persons with aphasia. Method & Results We explore several different design and methodological considerations for gesture research specific to persons with aphasia, such as video angle specifications, data collection techniques, and analysis considerations. The goal of these suggestions is to develop transparent and reproducible methods for evaluating gesture in aphasia to build a solid foundation for continued work in this area. Conclusions We have proposed that it is critical to evaluate multimodal communication in a methodologically robust way to facilitate increased knowledge about the relationship of gesture to spoken language, cognition, and to other aspects of living with aphasia and recovery from aphasia. We conclude by postulating future directions for gesture research in aphasia.

The Relation Between Cognitive Abilities and the Distribution of Semantic Features Across Speech and Gesture in 4-year-olds

When young children learn to use language, they start to use their hands in co-verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) analysis to characterize combinations of speech and gesture meaning and related them to the child's visual-spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON-R 2 1 2 - 7 test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities.

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Speakers use spontaneous hand gestures as they speak and think. These gestures serve many functions for speakers who produce them as well as for listeners who observe them. To date, studies in the gesture literature mostly focused on group-comparisons or the external sources of variation to examine when people use, process, and benefit from using and observing gestures. However, there are also internal sources of variation in gesture use and processing. People differ in how frequently they use gestures, how salient their gestures are, for what purposes they produce gestures, and how much they benefit from using and seeing gestures during comprehension and learning depending on their cognitive dispositions. This review addresses how individual differences in different cognitive skills relate to how people employ gestures in production and comprehension across different ages (from infancy through adulthood to healthy aging) from a functionalist perspective. We conclude that speakers and listeners can use gestures as a compensation tool during communication and thinking that interacts with individuals' cognitive dispositions.

The Production of Gesture and Speech by People With Aphasia: Influence of Communicative Constraints

Purpose People with aphasia (PWA) use different kinds of gesture spontaneously when they communicate. Although there is evidence that the nature of the communicative task influences the linguistic performance of PWA, so far little is known about the influence of the communicative task on the production of gestures by PWA. We aimed to investigate the influence of varying communicative constraints on the production of gesture and spoken expression by PWA in comparison to persons without language impairment. Method Twenty-six PWA with varying aphasia severities and 26 control participants (CP) without language impairment participated in the study. Spoken expression and gesture production were investigated in 2 different tasks: (a) spontaneous conversation about topics of daily living and (b) a cartoon narration task, that is, retellings of short cartoon clips. The frequencies of words and gestures as well as of different gesture types produced by the participants were analyzed and tested for potential effects of group and task. Results Main results for task effects revealed that PWA and CP used more iconic gestures and pantomimes in the cartoon narration task than in spontaneous conversation. Metaphoric gestures, deictic gestures, number gestures, and emblems were more frequently used in spontaneous conversation than in cartoon narrations by both participant groups. Group effects show that, in both tasks, PWA's gesture-to-word ratios were higher than those for the CP. Furthermore, PWA produced more interactive gestures than the CP in both tasks, as well as more number gestures and pantomimes in spontaneous conversation. Conclusions The current results suggest that PWA use gestures to compensate for their verbal limitations under varying communicative constraints. The properties of the communicative task influence the use of different gesture types in people with and without aphasia. Thus, the influence of communicative constraints needs to be considered when assessing PWA's multimodal communicative abilities.

Differential roles of gestures on spatial language in neurotypical elderly adults and individuals with focal brain injury

Gestures might serve communicative functions by supplementing spoken expressions or restorative functions by facilitating speech production. Also, speakers with speech deficits use gestures to compensate for their speech impairments. In this study, we examined gesture use in speakers with and without speech impairments and how spoken spatial expressions changed when gestures were restrained. Six patients with speech problems and with left frontal and/or temporal lesions and 20 neurotypical controls described motion events in 3 different conditions (spontaneous gesture, only speech, and only gesture). In addition to the group analyses, we ran case analyses. Results showed that patients used more gestures compared to controls. Gestures served both communicative and restorative functions for patients whereas controls only used gestures for communicative purposes. Case analyses revealed that there were differential patterns among patients. Overall, gesture production is multifunctional and gestures serve different functions for different populations as well as within a population.

The relationship between co-speech gesture production and macrolinguistic discourse abilities in people with focal brain injury

Brain damage is associated with linguistic deficits and might alter co-speech gesture production. Gesture production after focal brain injury has been mainly investigated with respect to intrasentential rather than discourse-level linguistic processing. In this study, we examined 1) spontaneous gesture production patterns of people with left hemisphere damage (LHD) or right hemisphere damage (RHD) in a narrative setting, 2) the neural structures associated with deviations in spontaneous gesture production in these groups, and 3) the relationship between spontaneous gesture production and discourse level linguistic processes (narrative complexity and evaluation competence). Individuals with LHD or RHD (17 people in each group) and neurotypical controls (n = 13) narrated a story from a picture book. Results showed that increase in gesture production for LHD individuals was associated with less complex narratives and lesions of individuals who produced more gestures than neurotypical individuals overlapped in frontal-temporal structures and basal ganglia. Co-speech gesture production of RHD individuals positively correlated with their evaluation competence in narrative. Lesions of RHD individuals who produced more gestures overlapped in the superior temporal gyrus and the inferior parietal lobule. Overall, LHD individuals produced more gestures than neurotypical individuals. The groups did not differ in their use of different gesture forms except that LHD individuals produced more deictic gestures per utterance than RHD individuals and controls. Our findings are consistent with the hypothesis that co-speech gesture production interacts with macro-linguistic levels of discourse and this interaction is affected by the hemispheric lateralization of discourse abilities.

Multimodal Communication in Aphasia: Perception and Production of Co-speech Gestures During Face-to-Face Conversation

The role of nonverbal communication in patients with post-stroke language impairment (aphasia) is not yet fully understood. This study investigated how aphasic patients perceive and produce co-speech gestures during face-to-face interaction, and whether distinct brain lesions would predict the frequency of spontaneous co-speech gesturing. For this purpose, we recorded samples of conversations in patients with aphasia and healthy participants. Gesture perception was assessed by means of a head-mounted eye-tracking system, and the produced co-speech gestures were coded according to a linguistic classification system. The main results are that meaning-laden gestures (e.g., iconic gestures representing object shapes) are more likely to attract visual attention than meaningless hand movements, and that patients with aphasia are more likely to fixate co-speech gestures overall than healthy participants. This implies that patients with aphasia may benefit from the multimodal information provided by co-speech gestures. On the level of co-speech gesture production, we found that patients with damage to the anterior part of the arcuate fasciculus showed a higher frequency of meaning-laden gestures. This area lies in close vicinity to the premotor cortex and is considered to be important for speech production. This may suggest that the use of meaning-laden gestures depends on the integrity of patients’ speech production abilities.

Comprehensibility and neural substrate of communicative gestures in severe aphasia

Communicative gestures can compensate incomprehensibility of oral speech in severe aphasia, but the brain damage that causes aphasia may also have an impact on the production of gestures. We compared the comprehensibility of gestural communication of persons with severe aphasia and non-aphasic persons and used voxel based lesion symptom mapping (VLSM) to determine lesion sites that are responsible for poor gestural expression in aphasia. On group level, persons with aphasia conveyed more information via gestures than controls indicating a compensatory use of gestures in persons with severe aphasia. However, individual analysis showed a broad range of gestural comprehensibility. VLSM suggested that poor gestural expression was associated with lesions in anterior temporal and inferior frontal regions. We hypothesize that likely functional correlates of these localizations are selection of and flexible changes between communication channels as well as between different types of gestures and between features of actions and objects that are expressed by gestures.

Investigating structure and function in the healthy human brain: validity of acute versus chronic lesion-symptom mapping

Modern voxel-based lesion-symptom mapping (VLSM) analyses techniques provide powerful tools to examine the relationship between structure and function of the healthy human brain. However, there is still uncertainty on the type of and the appropriate time point of imaging and of behavioral testing for such analyses. Here we tested the validity of the three most common combinations of structural imaging data and behavioral scores used in VLSM analyses. Given the established knowledge about the neural substrate of the primary motor system in humans, we asked the mundane question of where the motor system is represented in the normal human brain, analyzing individual arm motor function of 60 unselected stroke patients. Only the combination of acute behavioral scores and acute structural imaging precisely identified the principal brain area for the emergence of hemiparesis after stroke, i.e., the corticospinal tract (CST). In contrast, VLSM analyses based on chronic behavior-in combination with either chronic or acute imaging-required the exclusion of patients who had recovered from an initial paresis to reveal valid anatomical results. Thus, if the primary research aim of a VLSM lesion analysis is to uncover the neural substrates of a certain function in the healthy human brain and if no longitudinal designs with repeated evaluations are planned, the combination of acute imaging and behavior represents the ideal dataset.

Spontaneous gesture and spatial language: Evidence from focal brain injury

People often use spontaneous gestures when communicating spatial information. We investigated focal brain-injured individuals to test the hypotheses that (1) naming motion event components of manner-path (represented by verbs-prepositions in English) are impaired selectively, (2) gestures compensate for impaired naming. Patients with left or right hemisphere damage (LHD or RHD) and elderly control participants were asked to describe motion events (e.g., running across) depicted in brief videos. Damage to the left posterior middle frontal gyrus, left inferior frontal gyrus, and left anterior superior temporal gyrus (aSTG) produced impairments in naming paths of motion; lesions to the left caudate and adjacent white matter produced impairments in naming manners of motion. While the frequency of spontaneous gestures were low, lesions to the left aSTG significantly correlated with greater production of path gestures. These suggest that producing prepositions-verbs can be separately impaired and gesture production compensates for naming impairments when damage involves left aSTG.

Gestures make memories, but what kind? Patients with impaired procedural memory display disruptions in gesture production and comprehension

Hand gesture, a ubiquitous feature of human interaction, facilitates communication. Gesture also facilitates new learning, benefiting speakers and listeners alike. Thus, gestures must impact cognition beyond simply supporting the expression of already-formed ideas. However, the cognitive and neural mechanisms supporting the effects of gesture on learning and memory are largely unknown. We hypothesized that gesture's ability to drive new learning is supported by procedural memory and that procedural memory deficits will disrupt gesture production and comprehension. We tested this proposal in patients with intact declarative memory, but impaired procedural memory as a consequence of Parkinson's disease (PD), and healthy comparison participants with intact declarative and procedural memory. In separate experiments, we manipulated the gestures participants saw and produced in a Tower of Hanoi (TOH) paradigm. In the first experiment, participants solved the task either on a physical board, requiring high arching movements to manipulate the discs from peg to peg, or on a computer, requiring only flat, sideways movements of the mouse. When explaining the task, healthy participants with intact procedural memory displayed evidence of their previous experience in their gestures, producing higher, more arching hand gestures after solving on a physical board, and smaller, flatter gestures after solving on a computer. In the second experiment, healthy participants who saw high arching hand gestures in an explanation prior to solving the task subsequently moved the mouse with significantly higher curvature than those who saw smaller, flatter gestures prior to solving the task. These patterns were absent in both gesture production and comprehension experiments in patients with procedural memory impairment. These findings suggest that the procedural memory system supports the ability of gesture to drive new learning.

Perception of co-speech gestures in aphasic patients: a visual exploration study during the observation of dyadic conversations

BACKGROUND

Co-speech gestures are part of nonverbal communication during conversations. They either support the verbal message or provide the interlocutor with additional information. Furthermore, they prompt as nonverbal cues the cooperative process of turn taking. In the present study, we investigated the influence of co-speech gestures on the perception of dyadic dialogue in aphasic patients. In particular, we analysed the impact of co-speech gestures on gaze direction (towards speaker or listener) and fixation of body parts. We hypothesized that aphasic patients, who are restricted in verbal comprehension, adapt their visual exploration strategies.

METHODS

Sixteen aphasic patients and 23 healthy control subjects participated in the study. Visual exploration behaviour was measured by means of a contact-free infrared eye-tracker while subjects were watching videos depicting spontaneous dialogues between two individuals. Cumulative fixation duration and mean fixation duration were calculated for the factors co-speech gesture (present and absent), gaze direction (to the speaker or to the listener), and region of interest (ROI), including hands, face, and body.

RESULTS

Both aphasic patients and healthy controls mainly fixated the speaker's face. We found a significant co-speech gesture × ROI interaction, indicating that the presence of a co-speech gesture encouraged subjects to look at the speaker. Further, there was a significant gaze direction × ROI × group interaction revealing that aphasic patients showed reduced cumulative fixation duration on the speaker's face compared to healthy controls.

CONCLUSION

Co-speech gestures guide the observer's attention towards the speaker, the source of semantic input. It is discussed whether an underlying semantic processing deficit or a deficit to integrate audio-visual information may cause aphasic patients to explore less the speaker's face.