Looking while listening and speaking: eye-to-face gaze in adolescents with and without traumatic brain injury

Attentional switch to memory: An early and critical phase of the cognitive cascade allowing autobiographical memory retrieval

Remembering and mentally reliving yesterday's lunch is a typical example of episodic autobiographical memory retrieval. In the present review, we reappraised the complex cascade of cognitive processes involved in memory retrieval, by highlighting one particular phase that has received little interest so far: attentional switch to memory (ASM). As attention cannot be simultaneously directed toward external stimuli and internal memories, there has to be an attentional switch from the external to the internal world in order to initiate memory retrieval. We formulated hypotheses and developed hypothetical models of both the cognitive and brain processes that accompany ASM. We suggest that gaze aversion could serve as an objective temporal marker of the point at which people switch their attention to memory, and highlight several fields (neuropsychology, neuroscience, social cognition, comparative psychology) in which ASM markers could be essential. Our review thus provides a new framework for understanding the early stages of autobiographical memory retrieval.

Differential Effects of a Behavioral Treatment Probe on Social Gaze Behavior in Fragile X Syndrome and Non-Syndromic Autism Spectrum Disorder

The purpose of this study was to examine potential differences in social learning between individuals with fragile X syndrome (FXS), the leading known inherited cause of intellectual disability, and individuals with non-syndromic autism spectrum disorder (ASD). Thirty school-aged males with FXS and 26 age and symptom-matched males with non-syndromic ASD, were administered a behavioral treatment probe designed to improve levels of social gaze during interactions with others. The treatment probe was administered by a trained behavior therapist over two days in our laboratory and included reinforcement of social gaze in two alternating training conditions - looking while listening and looking while speaking. Prior to each session, children in each group were taught progressive muscle relaxation and breathing techniques to counteract potential increased hyperarousal. Measures included the rate of learning in each group during treatment, in addition to levels of social gaze and heart rate obtained during administration of a standardized social conversation task administered prior to and following the treatment probe. Results showed that learning rates obtained during administration of the treatment probe were significantly less steep and less variable for males with FXS compared to males with non-syndromic ASD. Significant improvements in social gaze were also observed for males with FXS during the social conversation task. There was no effect of the treatment probe on heart rate in either group. These data reveal important differences in social learning between the two groups and have implications for early interventions in the two conditions.

Gaze direction as a facial cue of memory retrieval state

Gaze direction is a powerful social cue that indicates the direction of attention and can be used to decode others' mental states. When an individual looks at an external object, inferring where their attention is focused from their gaze direction is easy. But when people are immersed in memories, their attention is oriented towards their inner world. Is there any specific gaze direction in this situation, and if so, which one? While trying to remember, a common behavior is gaze aversion, which has mostly been reported as an upward-directed gaze. Our primary aim was to evaluate whether gaze direction plays a role in the inference of the orientation of attention-i.e., external vs. internal-in particular, whether an upward direction is considered as an indicator of attention towards the internal world. Our secondary objective was to explore whether different gaze directions are consistently attributed to different types of internal mental states and, more specifically, memory states (autobiographical or semantic memory retrieval, or working memory). Gaze aversion is assumed to play a role in perceptual decoupling, which is supposed to support internal attention. We therefore also tested whether internal attention was associated with high gaze eccentricity because the mismatch between head and eye direction alters visual acuity. We conducted two large-sample (160-163 participants) online experiments. Participants were asked to choose which mental state-among different internal and external attentional states-they would attribute to faces with gazes oriented in different directions. Participants significantly associated internal attention with an upward-averted gaze across experiments, while external attention was mostly associated with a gaze remaining on the horizontal axis. This shows that gaze direction is robustly used by observers to infer others' mental states. Unexpectedly, internal attentional states were not more associated with gaze eccentricity at high (30°) than low (10°) eccentricity and we found that autobiographical memory retrieval, but not the other memory states, was highly associated with 10° downward gaze. This reveals the possible existence of different types of gaze aversion for different types of memories and opens new perspectives.

Computer-Mediated Communication in Adults With and Without Moderate-to-Severe Traumatic Brain Injury: Survey of Social Media Use

BACKGROUND

Individuals with a history of traumatic brain injury (TBI) report fewer social contacts, less social participation, and more social isolation than noninjured peers. Cognitive-communication disabilities may prevent individuals with TBI from accessing the opportunities for social connection afforded by computer-mediated communication, as individuals with TBI report lower overall usage of social media than noninjured peers and substantial challenges with accessibility and usability. Although adaptations for individuals with motor and sensory impairments exist to support social media use, there have been no parallel advances to support individuals with cognitive disabilities, such as those exhibited by some people with TBI. In this study, we take a preliminary step in the development process by learning more about patterns of social media use in individuals with TBI as well as their input and priorities for developing social media adaptations.

OBJECTIVE

This study aims to characterize how and why adults with TBI use social media and computer-mediated communication platforms, to evaluate changes in computer-mediated communication after brain injury, and to elicit suggestions from individuals with TBI to improve access to social media after injury.

METHODS

We conducted a web-based survey of 53 individuals with a chronic history of moderate-to-severe TBI and a demographically matched group of 51 noninjured comparison peers.

RESULTS

More than 90% of participants in both groups had an account on at least one computer-mediated communication platform, with Facebook and Facebook Messenger being the most popular platforms in both groups. Participants with and without a history of TBI reported that they use Facebook more passively than actively and reported that they most frequently maintain web-based relationships with close friends and family members. However, participants with TBI reported less frequently than noninjured comparison participants that they use synchronous videoconferencing platforms, are connected with acquaintances on the web, or use social media as a gateway for offline social connection (eg, to find events). Of the participants with TBI, 23% (12/53) reported a change in their patterns of social media use caused by brain injury and listed concerns about accessibility, safety, and usability as major barriers.

CONCLUSIONS

Although individuals with TBI maintain social media accounts to the same extent as healthy comparisons, some may not use them in a way that promotes social connection. Thus, it is important to design social media adaptations that address the needs and priorities of individuals with TBI, so they can also reap the benefits of social connectedness offered by these platforms. By considering computer-mediated communication as part of individuals' broader social health, we may be able to increase web-based participation in a way that is meaningful, positive, and beneficial to broader social life.

The Role of Gesture in Communication and Cognition: Implications for Understanding and Treating Neurogenic Communication Disorders

When people talk, they gesture. Gesture is a fundamental component of language that contributes meaningful and unique information to a spoken message and reflects the speaker's underlying knowledge and experiences. Theoretical perspectives of speech and gesture propose that they share a common conceptual origin and have a tightly integrated relationship, overlapping in time, meaning, and function to enrich the communicative context. We review a robust literature from the field of psychology documenting the benefits of gesture for communication for both speakers and listeners, as well as its important cognitive functions for organizing spoken language, and facilitating problem-solving, learning, and memory. Despite this evidence, gesture has been relatively understudied in populations with neurogenic communication disorders. While few studies have examined the rehabilitative potential of gesture in these populations, others have ignored gesture entirely or even discouraged its use. We review the literature characterizing gesture production and its role in intervention for people with aphasia, as well as describe the much sparser literature on gesture in cognitive communication disorders including right hemisphere damage, traumatic brain injury, and Alzheimer's disease. The neuroanatomical and behavioral profiles of these patient populations provide a unique opportunity to test theories of the relationship of speech and gesture and advance our understanding of their neural correlates. This review highlights several gaps in the field of communication disorders which may serve as a bridge for applying the psychological literature of gesture to the study of language disorders. Such future work would benefit from considering theoretical perspectives of gesture and using more rigorous and quantitative empirical methods in its approaches. We discuss implications for leveraging gesture to explore its untapped potential in understanding and rehabilitating neurogenic communication disorders.

The Measurement of Eye Contact in Human Interactions: A Scoping Review

Eye contact is a fundamental aspect of nonverbal communication and therefore important for understanding human interaction. Eye contact has been the subject of research in many disciplines, including communication sciences, social psychology, and psychiatry, and a variety of techniques have been used to measure it. The choice of measurement method has consequences for research outcomes and their interpretation. To ensure that research findings align with study aims and populations, it is essential that methodological choices are well substantiated. Therefore, to enhance the effective examination of eye contact, we performed a literature review of the methods used to study eye contact. We searched Medline, PsycINFO and Web of Science for empirical peer-reviewed articles published in English that described quantitative studies on human eye contact and included a methodological description. The identified studies (N = 109) used two approaches to assess eye contact: direct, i.e., assessing eye contact while it is occurring, and indirect, i.e., assessing eye contact retrospectively (e.g., from video recordings). Within these categories, eight specific techniques were distinguished. Variation was found regarding the reciprocity of eye contact between two individuals, the involvement of an assessor and the behavior of participants while being studied. Measures not involving the interactors in assessment of eye contact and have a higher spatial and temporal resolution, such as eye tracking, have gained popularity. Our results show wide methodological diversity regarding the measurement of eye contact. Although studies often define eye contact as gaze towards an exact location, this may not do justice to the subjective character of eye contact. The various methodologies have hardly ever been compared, limiting the ability to compare findings between studies. Future studies should take notice of the controversy surrounding eye contact measures.

Effects of animal-assisted therapy on social behaviour in patients with acquired brain injury: a randomised controlled trial

Animal-assisted therapy (AAT) is increasingly used to address impaired social competence in patients with acquired brain injury. However, the efficacy of AAT has not been tested in these patients. We used a randomised, controlled within subject trial to determine the effects of AAT on social competence in patients undergoing stationary neurorehabilitation. Participants received both AAT sessions and paralleled conventional therapy sessions. The patients’ social behaviour was systematically coded on the basis of video recordings of therapy sessions. Moreover, mood, treatment motivation and satisfaction was measured during each therapy session. We analysed 222 AAT and 219 control sessions of 19 patients with linear mixed models. Patients showed a significantly higher amount of social behaviour during AAT. Furthermore, patients’ positive emotions, verbal and non-verbal communication, mood, treatment motivation and satisfaction were increased in the presence of an animal. Neutral emotions were reduced but no effect was found regarding negative emotions. Our results show that AAT increases aspects of social competence and leads to higher emotional involvement of patients with acquired brain injury, reflected in higher social engagement, motivation and satisfaction during a therapeutic session.

Loneliness and the social monitoring system: Emotion recognition and eye gaze in a real-life conversation

Based on the belongingness regulation theory (Gardner et al., 2005, Pers. Soc. Psychol. Bull., 31, 1549), this study focuses on the relationship between loneliness and social monitoring. Specifically, we examined whether loneliness relates to performance on three emotion recognition tasks and whether lonely individuals show increased gazing towards their conversation partner's faces in a real-life conversation. Study 1 examined 170 college students (Mage = 19.26; SD = 1.21) who completed an emotion recognition task with dynamic stimuli (morph task) and a micro(-emotion) expression recognition task. Study 2 examined 130 college students (Mage = 19.33; SD = 2.00) who completed the Reading the Mind in the Eyes Test and who had a conversation with an unfamiliar peer while their gaze direction was videotaped. In both studies, loneliness was measured using the UCLA Loneliness Scale version 3 (Russell, 1996, J. Pers. Assess., 66, 20). The results showed that loneliness was unrelated to emotion recognition on all emotion recognition tasks, but that it was related to increased gaze towards their conversation partner's faces. Implications for the belongingness regulation system of lonely individuals are discussed.

Coordination of gaze and speech in communication between children with hearing impairment and normal-hearing peers

PURPOSE

To investigate gaze behavior during communication between children with hearing impairment (HI) and normal-hearing (NH) peers.

METHOD

Ten HI-NH and 10 NH-NH dyads performed a referential communication task requiring description of faces. During task performance, eye movements and speech were tracked. Using verbal event (questions, statements, back channeling, and silence) as the predictor variable, group characteristics in gaze behavior were expressed with Kaplan-Meier survival functions (estimating time to gaze-to-partner) and odds ratios (comparing number of verbal events with and without gaze-to-partner). Analyses compared the listeners in each dyad (HI: n = 10, mean age = 12;6 years, mean better ear pure-tone average = 33.0 dB HL; NH: n = 10, mean age = 13;7 years).

RESULTS

Log-rank tests revealed significant group differences in survival distributions for all verbal events, reflecting a higher probability of gaze to the partner's face for participants with HI. Expressed as odds ratios (OR), participants with HI displayed greater odds for gaze-to-partner (ORs ranging between 1.2 and 2.1) during all verbal events.

CONCLUSIONS

The results show an increased probability for listeners with HI to gaze at the speaker's face in association with verbal events. Several explanations for the finding are possible, and implications for further research are discussed.

Assessing the sensitivity of the evaluation of social interaction: comparing social skills in children with and without disabilities

OBJECTIVE

We examined the sensitivity of the Evaluation of Social Interaction (ESI) as a measure of the overall quality of social interaction in children as they engage in social exchanges in a natural context with typical social partners.

METHOD

We compared the ESI measures of 23 children with disabilities with those of 23 children without disabilities, matched by age and gender.

RESULTS

Paired t-test analysis revealed a statistically significant difference, t(22) = -4.065, p = .001, in the quality of social interaction for the two groups of children, indicating sensitivity of the ESI.

CONCLUSION

The results support the use of the ESI to measure quality of social interaction in a natural context.

Timing of gazes in child dialogues: a time-course analysis of requests and back channelling in referential communication

BACKGROUND

This study investigates gaze behaviour in child dialogues. In earlier studies the authors have investigated the use of requests for clarification and responses in order to study the co-creation of understanding in a referential communication task. By adding eye tracking, this line of research is now expanded to include non-verbal contributions in conversation.

AIMS

To investigate the timing of gazes in face-to-face interaction and to relate the gaze behaviour to the use of requests for clarification.

METHODS & PROCEDURES

Eight conversational pairs of typically developing 10-15 year olds participated. The pairs (director and executor) performed a referential communication task requiring the description of faces. During the dialogues both participants wore head-mounted eye trackers. All gazes were recorded and categorized according to the area fixated (Task, Face, Off). The verbal context for all instances of gaze at the partner's face was identified and categorized using time-course analysis.

OUTCOMES & RESULTS

The results showed that the executor spends almost 90% of the time fixating the gaze on the task, 10% on the director's face and less than 0.5% elsewhere. Turn shift, primarily requests for clarification, and back channelling significantly predicted the executors' gaze to the face of the task director. The distribution of types of requests showed that requests for previously unmentioned information were significantly more likely to be associated with gaze at the director.

CONCLUSIONS & IMPLICATIONS

The study shows that the executors' gaze at the director accompanies important dynamic shifts in the dialogue. The association with requests for clarification indicates that gaze at the director can be used to monitor the response with two modalities. Furthermore, the significantly higher association with requests for previously unmentioned information indicates that gaze may be used to emphasize the verbal content. The results will be used as a reference for studies of gaze behaviour in clinical populations with hearing and language impairments.

Analysing Conversational Discourse After Traumatic Brain Injury: Isn't It About Time?

Most studies of conversational discourse in traumatic brain injury (TBI) have used summary measures averaged over time. These measures have yielded useful aggregate data but do not capture the temporal dynamics of interpersonal interactions. To fully characterise the conversational discourse of individuals with TBI, it may be useful to consider individuals and behaviours as systems changing over time. In this article, we review dynamic systems approaches that have been used to study conversational interactions, and present data to illustrate their potential utility in characterising the conversational discourse of individuals with TBI.

Evaluation of social interaction during occupational engagement

The Evaluation of Social Interaction (ESI; Fisher & Griswold, 2008) assesses a person's performance of social interaction skills in the natural context with typical social partners during any area of occupation. We used Rasch analysis of 175 observations of 128 people, ages 4-73, to examine internal scale validity, the items' skill hierarchy and intended purpose, and the ESI's ability to differentiate between people with and without disabilities. The ESI demonstrated validity for 24 of 27 skills and six intended purposes, with a hierarchy of performance. Of the observations, 95.3% demonstrated goodness of fit to the Rasch model, indicating person response validity. People without a disability demonstrated significantly higher social skills performance than those with a known disability (t= 4.468, df= 83 p= .000). The ESI has the potential to provide a quantitative assessment of social interaction performance in the natural context of a person's desired occupation and may be useful for intervention planning and outcome measurement.