Affective Eye Contact: An Integrative Review

Why the gaze behavior of expert physicians and novice medical students differ during a simulated medical interview: A mixed methods study

Human cognition is reflected in gaze behavior, which involves eye movements to fixate or shift focus between areas. In natural interactions, gaze behavior serves two functions: signal transmission and information gathering. While expert gaze as a tool for gathering information has been studied, its underlying cognitive processes remain insufficiently explored. This study investigated differences in gaze behavior and cognition between expert physicians and novice medical students during a simulated medical interview with a simulated patient, drawing implications for medical education. This study employed an exploratory sequential mixed methods design. During the simulated medical interview, participants' gaze behavior was measured across five areas: the patient's eyes, face, body trunk, medical chart, and medical questionnaire. A hierarchical Bayesian model analyzed differences in gaze behavior between expert physicians and novice medical students. Then, a semi-structured interview was conducted with participants to discern their perceptions during their gaze behavior; their recorded gaze behavior was presented to them, and analyzed using a qualitative descriptive approach. Model analyses indicated that experts looked at the simulated patient's eyes less frequently compared to novices during the simulated medical interview. Expert physicians stated that because of the potential for discomfort, looking at the patient's eyes was less frequent, despite its importance for obtaining diagnostic findings. Conversely, novice medical students did not provide narratives for obtaining such findings, but increased the number of times they did so to improve patient satisfaction. This association between different perceptions of gaze behavior may lead to new approaches in medical education. This study highlights the importance of understanding gaze behavior in the context of medical education and suggests that different motivations underlie the gaze behavior of expert physicians and novice medical students. Incorporating training in effective gaze behavior may improve the quality of patient care and medical students' learning outcomes.

The emotion of sound target modulates the auditory gaze cueing effect

The auditory gaze cueing effect (auditory-GCE) is a faster response to auditory targets at an eye-gaze cue location than at a non-cue location. Previous research has found that auditory-GCE can be influenced by the integration of both gaze direction and emotion conveyed through facial expressions. However, it is unclear whether the emotional information of auditory targets can be cross-modally integrated with gaze direction to affect auditory-GCE. Here, we set neutral faces with different gaze directions as cues and three emotional sounds (fearful, happy, and neutral) as targets to investigate how the emotion of sound target modulates the auditory-GCE. Moreover, we conducted a controlled experiment using arrow cues. The results show that the emotional content of sound targets influences the auditory-GCE but only for those induced by facial cues. Specifically, fearful sounds elicit a significantly larger auditory-GCE compared to happy and neutral sounds, indicating that the emotional content of auditory targets plays a modulating role in the auditory-GCE. Furthermore, this modulation appears to occur only at a higher level of social meaning, involving the integration of emotional information from a sound with social gaze direction, rather than at a lower level, which involves the integration of direction and auditory emotion.

What is in a look? The accountability of gaze in trajectories to conflict

This study investigates the role of gaze in initiating episodes of conflict by examining, using multimodal conversation analysis, a set of cases in which a recipient is prompted to speak by another's extended gaze. In these cases, this recipient response may be, e.g., "What," or a more elaborate demand for an account, such as "Why are you looking at me like that for?" Here we investigate the characteristics of the gaze that prompts such responses, and what actions such responses constitute. While "What" compositionally resembles other-initiated repair, its sequential position characterizes it as a so-called "go-ahead" action. In these cases, the sequential positioning of such gazes, constituting it structurally as a so-called "pre," alongside its durational characteristics and facial expression, are examined to identify the normative associations of gaze and subsequent conduct that make such gazes accountable.

Social face processing in chronic severe traumatic brain injury: Altered decoding of emotions and mental states but preserved gaze cueing of attention

The processing of social information transmitted by facial stimuli is altered in individuals with traumatic brain injury (TBI). This study investigated whether these alterations also affect the mechanisms underlying the orienting of visual attention in response to eye-gaze signals. TBI patients and a control group of healthy individuals matched on relevant criteria completed a spatial cueing task. In this task, a lateral visual target was presented along with a task-irrelevant face, with the gaze averted to the left or right. Arrows pointing towards the left or right were also used as non-social control stimuli. Social cognition abilities were further investigated through tests based on decoding emotional expressions and mental states conveyed by facial stimuli. The decoding of emotions and mental states was worse in the TBI group than in the control group. However, both groups demonstrated reliable and comparable orienting of attention to both eye-gaze and arrow stimuli. Despite impairments in certain aspects of social face processing among TBI patients, gaze cueing of attention appears to be preserved in this neuropsychological population.

Infants' psychophysiological responses to eye contact with a human and with a humanoid robot

Eye contact with a human and with a humanoid robot elicits attention- and affect-related psychophysiological responses. However, these responses have mostly been studied in adults, leaving their developmental origin poorly understood. In this study, 114 infants (6-8 months old) viewed direct and averted gaze directions of a live human and an embodied humanoid robot while their heart rate deceleration (attention orienting), skin conductance (affective arousal), and facial muscle activity (affective valence) were measured. In addition, a non-humanoid object (a vase) was used as a control stimulus. Infants' attention orienting was stronger to averted versus direct gaze of a human and a robot, but indifferent to the averted versus direct orientation of the non-humanoid object. Moreover, infants' attention orienting was equally intensive toward a human and a robot, but less intensive toward a non-humanoid object. Affective arousal was insensitive to gaze direction and did not differ between the human, the robot, and the non-humanoid object. Facial muscle responses showed sensitivity to the gaze direction of a human and of a robot but not to the orientation of the non-humanoid object. These results suggest that infants recognize the attentional and affective/affiliative significance not only in a human's gaze but also in a robot's gaze.

Neural and affective responses to prolonged eye contact with parents in depressed and nondepressed adolescents

Eye contact improves mood, facilitates connectedness, and is assumed to strengthen the parent-child bond. Adolescent depression is linked to difficulties in social interactions, the parent-child bond included. Our goal was to elucidate adolescents' affective and neural responses to prolonged eye contact with one's parent in nondepressed adolescents (HC) and how these responses are affected in depressed adolescents. While in the scanner, 59 nondepressed and 19 depressed adolescents were asked to make eye contact with their parent, an unfamiliar peer, an unfamiliar adult, and themselves by using videos of prolonged direct and averted gaze, as an approximation of eye contact. After each trial, adolescents reported on their mood and feelings of connectedness, and eye movements and BOLD-responses were assessed. In HCs, eye contact boosted mood and feelings of connectedness and increased activity in inferior frontal gyrus (IFG), temporal pole, and superior frontal gyrus. Unlike HCs, eye contact did not boost the mood of depressed adolescents. While HCs reported increased mood and feelings of connectedness to the sight of their parent versus others, depressed adolescents did not. Depressed adolescents exhibited blunted overall IFG activity. These findings show that adolescents are particularly sensitive to eye contact and respond strongly to the sight of their parents. This sensitivity seems to be blunted in depressed adolescents. For clinical purposes, it is important to gain a better understanding of how the responsivity to eye contact in general and with their parents in particular, can be restored in adolescents with depression.

Off-camera gaze decreases evaluation scores in a simulated online job interview

During the pandemic, digital communication became paramount. Due to the discrepancy between the placement of the camera and the screen in typical smartphones, tablets and laptops, mutual eye contact cannot be made in standard video communication. Although the positive effect of eye contact in traditional communication has been well-documented, its role in virtual contexts remains less explored. In this study, we conducted experiments to gauge the impact of gaze direction during a simulated online job interview. Twelve university students were recruited as interviewees. The interview consisted of two recording sessions where they delivered the same prepared speech: in the first session, they faced the camera, and in the second, they directed their gaze towards the screen. Based on the recorded videos, we created three stimuli: one where the interviewee's gaze was directed at the camera (CAM), one where the interviewee's gaze was skewed downward (SKW), and a voice-only stimulus without camera recordings (VO). Thirty-eight full-time workers participated in the study and evaluated the stimuli. The results revealed that the SKW condition garnered significantly less favorable evaluations than the CAM condition and the VO condition. Moreover, a secondary analysis indicated a potential gender bias in evaluations: the female evaluators evaluated the interviewees of SKW condition more harshly than the male evaluators did, and the difference in some evaluation criteria between the CAM and SKW conditions was larger for the female interviewees than for the male interviewees. Our findings emphasize the significance of gaze direction and potential gender biases in online interactions.

Social Communication in Young Children With Sex Chromosome Trisomy (XXY, XXX, XYY): A Study With Eye Tracking and Heart Rate Measures

OBJECTIVE

Children with sex chromosome trisomy (SCT) have an increased risk for suboptimal development. Difficulties with language are frequently reported, start from a very young age, and encompass various domains. This cross-sectional study examined social orientation with eye tracking and physiological arousal responses to gain more knowledge on how children perceive and respond to communicative bids and evaluated the associations between social orientation and language outcomes, concurrently and 1 year later.

METHOD

In total, 107 children with SCT (33 XXX, 50 XXY, and 24 XYY) and 102 controls (58 girls and 44 boys) aged between 1 and 7 years were included. Assessments took place in the USA and Western Europe. A communicative bids eye tracking paradigm, physiological arousal measures, and receptive and expressive language outcomes were used.

RESULTS

Compared to controls, children with SCT showed reduced attention to the face and eyes of the on-screen interaction partner and reduced physiological arousal sensitivity in response to direct versus averted gaze. In addition, social orientation to the mouth was related to concurrent receptive and expressive language abilities in 1-year-old children with SCT.

CONCLUSIONS

Children with SCT may experience difficulties with social communication that extend past the well-recognized risk for early language delays. These difficulties may underlie social-behavioral problems that have been described in the SCT population and are an important target for early monitoring and support.

Depressive symptoms and visual attention to others' eyes in healthy individuals

BACKGROUND

Eye contact is a fundamental part of social interaction. In clinical studies, it has been observed that patients suffering from depression make less eye contact during interviews than healthy individuals, which could be a factor contributing to their social functioning impairments. Similarly, results from mood induction studies with healthy persons indicate that attention to the eyes diminishes as a function of sad mood. The present screen-based eye-tracking study examined whether depressive symptoms in healthy individuals are associated with reduced visual attention to other persons' direct gaze during free viewing.

METHODS

Gaze behavior of 44 individuals with depressive symptoms and 49 individuals with no depressive symptoms was analyzed in a free viewing task. Grouping was based on the Beck Depression Inventory using the cut-off proposed by Hautzinger et al. (2006). Participants saw pairs of faces with direct gaze showing emotional or neutral expressions. One-half of the face pairs was shown without face masks, whereas the other half was presented with face masks. Participants' dwell times and first fixation durations were analyzed.

RESULTS

In case of unmasked facial expressions, participants with depressive symptoms looked shorter at the eyes compared to individuals without symptoms across all expression conditions. No group difference in first fixation duration on the eyes of masked and unmasked faces was observed. Individuals with depressive symptoms dwelled longer on the mouth region of unmasked faces. For masked faces, no significant group differences in dwell time on the eyes were found. Moreover, when specifically examining dwell time on the eyes of faces with an emotional expression there were also no significant differences between groups. Overall, participants gazed significantly longer at the eyes in masked compared to unmasked faces.

CONCLUSIONS

For faces without mask, our results suggest that depressiveness in healthy individuals goes along with less visual attention to other persons' eyes but not with less visual attention to others' faces. When factors come into play that generally amplify the attention directed to the eyes such as face masks or emotions then no relationship between depressiveness and visual attention to the eyes can be established.

Sharing and Receiving Eye-Contact Predicts Mate Choice After a 5-Minute Conversation: Evidence from a Speed-Dating Study

In popular narratives, the first date with a potential mate often centers on their gaze as embodiment of interest and attraction. However, evidence is still lacking on the role of eye-contact as a potent signal in human social interaction in the context of dating. In addition, behavioral mechanisms of mate selection are not well understood. In the present study, we therefore examined mutual eye-contact and its influence on mate choice by applying dual mobile eye-tracking during naturalistic speed-dates. A total of 30 male and 30 female subjects attended four speed-dates each (N = 240). Subjects were more likely to choose those dating partners with whom they shared more eye-contact with. In addition, perceived attractiveness played an important role for mate choice. Interestingly, receiving but not giving eye-contact also predicted individual mate choice. Eye-contact thus acts as an important signal of romantic attraction when encountering a dating partner.

'Le regard des autres': the experience of walking in social environments as a person with Parkinson's disease

OBJECTIVE

To examine the experience of people with Parkinson's disease when walking in different social situations, and improve understanding of how this affects participation in meaningful activity.

METHODS

A convenience sample of fourteen people with Parkinson's disease and a history of gait dysfunction was recruited. In-depth interviews and direct observations were conducted in the participants' home environments. Specific examples from community mobility were reviewed using first person interviewing techniques with the support of video footage. Interview transcripts were analyzed using an interpretive phenomenological approach to derive key themes.

RESULTS

The feeling of 'being looked at' (le regard des autres) was the central theme in participant discourse. This sentiment was inextricably linked to the given norms of the social setting, and the relationships between participants and others within that environment. Participants sought to manage how they were perceived by others through modification of posture/gait patterns; disclosure of their neurological disease; and avoidance/withdrawal from social situations.

CONCLUSION

Further to the functional aspects of mobility, gait is important for maintaining self-image in people with Parkinson's disease. Affective gaze interactions have significant consequences upon participation restriction. These findings underscore the interest of activities which strengthen self-image and validate movement diversity in PD rehabilitation.

Autonomic and facial electromyographic responses to watching eyes

We measured participants' psychophysiological responses and gaze behavior while viewing a stimulus person's direct and averted gaze in three different conditions manipulating the participants' experience of being watched. The results showed that skin conductance responses and heart rate deceleration responses were greater to direct than averted gaze only in the condition in which the participants had the experience of being watched by the other individual. In contrast, gaze direction had no effects on these responses when the participants were manipulated to believe that the other individual could not watch them or when the stimulus person was presented in a pre-recorded video. Importantly, the eye tracking measures showed no differences in participants' looking behavior between these stimulus presentation conditions. The results of facial electromyography responses suggested that direct gaze elicited greater zygomatic and periocular responses than averted gaze did, independent of the presentation condition. It was concluded that the affective arousal and attention-orienting indexing autonomic responses to eye contact are driven by the experience of being watched. In contrast, the facial responses seem to reflect automatized affiliative responses which can be elicited even in conditions in which seeing another's direct gaze does not signal that the self is being watched.

Looking into troubled waters: Childhood emotional maltreatment modulates neural responses to prolonged gazing into one's own, but not others', eyes

One of the most prevalent nonverbal, social phenomena known to automatically elicit self- and other-referential processes is eye contact. By its negative effects on the perception of social safety and views about the self and others, childhood emotional maltreatment (CEM) may fundamentally affect these processes. To investigate whether the socioaffective consequences of CEM may become visible in response to (prolonged) eye gaze, 79 adult participants (mean [M]age = 49.87, standard deviation [SD]age = 4.62) viewed videos with direct and averted gaze of an unfamiliar other and themselves while we recorded self-reported mood, eye movements using eye-tracking, and markers of neural activity using fMRI. Participants who reported higher levels of CEM exhibited increased activity in ventromedial prefrontal cortex to one's own, but not to others', direct gaze. Furthermore, in contrast to those who reported fewer of such experiences, they did not report a better mood in response to a direct gaze of self and others, despite equivalent amounts of time spent looking into their own and other peoples' eyes. The fact that CEM is associated with enhanced neural activation in a brain area that is crucially involved in self-referential processing (i.e., vmPFC) in response to one's own direct gaze is in line with the chronic negative impact of CEM on a person's self-views. Interventions that directly focus on targeting maladaptive self-views elicited during eye gaze to self may be clinically useful.

The Time Course of Information Processing During Eye Direction Perception

Gaze directed at the observer (direct gaze) is a highly salient social signal. Despite the existence of a preferential orientation toward direct gaze, none of the studies carried out so far seem to have explicitly studied the time course of information processing during gaze direction judgment. In an eye direction judgment task, participants were presented with a sketch of a face. A temporal asynchrony was introduced between the presentation of the eyes and that of the rest of the face. Indeed, the face could be presented before the eyes, the eyes could be presented before the face, or the face and the eyes could be presented simultaneously. In a second time, the face direction was also manipulated. The results suggest that the time course of information processing during eye direction judgment follows a continuum that makes it possible to perceive the eyes first and then to use the facial context to judge the direction of gaze. Furthermore, the congruency between the direction of gaze and that of the face confirms this observation. Although these results are discussed in the light of existing theories about the mechanisms underlying gaze processing, our data provide new information suggesting that, despite their power to capture attention, the eyes probably have to stand out from a more general spatial configuration (i.e., the face) in order for their direction to be adequately processed.

No transfer of arousal from other's eyes in Williams syndrome

Typically developing humans automatically synchronize their arousal levels, resulting in pupillary contagion, or spontaneous adaptation of pupil size to that of others. This phenomenon emerges in infancy and is believed to facilitate social interaction. Williams syndrome (WS) is a genetic condition characterized by a hyper-social personality and social interaction challenges. Pupillary contagion was examined in individuals with WS (n = 44), age-parallel-matched typically developing children and adults (n = 65), and infants (n = 79). Bayesian statistics were used. As a group, people with WS did not show pupillary contagion (Bayes factors supporting the null: 25-50) whereas control groups did. This suggests a very early emerging atypical developmental trajectory. In WS, higher pupillary contagion was associated with lower autistic symptoms of social communication. Diminished synchronization of arousal may explain why individuals with WS have social challenges, whereas synchronization of arousal is not a necessary correlate of high social motivation.

Psychophysiological responses to eye contact with a humanoid robot: Impact of perceived intentionality

Eye contact with a social robot has been shown to elicit similar psychophysiological responses to eye contact with another human. However, it is becoming increasingly clear that the attention- and affect-related psychophysiological responses differentiate between direct (toward the observer) and averted gaze mainly when viewing embodied faces that are capable of social interaction, whereas pictorial or pre-recorded stimuli have no such capability. It has been suggested that genuine eye contact, as indicated by the differential psychophysiological responses to direct and averted gaze, requires a feeling of being watched by another mind. Therefore, we measured event-related potentials (N170 and frontal P300) with EEG, facial electromyography, skin conductance, and heart rate deceleration responses to seeing a humanoid robot's direct versus averted gaze, while manipulating the impression of the robot's intentionality. The results showed that the N170 and the facial zygomatic responses were greater to direct than to averted gaze of the robot, and independent of the robot's intentionality, whereas the frontal P300 responses were more positive to direct than to averted gaze only when the robot appeared intentional. The study provides further evidence that the gaze behavior of a social robot elicits attentional and affective responses and adds that the robot's seemingly autonomous social behavior plays an important role in eliciting higher-level socio-cognitive processing.

Depth from motion parallax: Deictic consistency, eye contact, and a serious problem with Zoom

The dynamics of head and eye gaze between two or more individuals displayed during verbal and nonverbal face-to-face communication contains a wealth of information and is used for both volitionary and unconscious signaling. Current video communication systems convey visual signals about gaze behavior and other directional cues, but the information they carry is often spurious and potentially misleading. I discuss the consequences of this situation, identify the source of the problem as a more general lack of deictic consistency, and demonstrate that using display technologies that simulate motion parallax are both necessary and sufficient to alleviate it. I then devise an avatar-based remote communication solution that achieves deictic consistency and provides natural, dynamic eye contact for computer-mediated audiovisual communication.

A dual mobile eye tracking study on natural eye contact during live interactions

Human eyes convey a wealth of social information, with mutual looks representing one of the hallmark gaze communication behaviors. However, it remains relatively unknown if such reciprocal communication requires eye-to-eye contact or if general face-to-face looking is sufficient. To address this question, while recording looking behavior in live interacting dyads using dual mobile eye trackers, we analyzed how often participants engaged in mutual looks as a function of looking towards the top (i.e., the Eye region) and bottom half of the face (i.e., the Mouth region). We further examined how these different types of mutual looks during an interaction connected with later gaze-following behavior elicited in an individual experimental task. The results indicated that dyads engaged in mutual looks in various looking combinations (Eye-to-eye, Eye-to-mouth, and Mouth-to-Mouth) but proportionately spent little time in direct eye-to-eye gaze contact. However, the time spent in eye-to-eye contact significantly predicted the magnitude of later gaze following response elicited by the partner's gaze direction. Thus, humans engage in looking patterns toward different face parts during interactions, with direct eye-to-eye looks occurring relatively infrequently; however, social messages relayed during eye-to-eye contact appear to carry key information that propagates to affect subsequent individual social behavior.

Hedonic and autonomic responses in promoting affective touch

Interpersonal touch is intrinsically reciprocal since it entails a person promoting and another receiving the touch. While several studies have investigated the beneficial effects of receiving affective touch, the affective experience of caressing another individual remains largely unknown. Here, we investigated the hedonic and autonomic responses (skin conductance and heart rate) in the person promoting affective touch. We also examined whether interpersonal relationship, gender, and eye contact modulate these responses. As expected, caressing the partner was perceived as more pleasant than caressing a stranger, especially if the affective touch occurred together with mutual eye contact. Promoting affective touch to the partner also resulted in a decrease of both autonomic responses and anxiety levels, suggesting the occurrence of a calming effect. Additionally, these effects were more pronounced in females compared to males, indicating that hedonic and autonomic aspects of affective touch are modulated by both social relationship and gender. These findings show for the first time that caressing a beloved one is not only pleasant but also reduces autonomic responses and anxiety in the person promoting the touch. This might suggest that affective touch has an instrumental role for romantic partners in promoting and reinforcing their affective bonding.

The impact of facial expression and communicative gaze of a humanoid robot on individual Sense of Agency

Sense of Agency (SoA) is the feeling of control over one's actions and their outcomes. A well-established implicit measure of SoA is the temporal interval estimation paradigm, in which participants estimate the time interval between a voluntary action and its sensory consequence. In the present study, we aimed to investigate whether the valence of action outcome modulated implicit SoA. The valence was manipulated through interaction partner's (i) positive/negative facial expression, or (ii) type of gaze (gaze contact or averted gaze). The interaction partner was the humanoid robot iCub. In Experiment 1, participants estimated the time interval between the onset of their action (head movement towards the robot), and the robot's facial expression (happy vs. sad face). Experiment 2 was identical, but the outcome of participants' action was the type of robot's gaze (gaze contact vs. averted). In Experiment 3, we assessed-in a within-subject design-the combined effect of robot's type of facial expression and type of gaze. Results showed that, while the robot's facial expression did not affect participants' SoA (Experiment 1), the type of gaze affected SoA in both Experiment 2 and Experiment 3. Overall, our findings showed that the robot's gaze is a more potent factor than facial expression in modulating participants' implicit SoA.

Zoom disrupts eye contact behaviour: problems and solutions

Natural, dynamic eye contact behaviour is critical to social interaction but is dysfunctional in video conferencing. In analysing the problem, I introduce the concept of directionality and emphasize the critical role of motion parallax. I then sketch approaches towards re-establishing directionality and enabling natural, dynamic eye contact in video conferences.

Research on social attention in autism and the challenges of the research domain criteria (RDoC) framework

The fuzzy nature of categories of psychopathology, such as autism, leads to significant research challenges. Alternatively, focusing research on the study of a common set of important and well-defined psychological constructs across psychiatric conditions may make the fundamental etiological processes of psychopathology easier to discern and treat (Cuthbert, 2022). The development of the research domain criteria (RDoC) framework is designed to guide this new research approach (Insel et al., 2010). However, progress in research may be expected to continually refine and reorganize the understanding of the specifics of these mental processes (Cuthbert & Insel, 2013). Moreover, knowledge gleaned from the study of both normative and atypical development can be mutually informative in the evolution of our understanding of these fundamental processes. A case in point is the study of social attention. This Autism 101 commentary provides an educational summary of research over the last few decades indicates that social attention is major construct in the study of human social-cognitive development, autism and other forms of psychopathology. The commentary also describes how this research can inform the Social Process dimension of the RDoC framework.

Explicit vs. implicit spatial processing in arrow vs. eye-gaze spatial congruency effects

Arrows and gaze stimuli lead to opposite spatial congruency effects. While standard congruency effects are observed for arrows (faster responses for congruent conditions), responses are faster when eye-gaze stimuli are presented on the opposite side of the gazed-at location (incongruent trials), leading to a reversed congruency effect (RCE). Here, we explored the effects of implicit vs. explicit processing of arrows and eye-gaze direction. Participants were required to identify the direction (explicit task) or the colour (implicit task) of left or right looking/pointing gaze or arrows, presented to either the left or right of the fixation point. When participants responded to the direction of stimuli, standard congruency effects for arrows and RCE for eye-gaze stimuli were observed. However, when participants responded to the colour of stimuli, no congruency effects were observed. These results suggest that it is necessary to explicitly pay attention to the direction of eye-gaze and arrows for the congruency effect to occur. The same pattern of data was observed when participants responded either manually or verbally, demonstrating that manual motor components are not responsible for the results observed. These findings are not consistent with some hypotheses previously proposed to explain the RCE observed with eye-gaze stimuli and, therefore, call for an alternative plausible hypothesis.

Skin conductance, facial EMG, and heart rate responses in multi-person gaze interactions

Previous literature has reported enhanced affective and attentional responses to faces with a direct vs. averted gaze. Typically, in these studies, only single faces were presented. However, daily social encounters often involve interaction with more than just one person. By employing an experimental set-up in which the participants believed they were interacting with two other persons, the present study, for the first time, investigated participants' skin conductance, facial electromyographic (EMG), and heart rate deceleration responses in multi-person eye contact situations. Responses were measured in two different social contexts; i) when the participants observed eye contact between two other persons ('vicarious eye contact effect'), and ii) when the participants themselves received direct gaze either from one or two persons. The results showed that the skin conductance, facial EMG, and heart rate deceleration responses elicited by observing two other persons making eye contact did not differ from those elicited by observing one person looking at the other while the other person was not reciprocating with their gaze. As a novel finding, the results showed that receiving direct gaze from two persons elicited greater affective arousal and zygomatic EMG, but smaller heart rate deceleration responses in participants than receiving direct gaze from one person only. The findings are thoroughly discussed and it is concluded that physiological responses in multi-person interaction contexts are influenced by many social effects between the interactors and can be markedly different from those observed in two-person interactions.

Perception of eye contact, self-referential thinking and age

Increased thinking about one's self has been proposed to widen the gaze cone, that is, the range of gaze deviations that an observer judges as looking directly at them (eye contact). This study investigated the effects of a self-referential thinking manipulation and demographic factors on the gaze cone. In a preregistered experiment (N = 200), the self-referential thinking manipulation, as compared to a control manipulation, did not influence the gaze cone, or the use of first-person pronouns in a manipulation check measuring self-referential processing. This may indicate a failure of the manipulation and participants' lack of effort. However, participants' age was significantly correlated with both measures: older people had wider gaze cones and used more self-referring pronouns. A second experiment (N = 300) further examined the effect of the manipulation and demographic factors on self-referential processing, and the results were replicated. These findings may reflect age-related self-reference and positivity effects.

I looked at you, you looked at me, I smiled at you, you smiled at me—The impact of eye contact on emotional mimicry

Eye contact is an essential element of human interaction and direct eye gaze has been shown to have effects on a range of attentional and cognitive processes. Specifically, direct eye contact evokes a positive affective reaction. As such, it has been proposed that obstructed eye contact reduces emotional mimicry (i.e., the imitation of our counterpart’s emotions). So far, emotional mimicry research has used averted-gaze faces or unnaturally covered eyes (with black censor bars) to analyze the effect of eye contact on emotional mimicry. However, averted gaze can also signal disinterest/ disengagement and censor bars obscure eye-adjacent areas as well and hence impede emotion recognition. In the present study (N = 44), we used a more ecological valid approach by showing photos of actors who expressed either happiness, sadness, anger, or disgust while either wearing mirroring sunglasses that obstruct eye contact or clear glasses. The glasses covered only the direct eye region but not the brows, nose ridge, and cheeks. Our results confirm that participants were equally accurate in recognizing the emotions of their counterparts in both conditions (sunglasses vs. glasses). Further, in line with our hypotheses, participants felt closer to the targets and mimicked affiliative emotions more intensely when their counterparts wore glasses instead of sunglasses. For antagonistic emotions, we found the opposite pattern: Disgust mimicry, which was interpreted as an affective reaction rather than genuine mimicry, could be only found in the sunglasses condition. It may be that obstructed eye contact increased the negative impression of disgusted facial expressions and hence the negative feelings disgust faces evoked. The present study provides further evidence for the notion that eye contact is an important prerequisite for emotional mimicry and hence for smooth and satisfying social interactions.

A prosocial function of head-gaze aversion and head-cocking in common marmosets

Gaze aversion is a behavior adopted by several mammalian and non-mammalian species in response to eye contact, and is usually interpreted as a reaction to a perceived threat. Unlike many other primate species, common marmosets (Callithrix jacchus) are thought to have a high tolerance for direct gaze, barely exhibiting gaze avoidance towards conspecifics and humans. Here we show that this does not hold for marmosets interacting with a familiar experimenter who suddenly establishes eye contact in a playful interaction (peekaboo). Video footage synchronously recorded from the perspective of the marmoset and the experimenter showed that the monkeys consistently alternated between eye contact and head-gaze aversion, and that these responses were often preceded by head-cocking. We hypothesize that this behavioral strategy helps marmosets to temporarily disengage from emotionally overwhelming social stimulation due to sight of another individual's face, in order to prepare for a new round of affiliative face-to-face interactions.

Neural and Affective Responses to Prolonged Eye Contact with One's Own Adolescent Child and Unfamiliar Others

Eye contact is crucial for the formation and maintenance of social relationships, and plays a key role in facilitating a strong parent-child bond. However, the precise neural and affective mechanisms through which eye contact impacts on parent-child relationships remain elusive. We introduce a task to assess parents' neural and affective responses to prolonged direct and averted gaze coming from their own child, and an unfamiliar child and adult. While in the scanner, 79 parents (n = 44 mothers and n = 35 fathers) were presented with prolonged (16-38 s) videos of their own child, an unfamiliar child, an unfamiliar adult, and themselves (i.e., targets), facing the camera with a direct or an averted gaze. We measured BOLD-responses, tracked parents' eye movements during the videos, and asked them to report on their mood and feelings of connectedness with the targets after each video. Parents reported improved mood and increased feelings of connectedness after prolonged exposure to direct versus averted gaze and these effects were amplified for unfamiliar targets compared to their own child, due to high affect and connectedness ratings after videos of their own child. Neuroimaging results showed that the sight of one's own child was associated with increased activity in middle occipital gyrus, fusiform gyrus and inferior frontal gyrus relative to seeing an unfamiliar child or adult. While we found no robust evidence of specific neural correlates of eye contact (i.e., contrast direct > averted gaze), an exploratory parametric analysis showed that dorsomedial prefrontal cortex (dmPFC) activity increased linearly with duration of eye contact (collapsed across all "other" targets). Eye contact-related dmPFC activity correlated positively with increases in feelings of connectedness, suggesting that this region may drive feelings of connectedness during prolonged eye contact with others. These results underline the importance of prolonged eye contact for affiliative processes and provide first insights into its neural correlates. This may pave the way for new research in individuals or pairs in whom affiliative processes are disrupted.

The Quality of Everyday Eye Contact in Williams Syndrome: Insights From Cross-Syndrome Comparisons

Past research shows that individuals with Williams syndrome (WS) have heightened and prolonged eye contact. Using parent report measures, we examined not only the presence of eye contact but also its qualitative features. Study 1 included individuals with WS (n = 22, ages 6.0-36.3). Study 2 included children with different neurodevelopmental (ND) conditions (WS, autism spectrum condition, fragile X syndrome, attention-deficit/hyperactivity disorder) and children with neurotypical development (NT; n = 262, ages 4.0-17.11). Unusual eye contact features, including staring, were found in approximately half of the WS samples. However, other features such as brief glances were frequently found in WS and in all ND conditions, but not NT. Future research in ND conditions should focus on qualitative as well as quantitative features of eye contact.

Misleading Robot Signals in a Classification Task Induce Cognitive Load as Measured by Theta Synchronization Between Frontal and Temporo-parietal Brain Regions

As technological advances progress, we find ourselves in situations where we need to collaborate with artificial agents (e.g., robots, autonomous machines and virtual agents). For example, autonomous machines will be part of search and rescue missions, space exploration and decision aids during monitoring tasks (e.g., baggage-screening at the airport). Efficient communication in these scenarios would be crucial to interact fluently. While studies examined the positive and engaging effect of social signals (i.e., gaze communication) on human-robot interaction, little is known about the effects of conflicting robot signals on the human actor's cognitive load. Moreover, it is unclear from a social neuroergonomics perspective how different brain regions synchronize or communicate with one another to deal with the cognitive load induced by conflicting signals in social situations with robots. The present study asked if neural oscillations that correlate with conflict processing are observed between brain regions when participants view conflicting robot signals. Participants classified different objects based on their color after a robot (i.e., iCub), presented on a screen, simulated handing over the object to them. The robot proceeded to cue participants (with a head shift) to the correct or incorrect target location. Since prior work has shown that unexpected cues can interfere with oculomotor planning and induces conflict, we expected that conflicting robot social signals which would interfere with the execution of actions. Indeed, we found that conflicting social signals elicited neural correlates of cognitive conflict as measured by mid-brain theta oscillations. More importantly, we found higher coherence values between mid-frontal electrode locations and posterior occipital electrode locations in the theta-frequency band for incongruent vs. congruent cues, which suggests that theta-band synchronization between these two regions allows for communication between cognitive control systems and gaze-related attentional mechanisms. We also find correlations between coherence values and behavioral performance (Reaction Times), which are moderated by the congruency of the robot signal. In sum, the influence of irrelevant social signals during goal-oriented tasks can be indexed by behavioral, neural oscillation and brain connectivity patterns. These data provide insights about a new measure for cognitive load, which can also be used in predicting human interaction with autonomous machines.

Eye Contact in Video Communication: Experiences of Co-creating Relationships

As a result of the COVID-19 pandemic, increased number of persons have been forced to limit their interactions with friends and families to contact via video, which excludes eye-contact. The aim of this study was to examine individuals’ experiences of the difference between forced skewed visuality and the ability for eye-contact in conversations. Two custom-made units allowed 15 participants interacting in dyads to alternate between being able to make eye contact and having that ability removed through skewed visuality. Participants reported their experiences in semi-structured interviews. Data analyzed with qualitative content analysis resulted in three themes: Shared eye contact allows us to create our relationship together; With eye contact, we adjust to each other to feel more connected and less intimidated; and We get more self-conscious when the visuality is skewed or shifting. The results imply that skewed visuality as forced lack of eye-contact in video conversations effects embodied non-verbal processes related to sense of connectedness and participatory sensemaking, creating a sense of both emotional and physical distance, as well as heightening self-awareness about the need of actively regulating the other. We argue that this is one of the ways to understand the impact of moving interactions to online communication.

Dysfunctional temporal stages of eye-gaze perception in adults with ADHD: a high-density EEG study

ADHD has been associated with social cognitive impairments across the lifespan, but no studies have specifically addressed the presence of abnormalities in eye-gaze processing in the adult brain. This study investigated the neural basis of eye-gaze perception in adults with ADHD using event-related potentials (ERP). Twenty-three ADHD and 23 controls performed a delayed face-matching task with neutral faces that had either direct or averted gaze. ERPs were classified using microstate analyses. ADHD and controls displayed similar P100 and N170 microstates. ADHD was associated with cluster abnormalities in the attention-sensitive P200 to direct gaze, and in the N250 related to facial recognition. For direct gaze, source localization revealed reduced activity in ADHD for the P200 in the left/midline cerebellum, as well as in a cingulate-occipital network at the N250. These results suggest brain impairments involving eye-gaze decoding in adults with ADHD, suggestive of neural signatures associated with this disorder in adulthood.

Pupil dilation predicts modulation of direct gaze on action value calculations

Perceiving direct gaze facilitates social cognition and behaviour. We hypothesized that direct gaze modulates decision-making, particularly calculations of action values. To test our hypothesis, we used the reinforcement learning paradigm in situations with or without direct gaze. Forty adults were recruited and participated in pupil size measurements and a two-armed bandit task. The task was conducted with 70% and 30% reward probabilities for each option. During the task, a female showing the Direct Gaze (DG) or Closed Eyes (CE) condition was presented from the start of each trial. The results showed that behavioural bias to choices with 70% reward probability increased more in the DG condition than in the CE condition and the expected reward value. This bias to choices with 70% reward in the DG condition was predicted by pupil dilation to DG. These results suggest that participants over-evaluated the expected reward value in the DG condition, and this DG effect may be related to subjective expectations of rewarding events indexed by pupil dilations. It is considered that perceiving direct gaze is a driver of reward expectations that modulate action value calculations and then cognitive processing and behaviours are facilitated.

Williams syndrome: reduced orienting to other's eyes in a hypersocial phenotype

Williams syndrome (WS) is a rare genetic condition associated with high sociability, intellectual disability, and social cognitive challenges. Attention to others’ eyes is crucial for social understanding. Orienting to, and from other’s eyes was studied in WS (n = 37, mean age = 23, age range 9–53). The WS group was compared to a typically developing comparison participants (n = 167) in stratified age groups from infancy to adulthood. Typically developing children and adults were quicker and more likely to orient to eyes than the mouth. This bias was absent in WS. The WS group had reduced peak saccadic velocities, indicating hypo-arousal. The current study indicates reduced orienting to others’ eyes in WS, which may affect social interaction skills.

Emotional experiences in technology-mediated and in-person interactions: an experience-sampling study

As the ubiquity of technology-mediated communication grows, so does the number of questions about the costs and benefits of replacing in-person interactions with technology-mediated ones. In the present study, we used a daily diary design to examine how people's emotional experiences vary across in-person, video-, phone-, and text-mediated interactions in day-to-day life. We hypothesised that individuals would report less positive affect and more negative affect after less life-like interactions (where in-person is defined as the most life-like and text-mediated as the least life-like). In line with this hypothesis, the analysis of 527 unique interactions reported by 102 individuals (mean age = 19.3; 85.6% female) over the course of 7 days reveals that people feel lonelier, sadder, less affectionate, less supported, and less happy following less life-like interactions. Additional analyses show that the links between life-like communication and momentary experiences are independent of properties of individual interactions such as interaction length and participants' overall evaluations of interaction quality. These findings provide initial evidence that there may be inherent properties of common technology-mediated communication tools that may lead to momentary changes in affective experiences and make social connection more challenging.

Direct Gaze Holds Attention, but Not in Individuals with Obsessive-Compulsive Disorder

The attentional response to eye-gaze stimuli is still largely unexplored in individuals with obsessive-compulsive disorder (OCD). Here, we focused on an attentional phenomenon according to which a direct-gaze face can hold attention in a perceiver. Individuals with OCD and a group of matched healthy controls were asked to discriminate, through a speeded manual response, a peripheral target. Meanwhile, a task-irrelevant face displaying either direct gaze (in the eye-contact condition) or averted gaze (in the no-eye-contact condition) was also presented at the centre of the screen. Overall, the latencies were slower for faces with direct gaze than for faces with averted gaze; however, this difference was reliable in the healthy control group but not in the OCD group. This suggests the presence of an unusual attentional response to direct gaze in this clinical population.

The Bidirectional Social-Cognitive Mechanisms of the Social-Attention Symptoms of Autism

Differences in social attention development begin to be apparent in the 6th to 12th month of development in children with Autism Spectrum Disorder (ASD) and theoretically reflect important elements of its neurodevelopmental endophenotype. This paper examines alternative conceptual views of these early social attention symptoms and hypotheses about the mechanisms involved in their development. One model emphasizes mechanism involved in the spontaneous allocation of attention to faces, or social orienting. Alternatively, another model emphasizes mechanisms involved in the coordination of attention with other people, or joint attention, and the socially bi-directional nature of its development. This model raises the possibility that atypical responses of children to the attention or the gaze of a social partner directed toward themselves may be as important in the development of social attention symptoms as differences in the development of social orienting. Another model holds that symptoms of social attention may be important to early development, but may not impact older individuals with ASD. The alterative model is that the social attention symptoms in infancy (social orienting and joint attention), and social cognitive symptoms in childhood and adulthood share common neurodevelopmental substrates. Therefore, differences in early social attention and later social cognition constitute a developmentally continuous axis of symptom presentation in ASD. However, symptoms in older individuals may be best measured with in vivo measures of efficiency of social attention and social cognition in social interactions rather than the accuracy of response on analog tests used in measures with younger children. Finally, a third model suggests that the social attention symptoms may not truly be a symptom of ASD. Rather, they may be best conceptualized as stemming from differences domain general attention and motivation mechanisms. The alternative argued for here that infant social attention symptoms meet all the criteria of a unique dimension of the phenotype of ASD and the bi-directional phenomena involved in social attention cannot be fully explained in terms of domain general aspects of attention development.

Attention to live eye contact in adolescents with autism spectrum disorder

A number of studies have reported diminished attention to the eyes in individuals with autism spectrum disorder (ASD). These studies predominantly used static images of faces as stimuli. Recent studies, however, have shown enhanced response to eye contact in typically developing (TD) individuals when they observe a person in a live interaction. We investigated physiological orienting to perceived eye contact in adolescents with ASD and TD adolescents when they observed a person in live interaction or viewed a photograph of the same person's face. We measured heart rate (HR) deceleration as an index of attentional orienting. Adolescents with ASD, as well as TD adolescents, showed significant HR deceleration for the direct gaze compared to an averted gaze in the live condition, but not in the photographic condition. The results suggest an intact response to perceived eye contact in individuals with ASD during a live face-to-face interaction. LAY SUMMARY: Individuals with autism spectrum disorder (ASD) have a different eye gaze pattern when observing photographic faces. However, little is known about how individuals with ASD process a real person's face. We measured heart rate (HR) and found that adolescents with ASD showed the typical decline in HR when they made eye contact with a real person, which suggests that both groups of individuals directed their attention to eye contact in a live face-to-face interaction.

Avoidance behaviour generalizes to eye processing in posttraumatic stress disorder

BACKGROUND

Avoidance describes any action designed to prevent an uncomfortable situation or emotion from occurring. Although it is a common reaction to trauma, avoidance becomes problematic when it is the primary coping strategy, and plays a major role in the development and maintenance of posttraumatic stress disorder (PTSD). Avoidance in PTSD may generalize to non-harmful environmental cues that are perceived to be unsafe.

OBJECTIVE

We tested whether avoidance extends to social cues (i.e. emotional gazes) that are unrelated to trauma.

METHOD

A total of 159 participants (103 who had been exposed to the 2015 Paris terrorist attacks and 56 who had not) performed a gaze-cueing task featuring sad, happy and neutral faces. Attention to the eye area was recorded using an eyetracker. Of the exposed participants, 52 had been diagnosed with PTSD (PTSD+) and 51 had not developed PTSD (PTSD-). As a result of the preprocessing stages, 52 PTSD+ (29 women), 50 PTSD- (20 women) and 53 nonexposed (31 women) participants were included in the final analyses.

RESULTS

PTSD+ participants looked at sad eyes for significantly less time than PTSD- and nonexposed individuals. This effect was negatively correlated with the intensity of avoidance symptoms. No difference was found for neutral and happy faces.

CONCLUSIONS

These findings suggest that maladaptive avoidance in PTSD extends to social processing, in terms of eye contact and others' emotions that are unrelated to trauma. New therapeutic directions could include targeting sociocognitive deficits. Our findings open up new and indirect avenues for overcoming maladaptive avoidance behaviours by remediating eye processing.Trial registration: ClinicalTrials.gov identifier: NCT02810197.

HIGHLIGHTS

Avoidance is a key symptom of posttraumatic stress disorder (PTSD).Avoidance is often viewed as limited to reminders linked to the trauma.Results show that attention to the eyes of sad faces is also affected by PTSD. This effect is correlated with avoidance symptoms in PTSD.

Irrelevant Robot Signals in a Categorization Task Induce Cognitive Conflict in Performance, Eye Trajectories, the N2 ERP-EEG Component, and Frontal Theta Oscillations

Understanding others' nonverbal behavior is essential for social interaction, as it allows, among others, to infer mental states. Although gaze communication, a well-established nonverbal social behavior, has shown its importance in inferring others' mental states, not much is known about the effects of irrelevant gaze signals on cognitive conflict markers during collaborative settings. Here, participants completed a categorization task where they categorized objects based on their color while observing images of a robot. On each trial, participants observed the robot iCub grasping an object from a table and offering it to them to simulate a handover. Once the robot "moved" the object forward, participants were asked to categorize the object according to its color. Before participants were allowed to respond, the robot made a lateral head/gaze shift. The gaze shifts were either congruent or incongruent with the object's color. We expected that incongruent head cues would induce more errors (Study 1), would be associated with more curvature in eye-tracking trajectories (Study 2), and induce larger amplitude in electrophysiological markers of cognitive conflict (Study 3). Results of the three studies show more oculomotor interference as measured in error rates (Study 1), larger curvatures eye-tracking trajectories (Study 2), and higher amplitudes of the N2 ERP of the EEG signals as well as higher event-related spectral perturbation amplitudes (Study 3) for incongruent trials compared with congruent trials. Our findings reveal that behavioral, ocular, and electrophysiological markers can index the influence of irrelevant signals during goal-oriented tasks.

Manipulating Sense of Participation in Multipartite Conversations by Manipulating Head Attitude and Gaze Direction

Interpersonal communication is so important in everyday life that it is desirable everyone who participates in the conversation is satisfied. However, every participant of the conversation cannot be satisfied in such cases as those wherein only one person cannot keep up with the conversation and feels alienated, or wherein someone cannot communicate non-verbal expressions with his/her conversation partner adequately. In this study, we have focused on facial direction and gaze among the various factors that are said to affect conversational satisfaction. We have attempted to lessen any sense of non-participation in the conversation and increase the conversational satisfaction of the non-participant in a tripartite conversation by modulating the visual information in such a way that the remaining two parties turn toward the non-participating party. In the experiments we have conducted in VR environments, we have reproduced a conversation of two male adults recorded in actual environments using two avatars. The experimental subjects have watched this over their HMDs. The experiments have found that visually modulating the avatars’ faces and gazes such that they appear to turn toward the subjects has increased the subjects’ sense of participation in the conversation. Nevertheless, the experiments have not increased the subjects’ conversational enjoyment, a component of the factors for conversational satisfaction.

Eye Direction Detection and Perception as Premises of a Social Brain: A Narrative Review of Behavioral and Neural Data

The eyes and the gaze are important stimuli for social interaction in humans. Impaired recognition of facial identity, facial emotions, and inference of the intentions of others may result from difficulties in extracting information relevant to the eye region, mainly the direction of gaze. Therefore, a review of these data is of interest. Behavioral data demonstrating the importance of the eye region and how humans respond to gaze direction are reviewed narratively, and several theoretical models on how visual information on gaze is processed are discussed to propose a unified hypothesis. Several issues that have not yet been investigated are identified. The authors tentatively suggest experiments that might help progress research in this area. The neural aspects are subsequently reviewed to best describe the low-level and higher-level visual information processing stages in the targeted subcortical and cortical areas. A specific neural network is proposed on the basis of the literature. Various gray areas, such as the temporality of the processing of visual information, the question of salience priority, and the coordination between the two hemispheres, remain unclear and require further investigations. Finally, disordered gaze direction detection mechanisms and their consequences on social cognition and behavior are discussed as key deficiencies in several conditions, such as autism spectrum disorder, 22q11.2 deletion, schizophrenia, and social anxiety disorder. This narrative review provides significant additional data showing that the detection and perception of someone's gaze is an essential part of the development of our social brain.

How Neuroimaging Can Aid the Interpretation of Art

Cognitive neuroscience of art continues to be criticized for failing to provide interesting results about art itself. In particular, results of brain imaging experiments have not yet been utilized in interpretation of particular works of art. Here we revisit a recent study in which we explored the neuronal and behavioral response to painted portraits with a direct versus an averted gaze. We then demonstrate how fMRI results can be related to the art historical interpretation of a specific painting. The evidentiary status of neuroimaging data is not different from any other extra-pictorial facts that art historians uncover in their research and relate to their account of the significance of a work of art. They are not explanatory in a strong sense, yet they provide supportive evidence for the art writer’s inference about the intended meaning of a given work. We thus argue that brain imaging can assume an important role in the interpretation of particular art works.

Hidden intentions: Visual awareness prioritizes perceived attention even without eyes or faces

Eye contact is a powerful social signal, and it readily captures attention. Recent work has suggested that direct gaze is prioritized even unconsciously: faces rendered invisible via interocular suppression enter awareness faster when they look directly at (vs. away from) you. Such effects may be driven in a relatively low level way by the special visual properties of eyes, per se, but here we asked whether they might instead arise from the perception of a deeper property: being the focus of another agent's attention and/or intentions. We report five experiments which collectively explore whether visual awareness also prioritizes distinctly non-eye-like stimuli that nevertheless convey directedness. We first showed that directed (vs. averted) 'mouth' shapes also break through into awareness faster, after being rendered invisible by continuous flash suppression - a direct 'gaze' effect without any eyes. But such effects could still be specific to faces (if not eyes), so we next asked whether the prioritization of directed intentions would still occur even for stimuli that have no faces at all. In fact, even simple geometric shapes can be seen as intentional, as when numerous randomly scattered cones are all consistently pointing at you. And indeed, even such directed (vs. averted) cones entered awareness faster - a direct 'gaze' effect without any facial cues. Additional control experiments ruled out effects of both symmetry and response biases. We conclude that the perception of directed intentions is sufficient to boost objects into awareness, and that putative eye-contact effects might instead reflect more general phenomena of 'mind contact'.

Mutual gaze with a robot affects human neural activity and delays decision-making processes

In most everyday life situations, the brain needs to engage not only in making decisions but also in anticipating and predicting the behavior of others. In such contexts, gaze can be highly informative about others’ intentions, goals, and upcoming decisions. Here, we investigated whether a humanoid robot’s gaze (mutual or averted) influences the way people strategically reason in a social decision-making context. Specifically, participants played a strategic game with the robot iCub while we measured their behavior and neural activity by means of electroencephalography (EEG). Participants were slower to respond when iCub established mutual gaze before their decision, relative to averted gaze. This was associated with a higher decision threshold in the drift diffusion model and accompanied by more synchronized EEG alpha activity. In addition, we found that participants reasoned about the robot’s actions in both conditions. However, those who mostly experienced the averted gaze were more likely to adopt a self-oriented strategy, and their neural activity showed higher sensitivity to outcomes. Together, these findings suggest that robot gaze acts as a strong social signal for humans, modulating response times, decision threshold, neural synchronization, as well as choice strategies and sensitivity to outcomes. This has strong implications for all contexts involving human-robot interaction, from robotics to clinical applications.

I can see it in your eyes: Perceived gaze direction impacts ERP and behavioural measures of affective theory of mind

Looking at someone's eyes is thought to be important for affective theory of mind (aTOM), our ability to infer their emotional state. However, it is unknown whether an individual's gaze direction influences our aTOM judgements and what the time course of this influence might be. We presented participants with sentences describing individuals in positive, negative or neutral scenarios, followed by direct or averted gaze neutral face pictures of those individuals. Participants made aTOM judgements about each person's mental state, including their affective valence and arousal, and we investigated whether the face gaze direction impacted those judgements. Participants rated that gazers were feeling more positive when they displayed direct gaze as opposed to averted gaze, and that they were feeling more aroused during negative contexts when gaze was averted as opposed to direct. Event-related potentials associated with face perception and affective processing were examined using mass-univariate analyses to track the time-course of this eye-gaze and affective processing interaction at a neural level. Both positive and negative trials were differentiated from neutral trials at many stages of processing. This included the early N200 and EPN components, believed to reflect automatic emotion areas activation and attentional selection respectively. This also included the later P300 and LPP components, thought to reflect elaborative cognitive appraisal of emotional content. Critically, sentence valence and gaze direction interacted over these later components, which may reflect the incorporation of eye-gaze in the cognitive evaluation of another's emotional state. The results suggest that gaze perception directly impacts aTOM processes, and that altered eye-gaze processing in clinical populations may contribute to associated aTOM impairments.