Eye contact with neutral and smiling faces: effects on autonomic responses and frontal EEG asymmetry

The neural correlates of reinforcement sensitivity theory: A systematic review of the frontal asymmetry and spectral power literature

The original Reinforcement Sensitivity Theory (oRST) proposes two systems of approach (BAS) and avoidance (BIS) motivation to underpin personality and behavior. The revised-RST (rRST) model separates avoidance motivation into passive (BIS; anxiety) and active (FFFS; fear) systems. Prior research has attempted to map RST onto lateralized frontal asymmetry to provide a neurophysiological marker of RST. The main aim is to examine the relationships of the o/rRST scales with trait (baseline) and state (manipulated through experimental paradigms) frontal asymmetry. A systematic review was conducted, resulting in 158 studies designated to neuroimaging research. In total, 54 studies were included in this review using either frontal asymmetry or spectral power. The results were split into three main categories: resting frontal alpha asymmetry (N = 23), emotional induction and state-related frontal alpha asymmetry (N = 20), and spectral analysis (N = 16). Findings indicated that BAS was associated with enhanced left frontal asymmetry at baseline and during state-related paradigms. Findings for BIS were more inconsistent, especially at rest, suggesting that BIS, in particular, may require active engagement with the environment. Only 9 of the 54 papers included used the revised RST model, highlighting the need for more rRST research.

Alpha-band oscillations and emotion: A review of studies on picture perception

Although alpha-band activity has long been a focus of psychophysiological research, its modulation by emotional value during picture perception has only recently been studied systematically. Here, we review these studies and report that the most consistent alpha oscillatory pattern indexing emotional processing is an enhanced desynchronization (ERD) over posterior sensors when viewing emotional compared with neutral pictures. This enhanced alpha ERD is not specific to unpleasant picture content, as previously proposed for other measures of affective response, but has also been observed for pleasant stimuli. Evidence suggests that this effect is not confined to the alpha band but that it also involves a desynchronization of the lower beta frequencies (8-20 Hz). The emotional modulation of alpha ERD occurs even after massive stimulus repetition and when emotional cues serve as task-irrelevant distractors, consistent with the hypothesis that evaluative processes are mandatory in emotional picture processing. A similar enhanced ERD has been observed for other significant cues (e.g., conditioned aversive stimuli, or in anticipation of a potential threat), suggesting that it reflects cortical excitability associated with the engagement of the motivational systems.

Psychophysiological responses to eye contact with a humanoid robot: Impact of perceived intentionality

Eye contact with a social robot has been shown to elicit similar psychophysiological responses to eye contact with another human. However, it is becoming increasingly clear that the attention- and affect-related psychophysiological responses differentiate between direct (toward the observer) and averted gaze mainly when viewing embodied faces that are capable of social interaction, whereas pictorial or pre-recorded stimuli have no such capability. It has been suggested that genuine eye contact, as indicated by the differential psychophysiological responses to direct and averted gaze, requires a feeling of being watched by another mind. Therefore, we measured event-related potentials (N170 and frontal P300) with EEG, facial electromyography, skin conductance, and heart rate deceleration responses to seeing a humanoid robot's direct versus averted gaze, while manipulating the impression of the robot's intentionality. The results showed that the N170 and the facial zygomatic responses were greater to direct than to averted gaze of the robot, and independent of the robot's intentionality, whereas the frontal P300 responses were more positive to direct than to averted gaze only when the robot appeared intentional. The study provides further evidence that the gaze behavior of a social robot elicits attentional and affective responses and adds that the robot's seemingly autonomous social behavior plays an important role in eliciting higher-level socio-cognitive processing.

A Review on the Role of Affective Stimuli in Event-Related Frontal Alpha Asymmetry

Frontal alpha asymmetry refers to the difference between the right and left alpha activity over the frontal brain region. Increased activity in the left hemisphere has been linked to approach motivation and increased activity in the right hemisphere has been linked to avoidance or withdrawal. However, research on alpha asymmetry is diverse and has shown mixed results, which may partly be explained by the potency of the used stimuli to emotionally and motivationally engage participants. This review gives an overview of the types of affective stimuli utilized with the aim to identify which stimuli elicit a strong approach-avoidance effect in an affective context. We hope this contributes to better understanding of what is reflected by alpha asymmetry, and in what circumstances it may be an informative marker of emotional state. We systematically searched the literature for studies exploring event-related frontal alpha asymmetry in affective contexts. The search resulted in 61 papers, which were categorized in five stimulus categories that were expected to differ in their potency to engage participants: images & sounds, videos, real cues, games and other tasks. Studies were viewed with respect to the potency of the stimuli to evoke significant approach-avoidance effects on their own and in interaction with participant characteristics or condition. As expected, passively perceived stimuli that are multimodal or realistic, seem more potent to elicit alpha asymmetry than unimodal stimuli. Games, and other stimuli with a strong task-based component were expected to be relatively engaging but approach-avoidance effects did not seem to be much clearer than the studies using perception of videos and real cues. While multiple factors besides stimulus characteristics determine alpha asymmetry, and we did not identify a type of affective stimulus that induces alpha asymmetry highly consistently, our results indicate that strongly engaging, salient and/or personally relevant stimuli are important to induce an approach-avoidance effect.

Are You “Gazing” at Me? How Others' Gaze Direction and Facial Expression Influence Gaze Perception and Postural Control

In everyday life, interactions between humans are generally modulated by the value attributed to the situation, which partly relies on the partner's behavior. A pleasant or cooperating partner may trigger an approach behavior in the observer, while an unpleasant or threatening partner may trigger an avoidance behavior. In this context, the correct interpretation of other's intentions is crucial to achieve satisfying social interactions. Social cues such as gaze direction and facial expression are both fundamental and interrelated. Typically, whenever gaze direction and facial expression of others communicate the same intention, it enhances both the interlocutor's gaze direction and the perception of facial expressions (i.e., shared signal hypothesis). For instance, an angry face with a direct gaze is perceived as more intense since it represents a threat to the observer. In this study, we propose to examine how the combination of others' gaze direction (direct or deviated) and emotional facial expressions (i.e., happiness, fear, anger, sadness, disgust, and neutrality) influence the observer's gaze perception and postural control. Gaze perception was indexed by the cone of direct gaze (CoDG) referring to the width over which an observer feels someone's gaze is directed at them. A wider CoDG indicates that the observer perceived the face as looking at them over a wider range of gaze directions. Conversely, a narrower CoDG indicates a decrease in the range of gaze directions perceived as direct. Postural control was examined through the center of pressure displacements reflecting postural stability and approach-avoidance tendencies. We also investigated how both gaze perception and postural control may vary according to participants' personality traits and emotional states (e.g., openness, anxiety, etc.). Our results confirmed that gaze perception is influenced by emotional faces: a wider CoDGs was observed with angry and disgusted faces while a narrower CoDG was observed for fearful faces. Furthermore, facial expressions combined with gaze direction influence participants' postural stability but not approach-avoidance behaviors. Results are discussed in the light of the approach-avoidance model, by considering how some personality traits modulate the relation between emotion and posture.

The Effects of Dynamic and Static Emotional Facial Expressions of Humans and Their Avatars on the EEG: An ERP and ERD/ERS Study

This study aimed to examine whether the cortical processing of emotional faces is modulated by the computerization of face stimuli ("avatars") in a group of 25 healthy participants. Subjects were passively viewing 128 static and dynamic facial expressions of female and male actors and their respective avatars in neutral or fearful conditions. Event-related potentials (ERPs), as well as alpha and theta event-related synchronization and desynchronization (ERD/ERS), were derived from the EEG that was recorded during the task. All ERP features, except for the very early N100, differed in their response to avatar and actor faces. Whereas the N170 showed differences only for the neutral avatar condition, later potentials (N300 and LPP) differed in both emotional conditions (neutral and fear) and the presented agents (actor and avatar). In addition, we found that the avatar faces elicited significantly stronger reactions than the actor face for theta and alpha oscillations. Especially theta EEG frequencies responded specifically to visual emotional stimulation and were revealed to be sensitive to the emotional content of the face, whereas alpha frequency was modulated by all the stimulus types. We can conclude that the computerized avatar faces affect both, ERP components and ERD/ERS and evoke neural effects that are different from the ones elicited by real faces. This was true, although the avatars were replicas of the human faces and contained similar characteristics in their expression.

A New Neurorehabilitative Postsurgery Intervention for Facial Palsy Based on Smile Observation and Hand-Mouth Motor Synergies

Objective

To perform a preliminary test of a new rehabilitation treatment (FIT-SAT), based on mirror mechanisms, for gracile muscles after smile surgery.

Method

A pre- and postsurgery longitudinal design was adopted to study the efficacy of FIT-SAT. Four patients with bilateral facial nerve paralysis (Moebius syndrome) were included. They underwent two surgeries with free muscle transfers, one year apart from each other. The side of the face first operated on was rehabilitated with the traditional treatment, while the second side was rehabilitated with FIT-SAT. The FIT-SAT treatment includes video clips of an actor performing a unilateral or a bilateral smile to be imitated (FIT condition). In addition to this, while smiling, the participants close their hand in order to exploit the overlapped cortical motor representation of the hand and the mouth, which may facilitate the synergistic activity of the two effectors during the early phases of recruitment of the transplanted muscles (SAT). The treatment was also aimed at avoiding undesired movements such as teeth grinding. Discussion. Results support FIT-SAT as a viable alternative for smile rehabilitation after free muscle transfer. We propose that the treatment potentiates the effect of smile observation by activating the same neural structures responsible for the execution of the smile and therefore by facilitating its production. Closing of the hand induces cortical recruitment of hand motor neurons, recruiting the transplanted muscles, and reducing the risk of associating other unwanted movements such as teeth clenching to the smile movements.

A Review of Hyperscanning and Its Use in Virtual Environments

Hyperscanning is a technique which simultaneously records the neural activity of two or more people. This is done using one of several neuroimaging methods, such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The use of hyperscanning has seen a dramatic rise in recent years to monitor social interactions between two or more people. Similarly, there has been an increase in the use of virtual reality (VR) for collaboration, and an increase in the frequency of social interactions being carried out in virtual environments (VE). In light of this, it is important to understand how interactions function within VEs, and how they can be enhanced to improve their quality in a VE. In this paper, we present some of the work that has been undertaken in the field of social neuroscience, with a special emphasis on hyperscanning. We also cover the literature detailing the work that has been carried out in the human–computer interaction domain that addresses remote collaboration. Finally, we present a way forward where these two research domains can be combined to explore how monitoring the neural activity of a group of participants in VE could enhance collaboration among them.

The Measurement of Eye Contact in Human Interactions: A Scoping Review

Eye contact is a fundamental aspect of nonverbal communication and therefore important for understanding human interaction. Eye contact has been the subject of research in many disciplines, including communication sciences, social psychology, and psychiatry, and a variety of techniques have been used to measure it. The choice of measurement method has consequences for research outcomes and their interpretation. To ensure that research findings align with study aims and populations, it is essential that methodological choices are well substantiated. Therefore, to enhance the effective examination of eye contact, we performed a literature review of the methods used to study eye contact. We searched Medline, PsycINFO and Web of Science for empirical peer-reviewed articles published in English that described quantitative studies on human eye contact and included a methodological description. The identified studies (N = 109) used two approaches to assess eye contact: direct, i.e., assessing eye contact while it is occurring, and indirect, i.e., assessing eye contact retrospectively (e.g., from video recordings). Within these categories, eight specific techniques were distinguished. Variation was found regarding the reciprocity of eye contact between two individuals, the involvement of an assessor and the behavior of participants while being studied. Measures not involving the interactors in assessment of eye contact and have a higher spatial and temporal resolution, such as eye tracking, have gained popularity. Our results show wide methodological diversity regarding the measurement of eye contact. Although studies often define eye contact as gaze towards an exact location, this may not do justice to the subjective character of eye contact. The various methodologies have hardly ever been compared, limiting the ability to compare findings between studies. Future studies should take notice of the controversy surrounding eye contact measures.

The effect of being watched on facial EMG and autonomic activity in response to another individual's facial expressions

We tested if facial reactions to another person's facial expressions depend on the self-relevance of the observed expressions. In the present study (n = 44), we measured facial electromyographic (zygomatic and corrugator) activity and autonomic arousal (skin conductance) responses to a live model's smiling and neutral faces. In one condition, the participant and the model were able to see each other normally, whereas in the other condition, the participant was led to believe that the model could not see the participant. The results showed that the increment of zygomatic activity in response to smiling faces versus neutral faces was greater when the participants believed they were being watched than it was when the participants believed they were not being watched. However, zygomatic responses to smiles did not differ between the conditions, while the results suggested that the participants' zygomatic responses to neutral faces seemed to attenuate in the condition of believing they were being watched. Autonomic responses to smiling faces were greater in the belief of being watched than in the belief of not being watched condition. The results suggest that the self-relevance of another individual's facial expression modulates autonomic arousal responses and to a lesser extent facial EMG responses.

Hemispheric asymmetries and emotions: Evidence from effective connectivity

The Right Hemisphere Hypothesis (RHH) posits that the right hemisphere is specialized in processing all emotions; the Valence Hypothesis (VH) suggests a left/right-hemispheric specialization for positive/negative emotions, respectively. Behavioural, neuroimaging and physiological investigations alternatively support either the RHH or the VH, but connectivity analyses have been hardly exploited in this field. In the present study, unilateral and bilateral presentations of positive (happy) and negative (angry) emotional faces were used during electroencephalographic (EEG) recordings, and estimation of effective connectivity was performed using the Directed Transfer Function, to estimate causal influences between brain regions (Granger causality approach). The results show a strong pattern of connectivity among different frontal areas (orbitofrontal and dorsolateral prefrontal cortex), attentional network (frontal eye field, intraparietal sulcus), visual occipital areas and temporal sites, mainly lateralized in the right hemisphere for all emotions, in accordance with the RHH. Moreover, a stronger pattern of connectivity is evident when stimuli are presented in accordance with the VH (positive/negative emotions to the left/right hemisphere, respectively). Finally, the results suggest a crucial role of the right dorsolateral prefrontal cortex in a top-down regulation toward different areas involved in emotional processing. We conclude that the RHH and the VH are not mutually exclusive, but they seem to coexist during affective perception.

Placebo treatment facilitates social trust and approach behavior

Placebo effect refers to beneficial changes induced by the use of inert treatment, such as placebo-induced relief of physical pain and attenuation of negative affect. To date, we know little about whether placebo treatment could facilitate social functioning, a crucial aspect for well-being of a social species. In the present study, we develop and validate a paradigm to induce placebo effects on social trust and approach behavior (social placebo effect), and show robust evidence that placebo treatment promotes trust in others and increases preference for a closer interpersonal distance. We further examine placebo effects in real-life social interaction and show that placebo treatment makes single, but not pair-bonded, males keep closer to an attractive first-met female and perceive less social anxiety in the female. Finally, we show evidence that the effects of placebo treatment on social trust and approach behavior can be as strong as the effect of intranasal administration of oxytocin, a neuropeptide known for its function in facilitating social cognition and behavior. The finding of the social placebo effect extends our understanding of placebo effects on improvement of physical, mental, and social well-being and suggests clinical potentials in the treatment of social dysfunction.

Frontal temporal and parietal systems synchronize within and across brains during live eye-to-eye contact

Human eye-to-eye contact is a primary source of social cues and communication. In spite of the biological significance of this interpersonal interaction, the underlying neural processes are not well-understood. This knowledge gap, in part, reflects limitations of conventional neuroimaging methods, including solitary confinement in the bore of a scanner and minimal tolerance of head movement that constrain investigations of natural, two-person interactions. However, these limitations are substantially resolved by recent technical developments in functional near-infrared spectroscopy (fNIRS), a non-invasive spectral absorbance technique that detects changes in blood oxygen levels in the brain by using surface-mounted optical sensors. Functional NIRS is tolerant of limited head motion and enables simultaneous acquisitions of neural signals from two interacting partners in natural conditions. We employ fNIRS to advance a data-driven theoretical framework for two-person neuroscience motivated by the Interactive Brain Hypothesis which proposes that interpersonal interaction between individuals evokes neural mechanisms not engaged during solo, non-interactive, behaviors. Within this context, two specific hypotheses related to eye-to-eye contact, functional specificity and functional synchrony, were tested. The functional specificity hypothesis proposes that eye-to-eye contact engages specialized, within-brain, neural systems; and the functional synchrony hypothesis proposes that eye-to-eye contact engages specialized, across-brain, neural processors that are synchronized between dyads. Signals acquired during eye-to-eye contact between partners (interactive condition) were compared to signals acquired during mutual gaze at the eyes of a picture-face (non-interactive condition). In accordance with the specificity hypothesis, responses during eye-to-eye contact were greater than eye-to-picture gaze for a left frontal cluster that included pars opercularis (associated with canonical language production functions known as Broca's region), pre- and supplementary motor cortices (associated with articulatory systems), as well as the subcentral area. This frontal cluster was also functionally connected to a cluster located in the left superior temporal gyrus (associated with canonical language receptive functions known as Wernicke's region), primary somatosensory cortex, and the subcentral area. In accordance with the functional synchrony hypothesis, cross-brain coherence during eye-to-eye contact relative to eye-to-picture gaze increased for signals originating within left superior temporal, middle temporal, and supramarginal gyri as well as the pre- and supplementary motor cortices of both interacting brains. These synchronous cross-brain regions are also associated with known language functions, and were partner-specific (i.e., disappeared with randomly assigned partners). Together, both within and across-brain neural correlates of eye-to-eye contact included components of previously established productive and receptive language systems. These findings reveal a left frontal, temporal, and parietal long-range network that mediates neural responses during eye-to-eye contact between dyads, and advance insight into elemental mechanisms of social and interpersonal interactions.

Affective pictures processing is reflected by an increased long-distance EEG connectivity

Analysis of affective picture processing by means of EEG has invaded the literature. The methodology of event-related EEG coherence is one of the essential methods used to analyze functional connectivity. The aims of the present study are to find out the long range EEG connectivity changes in perception of different affective pictures and analyze gender differences in these long range connected networks. EEGs of 28 healthy subjects (14 female) were recorded at 32 locations. The participants passively viewed emotional pictures (IAPS, unpleasant, pleasant, neutral). The long-distance intra-hemispheric event-related coherence was analyzed for delta (1-3.5 Hz), theta (4-7.5 Hz), and alpha (8-13 Hz) frequency ranges for F₃-T₇, F₄-T₈, F₃-TP₇, F₄-TP₈, F₃-P₃, F₄-P₄, F₃-O₁, F₄-O₂, C₃-O₁, C₄-O₂ electrode pairs. Unpleasant pictures elicited significantly higher delta coherence values than neutral pictures (p < 0.05), over fronto-parietal, fronto-occipital, and centro-occipital electrode pairs. Furthermore, unpleasant pictures elicited higher theta coherence values than pleasant (p < 0.05) and neutral pictures (p < 0.05). The present study showed that female subjects had higher delta (p < 0.05) and theta (p < 0.05) coherence values than male subjects. This difference was observed more for emotional pictures than for neutral pictures. This study showed that the brain connectivity was higher during emotional pictures than neutral pictures. Females had higher connectivity between different parts of the brain than males during emotional processes. According to these results, we may comment that increased valence and arousal caused increased brain activity. It seems that not just single sources but functional networks were also activated during perception of emotional pictures.

Gazing at me: the importance of social meaning in understanding direct-gaze cues

Direct gaze is an engaging and important social cue, but the meaning of direct gaze depends heavily on the surrounding context. This paper reviews some recent studies of direct gaze, to understand more about what neural and cognitive systems are engaged by this social cue and why. The data show that gaze can act as an arousal cue and can modulate actions, and can activate brain regions linked to theory of mind and self-related processing. However, all these results are strongly modulated by the social meaning of a gaze cue and by whether participants believe that another person is really watching them. The implications of these contextual effects and audience effects for our theories of gaze are considered.

The duality of gaze: eyes extract and signal social information during sustained cooperative and competitive dyadic gaze

In contrast to non-human primate eyes, which have a dark sclera surrounding a dark iris, human eyes have a white sclera that surrounds a dark iris. This high contrast morphology allows humans to determine quickly and easily where others are looking and infer what they are attending to. In recent years an enormous body of work has used photos and schematic images of faces to study these aspects of social attention, e.g., the selection of the eyes of others and the shift of attention to where those eyes are directed. However, evolutionary theory holds that humans did not develop a high contrast morphology simply to use the eyes of others as attentional cues; rather they sacrificed camouflage for communication, that is, to signal their thoughts and intentions to others. In the present study we demonstrate the importance of this by taking as our starting point the hypothesis that a cornerstone of non-verbal communication is the eye contact between individuals and the time that it is held. In a single simple study we show experimentally that the effect of eye contact can be quickly and profoundly altered merely by having participants, who had never met before, play a game in a cooperative or competitive manner. After the game participants were asked to make eye contact for a prolonged period of time (10 min). Those who had played the game cooperatively found this terribly difficult to do, repeatedly talking and breaking gaze. In contrast, those who had played the game competitively were able to stare quietly at each other for a sustained period. Collectively these data demonstrate that when looking at the eyes of a real person one both acquires and signals information to the other person. This duality of gaze is critical to non-verbal communication, with the nature of that communication shaped by the relationship between individuals, e.g., cooperative or competitive.

Perceiving where another person is looking: the integration of head and body information in estimating another person's gaze

The process through which an observer allocates his/her attention based on the attention of another person is known as joint attention. To be able to do this, the observer effectively has to compute where the other person is looking. It has been shown that observers integrate information from the head and the eyes to determine the gaze of another person. Most studies have documented that observers show a bias called the overshoot effect when eyes and head are misaligned. That is, when the head is not oriented straight to the observer, perceived gaze direction is sometimes shifted in the direction opposite to the head turn. The present study addresses whether body information is also used as a cue to compute perceived gaze direction. In Experiment 1, we observed a similar overshoot effect in both behavioral and saccadic responses when manipulating body orientation. In Experiment 2, we explored whether the overshoot effect was due to observers assuming that the eyes are oriented further than the head when head and body orientation are misaligned. We removed horizontal eye information by presenting the stimulus from a side view. Head orientation was now manipulated in a vertical direction and the overshoot effect was replicated. In summary, this study shows that body orientation is indeed used as a cue to determine where another person is looking.

More than a face: a unified theoretical perspective on nonverbal social cue processing in social anxiety

Processing of nonverbal social cues (NVSCs) is essential to interpersonal functioning and is particularly relevant to models of social anxiety. This article provides a review of the literature on NVSC processing from the perspective of social rank and affiliation biobehavioral systems (ABSs), based on functional analysis of human sociality. We examine the potential of this framework for integrating cognitive, interpersonal, and evolutionary accounts of social anxiety. We argue that NVSCs are uniquely suited to rapid and effective conveyance of emotional, motivational, and trait information and that various channels are differentially effective in transmitting such information. First, we review studies on perception of NVSCs through face, voice, and body. We begin with studies that utilized information processing or imaging paradigms to assess NVSC perception. This research demonstrated that social anxiety is associated with biased attention to, and interpretation of, emotional facial expressions (EFEs) and emotional prosody. Findings regarding body and posture remain scarce. Next, we review studies on NVSC expression, which pinpointed links between social anxiety and disturbances in eye gaze, facial expressivity, and vocal properties of spontaneous and planned speech. Again, links between social anxiety and posture were understudied. Although cognitive, interpersonal, and evolutionary theories have described different pathways to social anxiety, all three models focus on interrelations among cognition, subjective experience, and social behavior. NVSC processing and production comprise the juncture where these theories intersect. In light of the conceptualizations emerging from the review, we highlight several directions for future research including focus on NVSCs as indexing reactions to changes in belongingness and social rank, the moderating role of gender, and the therapeutic opportunities offered by embodied cognition to treat social anxiety.

What does Williams syndrome reveal about the determinants of social behavior?

Growing evidence on autonomic nervous system (ANS) function in individuals with Williams syndrome (WS) has begun to highlight aberrancies that may have important implications for the social profile characterized by enhanced social motivation and approach. In parallel, neurobiological investigations have identified alterations in the structure, function, and connectivity of the amygdala, as well as prosocial neuropeptide dysregulation, as some of the key neurogenetic features of WS. A recent social approach/withdrawal hypothesis (Kemp and Guastella, 2011) suggests that autonomic cardiac control may play a key role in regulating the relationship between oxytocin (OT) and social behavior. This article discusses evidence from these critical, new strands of research into social behavior in WS, to consider the extent to which data on WS may provide novel insight into the determinants of social behavior. Future research directions are suggested.