Human cortical activity evoked by gaze shift observation: an intracranial EEG study

Anatomo-functional basis of emotional and motor resonance elicited by facial expressions

Simulation theories predict that the observation of other's expressions modulates neural activity in the same centres controlling their production. This hypothesis has been developed by two models, postulating that the visual input is directly projected either to the motor system for action recognition (motor resonance) or to emotional/interoceptive regions for emotional contagion and social synchronization (emotional resonance). Here we investigated the role of frontal/insular regions in the processing of observed emotional expressions by combining intracranial recording, electrical stimulation and effective connectivity. First, we intracranially recorded from prefrontal, premotor or anterior insular regions of 44 patients during the passive observation of emotional expressions, finding widespread modulations in prefrontal/insular regions (anterior cingulate cortex, anterior insula, orbitofrontal cortex and inferior frontal gyrus) and motor territories (Rolandic operculum and inferior frontal junction). Subsequently, we electrically stimulated the activated sites, finding that (i) in the anterior cingulate cortex and anterior insula, the stimulation elicited emotional/interoceptive responses, as predicted by the 'emotional resonance model'; (ii) in the Rolandic operculum it evoked face/mouth sensorimotor responses, in line with the 'motor resonance' model; and (iii) all other regions were unresponsive or revealed functions unrelated to the processing of facial expressions. Finally, we traced the effective connectivity to sketch a network-level description of these regions, finding that the anterior cingulate cortex and the anterior insula are reciprocally interconnected while the Rolandic operculum is part of the parieto-frontal circuits and poorly connected with the former. These results support the hypothesis that the pathways hypothesized by the 'emotional resonance' and the 'motor resonance' models work in parallel, differing in terms of spatio-temporal fingerprints, reactivity to electrical stimulation and connectivity patterns.

Introducing HiBoP: a Unity-based visualization software for large iEEG datasets

BACKGROUND

Intracranial EEG data offer a unique spatio-temporal precision to investigate human brain functions. Large datasets have become recently accessible thanks to new iEEG data-sharing practices and tighter collaboration with clinicians. Yet, the complexity of such datasets poses new challenges, especially regarding the visualization and anatomical display of iEEG.

NEW METHOD

We introduce HiBoP, a multi-modal visualization software specifically designed for large groups of patients and multiple experiments. Its main features include the dynamic display of iEEG responses induced by tasks/stimulations, the definition of Regions and electrodes Of Interest, and the shift between group-level and individual-level 3D anatomo-functional data.

RESULTS

We provide a use-case with data from 36 patients to reveal the global cortical dynamics following tactile stimulation. We used HiBoP to visualize high-gamma responses [50-150 Hz], and define three major response components in primary somatosensory and premotor cortices and parietal operculum.

COMPARISON WITH EXISTING METHODS(S)

Several iEEG softwares are now publicly available with outstanding analysis features. Yet, most were developed in languages (Python/Matlab) chosen to facilitate the inclusion of new analysis by users, rather than the quality of the visualization. HiBoP represents a visualization tool developed with videogame standards (Unity/C#), and performs detailed anatomical analysis rapidly, across multiple conditions, patients, and modalities with an easy export toward third-party softwares.

CONCLUSION

HiBoP provides a user-friendly environment that greatly facilitates the exploration of large iEEG datasets, and helps users decipher subtle structure/function relationships.

From Motion to Emotion: Visual Pathways and Potential Interconnections

The two visual pathway description of [Ungerleider, L. G., & Mishkin, M. Two cortical visual systems. In D. J. Dingle, M. A. Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior (pp. 549-586). Cambridge, MA: MIT, 1982] changed the course of late 20th century systems and cognitive neuroscience. Here, I try to reexamine our laboratory's work through the lens of the [Pitcher, D., & Ungerleider, L. G. Evidence for a third visual pathway specialized for social perception. Trends in Cognitive Sciences, 25, 100-110, 2021] new third visual pathway. I also briefly review the literature related to brain responses to static and dynamic visual displays, visual stimulation involving multiple individuals, and compare existing models of social information processing for the face and body. In this context, I examine how the posterior STS might generate unique social information relative to other brain regions that also respond to social stimuli. I discuss some of the existing challenges we face with assessing how information flow progresses between structures in the proposed functional pathways and how some stimulus types and experimental designs may have complicated our data interpretation and model generation. I also note a series of outstanding questions for the field. Finally, I examine the idea of a potential expansion of the third visual pathway, to include aspects of previously proposed "lateral" visual pathways. Doing this would yield a more general entity for processing motion/action (i.e., "[inter]action") that deals with interactions between people, as well as people and objects. In this framework, a brief discussion of potential hemispheric biases for function, and different forms of neuropsychological impairments created by focal lesions in the posterior brain is highlighted to help situate various brain regions into an expanded [inter]action pathway.

Aberrant Effective Connectivity During Eye Gaze Processing Is Linked to Social Functioning and Symptoms in Schizophrenia

BACKGROUND

Patients with schizophrenia show abnormal gaze processing, which is associated with social dysfunction. These abnormalities are related to aberrant connectivity among brain regions that are associated with visual processing, social cognition, and cognitive control. In this study, we investigated 1) how effective connectivity during gaze processing is disrupted in schizophrenia and 2) how this may contribute to social dysfunction and clinical symptoms.

METHODS

Thirty-nine patients with schizophrenia/schizoaffective disorder (SZ) and 33 healthy control participants completed an eye gaze processing task during functional magnetic resonance imaging. Participants viewed faces with different gaze angles and performed explicit and implicit gaze processing. Four brain regions-the secondary visual cortex, posterior superior temporal sulcus, inferior parietal lobule, and posterior medial frontal cortex-were identified as nodes for dynamic causal modeling analysis.

RESULTS

Both the SZ and healthy control groups showed similar model structures for general gaze processing. Explicit gaze discrimination led to changes in effective connectivity, including stronger excitatory, bottom-up connections from the secondary visual cortex to the posterior superior temporal sulcus and inferior parietal lobule and inhibitory, top-down connections from the posterior medial frontal cortex to the secondary visual cortex. Group differences in top-down modulation from the posterior medial frontal cortex to the posterior superior temporal sulcus and inferior parietal lobule were noted, such that these inhibitory connections were attenuated in the healthy control group but further strengthened in the SZ group. Connectivity was associated with social dysfunction and symptom severity.

CONCLUSIONS

The SZ group showed notably stronger top-down inhibition during explicit gaze discrimination, which was associated with more social dysfunction but less severe symptoms among patients. These findings help pinpoint neural mechanisms of aberrant gaze processing and may serve as future targets for interventions that combine neuromodulation with social cognitive training.

Neurobehavioral indices of gaze perception are associated with social cognition across schizophrenia patients and healthy controls

BACKGROUND

Gaze perception is a basic building block of social cognition, which is impaired in schizophrenia (SZ) and contributes to functional outcomes. Few studies, however, have investigated neural underpinnings of gaze perception and their relation to social cognition. We address this gap.

METHOD

We recruited 77 SZ patients and 71 healthy controls, who completed various social-cognition tasks. During functional magnetic resonance imaging, participants (62 SZ, 54 controls) completed a gaze-perception task, where they judged whether faces with varying gaze angles were self-directed or averted; as a control condition, participants identified stimulus gender. Activation estimates were extracted based on (a) task versus baseline, (b) gaze-perception versus gender-identification, (c) parametric modulation by perception of stimuli as self-directed versus averted, and (d) parametric modulation by stimulus gaze angle. We used latent variable analysis to test associations among diagnostic group, brain activation, gaze perception, and social cognition.

RESULTS

Preferential activation to gaze perception was observed throughout dorsomedial prefrontal cortex, superior temporal sulcus, and insula. Activation was modulated by stimulus gaze angle and perception of stimuli as self-directed versus averted. More precise gaze perception and higher task-related activation were associated with better social cognition. Patients with SZ showed hyperactivation within left pre-/postcentral gyrus, which was associated with more precise gaze perception and fewer symptoms and thus may be a compensatory mechanism.

CONCLUSIONS

Neural and behavioral indices of gaze perception were related to social cognition, across patients and controls. This suggests gaze perception is an important perceptual building block for more complex social cognition. Results are discussed in the context of dimensional psychopathology and clinical heterogeneity. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The impact of emotional facial expressions on reflexive attention depends on the aim of dynamic gaze changes: An ERP study

The emotional expression and gaze direction of a face are important cues for human social interactions. However, the interplay of these factors and their neural correlates are only partially understood. In the current study, we investigated ERP correlates of gaze and emotion processing following the initial presentation of faces with different emotional expressions (happy, neutral, angry) and an averted or direct gaze direction as well as following a subsequent change in gaze direction that occurred in half of the trials. We focused on the time course and scalp topography of the N170 and EPN components. The N170 amplitude was larger to averted than direct gaze for the initial face presentation and larger to gaze changes from direct to averted than from averted to direct in response to the gaze change. For the EPN component in response to the initial face presentation, we replicate classic effects of emotion, which did not interact with gaze direction. As a major new finding, changes from direct to averted gaze elicited an EPN-like effect when the face showed a happy expression. No such effect was seen for angry expressions. We conclude that happy faces reflexively attract attention when they look at the observer rather than away from the observer. These results for happy expressions are in line with the shared signal hypothesis that posits a better processing of expressions if their approach or avoidance tendency is consistent with gaze direction. However, the shared signal hypothesis is not supported by the present results for angry faces.

Frequency-specific gaze modulation of emotional face processing in the human amygdala

Determining the social significance of emotional face expression is of major importance for adaptive behavior, and gaze direction provides critical information in this process. The amygdala is implicated in both emotion and gaze processing, but how and when it integrates expression and gaze cues remains unresolved. We tackled this question using intracranial electroencephalography in epileptic patients to assess both amygdala (n = 12) and orbitofrontal cortex (OFC; n = 11) time-frequency evoked responses to faces with different emotional expressions and different gaze directions. As predicted, self-relevant threat signals (averted fearful and directed angry faces) elicited stronger amygdala activity than self-irrelevant threat (directed fearful and averted angry faces). Fear effects started at early latencies in both amygdala and OFC (~110 and 160 ms, respectively), while gaze direction effects and their interaction with emotion occurred at later latencies. Critically, the amygdala showed differential gamma band increases to fearful averted gaze (starting ~550 ms) and to angry directed gaze (~470 ms). Moreover, when comparing the 2 self-relevant threat conditions among them, we found higher gamma amygdala activity for averted fearful faces and higher beta OFC activity for angry directed faces. Together, these results reveal for the first time frequency-specific effects of emotion and gaze on amygdala and OFC neural activity.

Visual Information Routes in the Posterior Dorsal and Ventral Face Network Studied with Intracranial Neurophysiology and White Matter Tract Endpoints

Occipitotemporal regions within the face network process perceptual and socioemotional information, but the dynamics and information flow between different nodes of this network are still debated. Here, we analyzed intracerebral EEG from 11 epileptic patients viewing a stimulus sequence beginning with a neutral face with direct gaze. The gaze could avert or remain direct, while the emotion changed to fearful or happy. N200 field potential peak latencies indicated that face processing begins in inferior occipital cortex and proceeds anteroventrally to fusiform and inferior temporal cortices, in parallel. The superior temporal sulcus responded preferentially to gaze changes with augmented field potential amplitudes for averted versus direct gaze, and large effect sizes relative to other network regions. An overlap analysis of posterior white matter tractography endpoints (from 1066 healthy brains) relative to active intracerebral electrodes in the 11 patients showed likely involvement of both dorsal and ventral posterior white matter pathways. Overall, our data provide new insight into the timing of face and social cue processing in the occipitotemporal brain and anchor the superior temporal cortex in dynamic gaze processing.

Dynamic causal modeling of eye gaze processing in schizophrenia

BACKGROUND

Abnormal eye gaze perception is related to symptoms and social functioning in schizophrenia. However, little is known about the brain network mechanisms underlying these abnormalities. Here, we employed dynamic causal modeling (DCM) of fMRI data to discover aberrant effective connectivity within networks associated with eye gaze processing in schizophrenia.

METHODS

Twenty-seven patients (schizophrenia/schizoaffective disorder, SZ) and 22 healthy controls (HC) completed an eye gaze processing task during fMRI. Participants viewed faces with different gaze angles and performed explicit gaze discrimination (Gaze: "Looking at you?" yes/no) or implicit gaze processing (Gender: "male or female?"). Four brain regions, the secondary visual cortex (Vis), posterior superior temporal sulcus (pSTS), inferior parietal lobule (IPL), and posterior medial frontal cortex (pMFC) were identified as nodes for subsequent DCM analysis.

RESULTS

SZ and HC showed similar generative model structure, but SZ showed altered connectivity for specific self-connections, inter-regional connections during all gaze processing (reduced excitatory bottom-up and enhanced inhibitory top-down connections), and modulation by explicit gaze discrimination (increased frontal inhibition of visual cortex). Altered effective connectivity was significantly associated with poorer social cognition and functioning.

CONCLUSIONS

General gaze processing in SZ is associated with distributed cortical dysfunctions and bidirectional connectivity between regions, while explicit gaze discrimination involves predominantly top-down abnormalities in the visual system. These results suggest plausible neural mechanisms underpinning gaze processing deficits and may serve as bio-markers for intervention.

Social perspective-taking shapes brain hemodynamic activity and eye movements during movie viewing

Putting oneself into the shoes of others is an important aspect of social cognition. We measured brain hemodynamic activity and eye-gaze patterns while participants were viewing a shortened version of the movie 'My Sister's Keeper' from two perspectives: that of a potential organ donor, who violates moral norms by refusing to donate her kidney, and that of a potential organ recipient, who suffers in pain. Inter-subject correlation (ISC) of brain activity was significantly higher during the potential organ donor's perspective in dorsolateral and inferior prefrontal, lateral and inferior occipital, and inferior-anterior temporal areas. In the reverse contrast, stronger ISC was observed in superior temporal, posterior frontal and anterior parietal areas. Eye-gaze analysis showed higher proportion of fixations on the potential organ recipient during both perspectives. Taken together, these results suggest that during social perspective-taking different brain areas can be flexibly recruited depending on the nature of the perspective that is taken.

Action execution and action observation elicit mirror responses with the same temporal profile in human SII

The properties of the secondary somatosensory area (SII) have been described by many studies in monkeys and humans. Recent studies on monkeys, however, showed that beyond somatosensory stimuli, SII responds to a wider number of stimuli, a finding requiring a revision that human SII is purely sensorimotor. By recording cortical activity with stereotactic electroencephalography (stereo-EEG), we examined the properties of SI and SII in response to a motor task requiring reaching, grasping and manipulation, as well as the observation of the same actions. Furthermore, we functionally characterized this area with a set of clinical tests, including tactile, acoustical, and visual stimuli. The results showed that only SII activates both during execution and observation with a common temporal profile, whereas SI response were limited to execution. Together with their peculiar response to tactile stimuli, we conclude that the role of SII is pivotal also in the observation of actions involving haptic control.

A drama movie activates brains of holistic and analytical thinkers differentially

People socialized in different cultures differ in their thinking styles. Eastern-culture people view objects more holistically by taking context into account, whereas Western-culture people view objects more analytically by focusing on them at the expense of context. Here we studied whether participants, who have different thinking styles but live within the same culture, exhibit differential brain activity when viewing a drama movie. A total of 26 Finnish participants, who were divided into holistic and analytical thinkers based on self-report questionnaire scores, watched a shortened drama movie during functional magnetic resonance imaging. We compared intersubject correlation (ISC) of brain hemodynamic activity of holistic vs analytical participants across the movie viewings. Holistic thinkers showed significant ISC in more extensive cortical areas than analytical thinkers, suggesting that they perceived the movie in a more similar fashion. Significantly higher ISC was observed in holistic thinkers in occipital, prefrontal and temporal cortices. In analytical thinkers, significant ISC was observed in right-hemisphere fusiform gyrus, temporoparietal junction and frontal cortex. Since these results were obtained in participants with similar cultural background, they are less prone to confounds by other possible cultural differences. Overall, our results show how brain activity in holistic vs analytical participants differs when viewing the same drama movie.

The influence of top-down modulation on the processing of direct gaze

Gaze or eye contact is one of the most important nonverbal social cues, which is fundamental to human social interactions. To achieve real time and dynamic face-to-face communication, our brain needs to process another person's gaze direction rapidly and without explicit instruction. In order to explain the fast and spontaneous processing of direct gaze, the fast-track modulator model was proposed. Here, we review recent developments in gaze processing research in the last decade to extend the fast-track modulator model. In particular, we propose that task demand or top-down modulation could play a more crucial role at gaze processing than formerly assumed. We suggest that under different task demands, top-down modulation can facilitate or interfere with the direct gaze effects for early visual processing. The proposed modification of the model extends the role of task demand and its implication on the direct gaze effect, as well as the need to better control for top-down processing in order to better disentangle the role of top-down and bottom-up processing on the direct gaze effect. This article is categorized under: Cognitive Biology > Evolutionary Roots of Cognition Psychology > Perception and Psychophysics Neuroscience > Cognition.

How attention gates social interactions

Social interactions are at the core of social life. However, humans selectively choose their exchange partners and do not engage in all available opportunities for social encounters. In this review, we argue that attentional systems play an important role in guiding the selection of social interactions. Supported by both classic and emerging literature, we identify and characterize the three core processes-perception, interpretation, and evaluation-that interact with attentional systems to modulate selective responses to social environments. Perceptual processes facilitate attentional prioritization of social cues. Interpretative processes link attention with understanding of cues' social meanings and agents' mental states. Evaluative processes determine the perceived value of the source of social information. The interplay between attention and these three routes of processing places attention in a powerful role to manage the selection of the vast amount of social information that individuals encounter on a daily basis and, in turn, gate the selection of social interactions.

Multiple time courses of somatosensory responses in human cortex

Here we show how anatomical and functional data recorded from patients undergoing stereo-EEG can be used to decompose the cortical processing following nerve stimulation in different stages characterized by specific topography and time course. Tibial, median and trigeminal nerves were stimulated in 96 patients, and the increase in gamma power was evaluated over 11878 cortical sites. All three nerve datasets exhibited similar clusters of time courses: phasic, delayed/prolonged and tonic, which differed in topography, temporal organization and degree of spatial overlap. Strong phasic responses of the three nerves followed the classical somatotopic organization of SI, with no overlap in either time or space. Delayed responses presented overlaps between pairs of body parts in both time and space, and were confined to the dorsal motor cortices. Finally, tonic responses occurred in the perisylvian region including posterior insular cortex and were evoked by the stimulation of all three nerves, lacking any spatial and temporal specificity. These data indicate that the somatosensory processing following nerve stimulation is a multi-stage hierarchical process common to all three nerves, with the different stages likely subserving different functions. While phasic responses represent the neural basis of tactile perception, multi-nerve tonic responses may represent the neural signature of processes sustaining the capacity to become aware of tactile stimuli.

Gamma oscillations in the superior colliculus and pulvinar in response to faces support discrimination performance in monkeys

The subcortical visual pathway including the superior colliculus (SC), pulvinar, and amygdala has been implicated in unconscious visual processing of faces, eyes, and gaze direction in blindsight. Our previous studies reported that monkey SC and pulvinar neurons responded preferentially to images of faces while performing a delayed non-matching to sample (DNMS) task to discriminate different visual stimuli (Nguyen et al., 2013, 2014). However, the contribution of SC and pulvinar neurons to the discrimination of the facial images and subsequent behavioral performance remains unknown. Since gamma oscillations have been implicated in sensory and cognitive processes as well as behavioral execution, we hypothesized that gamma oscillations during neuronal responses might contribute to achieving the appropriate behavioral performance (i.e., a correct response). In the present study, we re-analyzed those neuronal responses in the monkey SC and pulvinar to investigate possible relationships between gamma oscillations in these neurons and behavioral performance (correct response ratios) during the DNMS task. Gamma oscillations of SC and pulvinar neuronal activity were analyzed in three phases around the stimulus onset [inter-trial interval (ITI): 1000ms before trial onset; Early: 0-200ms after stimulus onset; and Late: 300-500ms after stimulus onset]. We found that human facial images elicited stronger gamma oscillations in the early phase than the ITI and late phase in both the SC and pulvinar neurons. Furthermore, there was a significant correlation between strengths of gamma oscillations in the early phase and behavioral performance in both the SC and pulvinar. The results suggest that gamma oscillatory activity in the SC and pulvinar contributes to successful behavioral performance during unconscious perceptual and behavioral processes.

Neuronal Encoding of Self and Others' Head Rotation in the Macaque Dorsal Prefrontal Cortex

Following gaze is a crucial skill, in primates, for understanding where and at what others are looking, and often requires head rotation. The neural basis underlying head rotation are deemed to overlap with the parieto-frontal attention/gaze-shift network. Here, we show that a set of neurons in monkey’s Brodmann area 9/46dr (BA 9/46dr), which is involved in orienting processes and joint attention, becomes active during self head rotation and that the activity of these neurons cannot be accounted for by saccade-related activity (head-rotation neurons). Another set of BA 9/46dr neurons encodes head rotation performed by an observed agent facing the monkey (visually triggered neurons). Among these latter neurons, almost half exhibit the intriguing property of encoding both execution and observation of head rotation (mirror-like neurons). Finally, by means of neuronal tracing techniques, we showed that BA 9/46dr takes part into two distinct networks: a dorso/mesial network, playing a role in spatial head/gaze orientation, and a ventrolateral network, likely involved in processing social stimuli and mirroring others’ head. The overall results of this study provide a new, comprehensive picture of the role of BA 9/46dr in encoding self and others’ head rotation, likely playing a role in head-following behaviors.

Gaze matters! The effect of gaze direction on emotional enhancement of memory for faces in patients with mesial temporal lobe epilepsy

PURPOSE

The aim of the study was to examine if gaze and emotional expression, both highly self-relevant social signals, affect the recollection accuracy of perceived faces in patients with mesial temporal lobe epilepsy (MTLE).

METHODS

Forty patients with MTLE (twenty-one without surgery and nineteen after anterior temporal lobectomy) as well as twenty healthy controls (HC) took part in the study. We used a set of 64 facial stimuli: 32 neutral and 32 emotional displays (16 fearful; 16 angry) from well-established affective stimuli databases. Half of the faces in each condition had eyes directed straight and half - away from the observer. Participants performed a gender identification task, and then, after a 45-minute delay were asked to identify the previously seen stimuli, presented among a new set of photos.

RESULTS

Increased automatic learning of angry and fearful compared to neutral expressions was found in HC. There was no emotional enhancement of memory in MTLE but an increased learning for faces with averted than direct gaze.

CONCLUSION

Our results expand on previous research by demonstrating that emotion expression and gaze direction can affect memory of faces. The study supports the hypothesis that healthy individuals and patients with temporal lobe abnormalities present different patterns of emotional gazes processing. The potential consequences of altered emotional gaze processing and social cognition impairments need to be further investigated to improve the quality of life of patients with MTLE.

Gamma Oscillations in the Temporal Pole in Response to Eyes

The eyes of an individual act as an indispensable communication medium during human social interactions. Functional neuroimaging studies have revealed that several brain regions are activated in response to eyes and eye gaze direction changes. However, it remains unclear whether the temporal pole is one of these regions. Furthermore, if the temporal pole is activated by these stimuli, the timing and manner in which it is activated also remain unclear. To investigate these issues, we analyzed intracranial electroencephalographic data from the temporal pole that were obtained during the presentation of eyes and mosaics in averted or straight directions and their directional changes. Time-frequency statistical parametric mapping analyses revealed that the bilateral temporal poles exhibited greater gamma-band activation beginning at 215 ms in response to eyes compared with mosaics, irrespective of the direction. Additionally, the right temporal pole showed greater gamma-band activation beginning at 197 ms in response to directional changes of the eyes compared with mosaics. These results suggest that gamma-band oscillations in the temporal pole were involved in the processing of the presence of eyes and changes in eye gaze direction at a relatively late temporal stage compared with the posterior cortices.

Four-dimensional maps of the human somatosensory system

A fine-grained description of the spatiotemporal dynamics of human brain activity is a major goal of neuroscientific research. Limitations in spatial and temporal resolution of available noninvasive recording and imaging techniques have hindered so far the acquisition of precise, comprehensive four-dimensional maps of human neural activity. The present study combines anatomical and functional data from intracerebral recordings of nearly 100 patients, to generate highly resolved four-dimensional maps of human cortical processing of nonpainful somatosensory stimuli. These maps indicate that the human somatosensory system devoted to the hand encompasses a widespread network covering more than 10% of the cortical surface of both hemispheres. This network includes phasic components, centered on primary somatosensory cortex and neighboring motor, premotor, and inferior parietal regions, and tonic components, centered on opercular and insular areas, and involving human parietal rostroventral area and ventral medial-superior-temporal area. The technique described opens new avenues for investigating the neural basis of all levels of cortical processing in humans.

Neural mechanisms underlying conscious and unconscious attentional shifts triggered by eye gaze

Behavioral studies have shown that eye gaze triggers attentional shifts both with and without conscious awareness. However, the neural substrates of conscious and unconscious attentional shifts triggered by eye gaze remain unclear. To investigate this issue, we measured brain activity using event-related functional magnetic resonance imaging while participants observed averted or straight eye-gaze cues presented supraliminally or subliminally in the central visual field and then localized a subsequent target in the peripheral visual field. Reaction times for localizing the targets were shorter under both supraliminal and subliminal conditions when eye-gaze cues were directionally congruent with the target locations than when they were directionally neutral. Conjunction analyses revealed that a bilateral cortical network, including the middle temporal gyri, inferior parietal lobules, anterior cingulate cortices, and superior and middle frontal gyri, was activated more in response to averted eyes than to straight eyes under both supraliminal and subliminal conditions. Interaction analyses revealed that the right inferior parietal lobule was specifically active when participants viewed averted eyes relative to straight eyes under the supraliminal condition; the bilateral subcortical regions, including the superior colliculus and amygdala, and the middle temporal and inferior frontal gyri in the right hemisphere were activated in response to averted versus straight eyes under the subliminal condition. These results suggest commonalities and differences in the neural mechanisms underlying conscious and unconscious attentional shifts triggered by eye gaze.

Social decisions affect neural activity to perceived dynamic gaze

Gaze direction, a cue of both social and spatial attention, is known to modulate early neural responses to faces e.g. N170. However, findings in the literature have been inconsistent, likely reflecting differences in stimulus characteristics and task requirements. Here, we investigated the effect of task on neural responses to dynamic gaze changes: away and toward transitions (resulting or not in eye contact). Subjects performed, in random order, social (away/toward them) and non-social (left/right) judgment tasks on these stimuli. Overall, in the non-social task, results showed a larger N170 to gaze aversion than gaze motion toward the observer. In the social task, however, this difference was no longer present in the right hemisphere, likely reflecting an enhanced N170 to gaze motion toward the observer. Our behavioral and event-related potential data indicate that performing social judgments enhances saliency of gaze motion toward the observer, even those that did not result in gaze contact. These data and that of previous studies suggest two modes of processing visual information: a 'default mode' that may focus on spatial information; a 'socially aware mode' that might be activated when subjects are required to make social judgments. The exact mechanism that allows switching from one mode to the other remains to be clarified.

Photographic but not line-drawn faces show early perceptual neural sensitivity to eye gaze direction

Our brains readily decode facial movements and changes in social attention, reflected in earlier and larger N170 event-related potentials (ERPs) to viewing gaze aversions vs. direct gaze in real faces (Puce et al., 2000). In contrast, gaze aversions in line-drawn faces do not produce these N170 differences (Rossi et al., 2014), suggesting that physical stimulus properties or experimental context may drive these effects. Here we investigated the role of stimulus-induced context on neurophysiological responses to dynamic gaze. Sixteen healthy adults viewed line-drawn and real faces, with dynamic eye aversion and direct gaze transitions, and control stimuli (scrambled arrays and checkerboards) while continuous electroencephalographic (EEG) activity was recorded. EEG data from 2 temporo-occipital clusters of 9 electrodes in each hemisphere where N170 activity is known to be maximal were selected for analysis. N170 peak amplitude and latency, and temporal dynamics from Event-Related Spectral Perturbations (ERSPs) were measured in 16 healthy subjects. Real faces generated larger N170s for averted vs. direct gaze motion, however, N170s to real and direct gaze were as large as those to respective controls. N170 amplitude did not differ across line-drawn gaze changes. Overall, bilateral mean gamma power changes for faces relative to control stimuli occurred between 150–350 ms, potentially reflecting signal detection of facial motion. Our data indicate that experimental context does not drive N170 differences to viewed gaze changes. Low-level stimulus properties, such as the high sclera/iris contrast change in real eyes likely drive the N170 changes to viewed aversive movements.

How action selection can be embodied: intracranial gamma band recording shows response competition during the Eriksen flankers test

Recent findings in monkeys suggest that action selection is based on a competition between various action options that are automatically planned by the motor system. Here we discuss data from intracranial EEG recordings in human premotor cortex (PMC) during a bimanual version of the Eriksen flankers test that suggest that the same principles apply to human action decisions. Recording sites in the dorsal PMC show an early but undifferentiated activation, a delayed response that depends on the experimental conditions and, finally, a movement related activation during action execution. Additionally, we found that the medial part of the PMC show a significant increase in response for ipsilateral trials, suggesting a role in inhibiting the wrong response. The ventral PMC seems to be involved in action execution, rather than action selection. Together these findings suggest that the human PMC is part of a network that specifies, selects, and executes actions.

From speech to thought: the neuronal basis of cognitive units in non-experimental, real-life communication investigated using ECoG

Exchange of thoughts by means of expressive speech is fundamental to human communication. However, the neuronal basis of real-life communication in general, and of verbal exchange of ideas in particular, has rarely been studied until now. Here, our aim was to establish an approach for exploring the neuronal processes related to cognitive “idea” units (IUs) in conditions of non-experimental speech production. We investigated whether such units corresponding to single, coherent chunks of speech with syntactically-defined borders, are useful to unravel the neuronal mechanisms underlying real-world human cognition. To this aim, we employed simultaneous electrocorticography (ECoG) and video recordings obtained in pre-neurosurgical diagnostics of epilepsy patients. We transcribed non-experimental, daily hospital conversations, identified IUs in transcriptions of the patients' speech, classified the obtained IUs according to a previously-proposed taxonomy focusing on memory content, and investigated the underlying neuronal activity. In each of our three subjects, we were able to collect a large number of IUs which could be assigned to different functional IU subclasses with a high inter-rater agreement. Robust IU-onset-related changes in spectral magnitude could be observed in high gamma frequencies (70–150 Hz) on the inferior lateral convexity and in the superior temporal cortex regardless of the IU content. A comparison of the topography of these responses with mouth motor and speech areas identified by electrocortical stimulation showed that IUs might be of use for extraoperative mapping of eloquent cortex (average sensitivity: 44.4%, average specificity: 91.1%). High gamma responses specific to memory-related IU subclasses were observed in the inferior parietal and prefrontal regions. IU-based analysis of ECoG recordings during non-experimental communication thus elicits topographically- and functionally-specific effects. We conclude that segmentation of spontaneous real-world speech in linguistically-motivated units is a promising strategy for elucidating the neuronal basis of mental processing during non-experimental communication.

Neural correlates of apparent motion perception of impoverished facial stimuli: a comparison of ERP and ERSP activity

Our brains readily decode human movements, as shown by neural responses to face and body motion. N170 event-related potentials (ERPs) are earlier and larger to mouth opening movements relative to closing in both line-drawn and natural faces, and gaze aversions relative to direct gaze in natural faces (Puce and Perrett, 2003; Puce et al., 2000). Here we extended this work by recording both ERP and oscillatory EEG activity (event-related spectral perturbations, ERSPs) to line-drawn faces depicting eye and mouth movements (Eyes: Direct vs Away; Mouth: Closed vs Open) and non-face motion controls. Neural activity was measured in 2 occipito-temporal clusters of 9 electrodes, one in each hemisphere. Mouth opening generated larger N170s than mouth closing, replicating earlier work. Eye motion elicited robust N170s that did not differ between gaze conditions. Control condition differences were seen, and generated the largest N170. ERSP difference plots across conditions in the occipito-temporal electrode clusters (Eyes: Direct vs Away; Mouth: Closed vs Open) showed statistically significant differences in beta and gamma bands for gaze direction changes and mouth opening at similar post-stimulus times and frequencies. In contrast, control stimuli showed activity in the gamma band with a completely different time profile and hemispheric distribution to facial stimuli. ERSP plots were generated in two 9 electrode clusters centered on central sites, C3 and C4. In the left cluster for all stimulus conditions, broadband beta suppression persisted from about 250ms post-motion onset. In the right cluster, beta suppression was seen for control conditions only. Statistically significant differences between conditions were confined between 4 and 15Hz, unlike the occipito-temporal sites where differences occurred at much higher frequencies (high beta/gamma). Our data indicate that N170 amplitude is sensitive to the amount of movement in the visual field, independent of stimulus type. In contrast, occipito-temporal beta and gamma activity differentiates between facial and non-facial motion. Context and stimulus configuration likely plays a role in shaping neural responses, based on comparisons of the current data to previously reported studies. Broadband suppression of central beta activity, and significant low frequency differences were likely stimulus driven and not contingent on behavioral responses.

Sequencing biological and physical events affects specific frequency bands within the human premotor cortex: an intracerebral EEG study

Evidence that the human premotor cortex (PMC) is activated by cognitive functions involving the motor domain is classically explained as the reactivation of a motor program decoupled from its executive functions, and exploited for different purposes by means of a motor simulation. In contrast, the evidence that PMC contributes to the sequencing of non-biological events cannot be explained by the simulationist theory. Here we investigated how motor simulation and event sequencing coexist within the PMC and how these mechanisms interact when both functions are executed. We asked patients with depth electrodes implanted in the PMC to passively observe a randomized arrangement of images depicting biological actions and physical events and, in a second block, to sequence them in the correct order. This task allowed us to disambiguate between the simple observation of actions, their sequencing (recruiting different motor simulation processes), as well as the sequencing of non-biological events (recruiting a sequencer mechanism non dependant on motor simulation). We analysed the response of the gamma, alpha and beta frequency bands to evaluate the contribution of each brain rhythm to the observation and sequencing of both biological and non-biological stimuli. We found that motor simulation (biological>physical) and event sequencing (sequencing>observation) differently affect the three investigated frequency bands: motor simulation was reflected on the gamma and, partially, in the beta, but not in the alpha band. In contrast, event sequencing was also reflected on the alpha band.