Intracranial markers of conscious face perception in humans

Conscious perception of fear in faces: Insights from high-density EEG and perceptual awareness scale with threshold stimuli

Contrary to the extensive research on processing subliminal and/or unattended emotional facial expressions, only a minority of studies have investigated the neural correlates of consciousness (NCCs) of emotions conveyed by faces. In the present high-density electroencephalography (EEG) study, we first employed a staircase procedure to identify each participant's perceptual threshold of the emotion expressed by the face and then compared the EEG signals elicited in trials where the participants were aware with the activity elicited in trials where participants were unaware of the emotions expressed by these, otherwise identical, faces. Drawing on existing knowledge of the neural mechanisms of face processing and NCCs, we hypothesized that activity in frontal electrodes would be modulated in relation to participants' awareness of facial emotional content. More specifically, we hypothesized that the NCC of fear seen on someone else's face could be detected as a modulation of a later and more anterior (i.e., at frontal sites) event-related potential (ERP) than the face-sensitive N170. By adopting a data-driven approach and cluster-based statistics to the analysis of EEG signals, the results were clear-cut in showing that visual awareness of fear was associated with the modulation of a frontal ERP component in a 150-300 msec interval. These insights are dissected and contextualized in relation to prevailing theories of visual consciousness and their proposed NCC benchmarks.

Decoding the temporal representation of facial expression in face-selective regions

The ability of humans to discern facial expressions in a timely manner typically relies on distributed face-selective regions for rapid neural computations. To study the time course in regions of interest for this process, we used magnetoencephalography (MEG) to measure neural responses participants viewed facial expressions depicting seven types of emotions (happiness, sadness, anger, disgust, fear, surprise, and neutral). Analysis of the time-resolved decoding of neural responses in face-selective sources within the inferior parietal cortex (IP-faces), lateral occipital cortex (LO-faces), fusiform gyrus (FG-faces), and posterior superior temporal sulcus (pSTS-faces) revealed that facial expressions were successfully classified starting from ∼100 to 150 ms after stimulus onset. Interestingly, the LO-faces and IP-faces showed greater accuracy than FG-faces and pSTS-faces. To examine the nature of the information processed in these face-selective regions, we entered with facial expression stimuli into a convolutional neural network (CNN) to perform similarity analyses against human neural responses. The results showed that neural responses in the LO-faces and IP-faces, starting ∼100 ms after the stimuli, were more strongly correlated with deep representations of emotional categories than with image level information from the input images. Additionally, we observed a relationship between the behavioral performance and the neural responses in the LO-faces and IP-faces, but not in the FG-faces and lpSTS-faces. Together, these results provided a comprehensive picture of the time course and nature of information involved in facial expression discrimination across multiple face-selective regions, which advances our understanding of how the human brain processes facial expressions.

Early cortical signals in visual stimulus detection

During visual conscious perception, the earliest responses linked to signal detection are little known. The current study aims to reveal the cortical neural activity changes in the earliest stages of conscious perception using recordings from intracranial electrodes. Epilepsy patients (N=158) were recruited from a multi-center collaboration and completed a visual word recall task. Broadband gamma activity (40-115Hz) was extracted with a band-pass filter and gamma power was calculated across subjects on a common brain surface. Our results show early gamma power increases within 0-50ms after stimulus onset in bilateral visual processing cortex, right frontal cortex (frontal eye fields, ventral medial/frontopolar, orbital frontal) and bilateral medial temporal cortex regardless of whether the word was later recalled. At the same early times, decreases were seen in the left rostral middle frontal gyrus. At later times after stimulus onset, gamma power changes developed in multiple cortical regions. These included sustained changes in visual and other association cortical networks, and transient decreases in the default mode network most prominently at 300-650ms. In agreement with prior work in this verbal memory task, we also saw greater increases in visual and medial temporal regions as well as prominent later (> 300ms) increases in left hemisphere language areas for recalled versus not recalled stimuli. These results suggest an early signal detection network in the frontal, medial temporal, and visual cortex is engaged at the earliest stages of conscious visual perception.

Continuous flash suppression: Known and unknowns

Studies utilizing continuous flash suppression (CFS) provide valuable information regarding conscious and nonconscious perception. There are, however, crucial unanswered questions regarding the mechanisms of suppression and the level of visual processing in the absence of consciousness with CFS. Research suggests that the answers to these questions depend on the experimental configuration and how we assess consciousness in these studies. The aim of this review is to evaluate the impact of different experimental configurations and the assessment of consciousness on the results of the previous CFS studies. We review studies that evaluated the influence of different experimental configuration on the depth of suppression with CFS and discuss how different assessments of consciousness may impact the results of CFS studies. Finally, we review behavioral and brain recording studies of CFS. In conclusion, previous studies provide evidence for survival of low-level visual information and complete impairment of high-level visual information under the influence of CFS. That is, studies suggest that nonconscious perception of lower-level visual information happens with CFS, but there is no evidence for nonconscious high-level recognition with CFS.

Time-varying effective EEG source connectivity: the optimization of model parameters

Adaptive estimation methods based on general Kalman filter are powerful tools to investigate brain networks dynamics given the non-stationary nature of neural signals. These methods rely on two parameters, the model order p and adaptation constant c, which determine the resolution and smoothness of the time-varying multivariate autoregressive estimates. A sub-optimal filtering may present consistent biases in the frequency domain and temporal distortions, leading to fallacious interpretations. Thus, the performance of these methods heavily depends on the accurate choice of these two parameters in the filter design. In this work, we sought to define an objective criterion for the optimal choice of these parameters. Since residual- and information-based criteria are not guaranteed to reach an absolute minimum, we propose to study the partial derivatives of these functions to guide the choice of p and c. To validate the performance of our method, we used a dataset of human visual evoked potentials during face perception where the generation and propagation of information in the brain is well understood and a set of simulated data where the ground truth is available.

Faces under continuous flash suppression capture attention faster than objects, but without a face-evoked steady-state visual potential: Is curvilinearity responsible for the behavioral effect?

Face perception is a vital part of human social interactions. The social value of faces makes their efficient detection evolutionarily advantageous. It has been suggested that this might occur nonconsciously, but experimental results are equivocal thus far. Here, we probe nonconscious face perception using a novel combination of binocular rivalry with continuous flash suppression and steady-state visually evoked potentials. In the first two experiments, participants viewed either non-face objects, neutral faces (Study 1), or fearful faces (Study 2). Consistent with the hypothesis that faces are processed nonconsciously, we found that faces broke through suppression faster than objects. We did not, however, observe a concomitant face-selective steady-state visually evoked potential. Study 3 was run to reconcile this paradox. We hypothesized that the faster breakthrough time was due to a mid-level visual feature, curvilinearity, rather than high-level category membership, which would explain the behavioral difference without neural evidence of face-selective processing. We tested this hypothesis by presenting participants with four different groups of stimuli outside of conscious awareness: rectilinear objects (e.g., chessboard), curvilinear objects (e.g., dartboard), faces, and objects that were not dominantly curvilinear or rectilinear. We found that faces and curvilinear objects broke through suppression faster than objects and rectilinear objects. Moreover, there was no difference between faces and curvilinear objects. These results support our hypothesis that the observed behavioral advantage for faces is due to their curvilinearity, rather than category membership.

Converging intracortical signatures of two separated processing timescales in human early auditory cortex

Neural oscillations in auditory cortex are argued to support parsing and representing speech constituents at their corresponding temporal scales. Yet, how incoming sensory information interacts with ongoing spontaneous brain activity, what features of the neuronal microcircuitry underlie spontaneous and stimulus-evoked spectral fingerprints, and what these fingerprints entail for stimulus encoding, remain largely open questions. We used a combination of human invasive electrophysiology, computational modeling and decoding techniques to assess the information encoding properties of brain activity and to relate them to a plausible underlying neuronal microarchitecture. We analyzed intracortical auditory EEG activity from 10 patients while they were listening to short sentences. Pre-stimulus neural activity in early auditory cortical regions often exhibited power spectra with a shoulder in the delta range and a small bump in the beta range. Speech decreased power in the beta range, and increased power in the delta-theta and gamma ranges. Using multivariate machine learning techniques, we assessed the spectral profile of information content for two aspects of speech processing: detection and discrimination. We obtained better phase than power information decoding, and a bimodal spectral profile of information content with better decoding at low (delta-theta) and high (gamma) frequencies than at intermediate (beta) frequencies. These experimental data were reproduced by a simple rate model made of two subnetworks with different timescales, each composed of coupled excitatory and inhibitory units, and connected via a negative feedback loop. Modeling and experimental results were similar in terms of pre-stimulus spectral profile (except for the iEEG beta bump), spectral modulations with speech, and spectral profile of information content. Altogether, we provide converging evidence from both univariate spectral analysis and decoding approaches for a dual timescale processing infrastructure in human auditory cortex, and show that it is consistent with the dynamics of a simple rate model.

Reduced Dynamic Complexity of BOLD Signals Differentiates Mild Cognitive Impairment From Normal Aging

Mild cognitive impairment (MCI) is characterized as a transitional phase between cognitive decline associated with normal aging and Alzheimer’s disease (AD). Resting-state functional magnetic resonance imaging (fMRI) measuring blood oxygenation level-dependent (BOLD) signals provides complementary information considered essential for understanding disease progression. Previous studies suggested that multi-scale entropy (MSE) analysis quantifying the complexity of BOLD signals is a novel and promising method for investigating neurodegeneration associated with cognitive decline in different stages of MCI. Therefore, the current study used MSE to explore the changes in the complexity of resting-state brain BOLD signals in patients with early MCI (EMCI) and late MCI (LMCI). We recruited 345 participants’ data from the Alzheimer’s Disease Neuroimaging Initiative database, including 176 normal control (NC) subjects, 87 patients with EMCI and 82 patients with LMCI. We observed a significant reduction of brain signal complexity toward regularity in the left fusiform gyrus region in the EMCI group and in the rostral anterior cingulate cortex in the LMCI group. Our results extend prior work by revealing that significant reductions of brain BOLD signal complexity can be detected in different stages of MCI independent of age, sex and regional atrophy. Notably, the reduction of BOLD signal complexity in the rostral anterior cingulate cortex was significantly associated with greater risk of progression to AD. The present study thus identified MSE as a potential imaging biomarker for the early diagnosis of pre-clinical Alzheimer’s disease and provides further insights into the neuropathology of cognitive decline in prodromal AD.

Mathematics and the Brain: A Category Theoretical Approach to Go Beyond the Neural Correlates of Consciousness

Consciousness is a central issue in neuroscience, however, we still lack a formal framework that can address the nature of the relationship between consciousness and its physical substrates. In this review, we provide a novel mathematical framework of category theory (CT), in which we can define and study the sameness between different domains of phenomena such as consciousness and its neural substrates. CT was designed and developed to deal with the relationships between various domains of phenomena. We introduce three concepts of CT which include (i) category; (ii) inclusion functor and expansion functor; and, most importantly, (iii) natural transformation between the functors. Each of these mathematical concepts is related to specific features in the neural correlates of consciousness (NCC). In this novel framework, we will examine two of the major theories of consciousness, integrated information theory (IIT) of consciousness and temporospatial theory of consciousness (TTC). We conclude that CT, especially the application of the notion of natural transformation, highlights that we need to go beyond NCC and unravels questions that need to be addressed by any future neuroscientific theory of consciousness.

Estimating EEG Source Dipole Orientation Based on Singular-value Decomposition for Connectivity Analysis

In the last decade, the use of high-density electrode arrays for EEG recordings combined with the improvements of source reconstruction algorithms has allowed the investigation of brain networks dynamics at a sub-second scale. One powerful tool for investigating large-scale functional brain networks with EEG is time-varying effective connectivity applied to source signals obtained from electric source imaging. Due to computational and interpretation limitations, the brain is usually parcelled into a limited number of regions of interests (ROIs) before computing EEG connectivity. One specific need and still open problem is how to represent the time- and frequency-content carried by hundreds of dipoles with diverging orientation in each ROI with one unique representative time-series. The main aim of this paper is to provide a method to compute a signal that explains most of the variability of the data contained in each ROI before computing, for instance, time-varying connectivity. As the representative time-series for a ROI, we propose to use the first singular vector computed by a singular-value decomposition of all dipoles belonging to the same ROI. We applied this method to two real datasets (visual evoked potentials and epileptic spikes) and evaluated the time-course and the frequency content of the obtained signals. For each ROI, both the time-course and the frequency content of the proposed method reflected the expected time-course and the scalp-EEG frequency content, representing most of the variability of the sources (~ 80%) and improving connectivity results in comparison to other procedures used so far. We also confirm these results in a simulated dataset with a known ground truth.

Conscious Perception as Integrated Information Patterns in Human Electrocorticography

A significant problem in neuroscience concerns the distinction between neural processing that is correlated with conscious percepts from processing that is not. Here, we tested if a hierarchical structure of causal interactions between neuronal populations correlates with conscious perception. We derived the hierarchical causal structure as a pattern of integrated information, inspired by the integrated information theory (IIT) of consciousness. We computed integrated information patterns from intracranial electrocorticography (ECoG) from six human neurosurgical patients with electrodes implanted over lateral and ventral cortices. During recording, subjects viewed continuous flash suppression (CFS) and backward masking (BM) stimuli intended to dissociate conscious percept from stimulus, and unmasked suprathreshold stimuli. Object-sensitive areas revealed correspondence between conscious percepts and integrated information patterns. We quantified this correspondence using unsupervised classification methods that revealed clustering of visual experiences with integrated information, but not with broader information measures including mutual information and entropy. Our findings point to a significant role of locally integrated information for understanding the neural substrate of conscious object perception.