Face percept formation in human ventral temporal cortex

Face-Specific Activity in the Ventral Stream Visual Cortex Linked to Conscious Face Perception

When presented with visual stimuli of face images, the ventral stream visual cortex of the human brain exhibits face-specific activity that is modulated by the physical properties of the input images. However, it is still unclear whether this activity relates to conscious face perception. We explored this issue by using the human intracranial electroencephalography technique. Our results showed that face-specific activity in the ventral stream visual cortex was significantly higher when the subjects subjectively saw faces than when they did not, even when face stimuli were presented in both conditions. In addition, the face-specific neural activity exhibited a more reliable neural response and increased posterior-anterior direction information transfer in the "seen" condition than the "unseen" condition. Furthermore, the face-specific neural activity was significantly correlated with performance. These findings support the view that face-specific activity in the ventral stream visual cortex is linked to conscious face perception.

Solving large-scale MEG/EEG source localisation and functional connectivity problems simultaneously using state-space models

State-space models are widely employed across various research disciplines to study unobserved dynamics. Conventional estimation techniques, such as Kalman filtering and expectation maximisation, offer valuable insights but incur high computational costs in large-scale analyses. Sparse inverse covariance estimators can mitigate these costs, but at the expense of a trade-off between enforced sparsity and increased estimation bias, necessitating careful assessment in low signal-to-noise ratio (SNR) situations. To address these challenges, we propose a three-fold solution: (1) Introducing multiple penalised state-space (MPSS) models that leverage data-driven regularisation; (2) Developing novel algorithms derived from backpropagation, gradient descent, and alternating least squares to solve MPSS models; (3) Presenting a K-fold cross-validation extension for evaluating regularisation parameters. We validate this MPSS regularisation framework through lower and more complex simulations under varying SNR conditions, including a large-scale synthetic magneto- and electro-encephalography (MEG/EEG) data analysis. In addition, we apply MPSS models to concurrently solve brain source localisation and functional connectivity problems for real event-related MEG/EEG data, encompassing thousands of sources on the cortical surface. The proposed methodology overcomes the limitations of existing approaches, such as constraints to small-scale and region-of-interest analyses. Thus, it may enable a more accurate and detailed exploration of cognitive brain functions.

Electrocorticography reveals the dynamics of famous voice responses in human fusiform gyrus

Voice and face processing occur through convergent neural systems that facilitate speaker recognition. Neuroimaging studies suggest that familiar voice processing engages early visual cortex, including the bilateral fusiform gyrus (FG) on the basal temporal lobe. However, what role the FG plays in voice processing and whether it is driven by bottom-up or top-down mechanisms is unresolved. In this study we directly examined neural responses to famous voices and faces in human FG with direct cortical surface recordings (electrocorticography) in epilepsy surgery patients. We tested the hypothesis that neural populations in human FG respond to famous voices and investigated the temporal properties of voice responses in FG. Recordings were acquired from five adult participants during a person identification task using visual and auditory stimuli from famous speakers (U.S. Presidents Barack Obama, George W. Bush, and Bill Clinton). Patients were presented with images of presidents or clips of their voices and asked to identify the portrait/speaker. Our results demonstrate that a subset of face-responsive sites in and near FG also exhibit voice responses that are both lower in magnitude and delayed (300-600 ms) compared with visual responses. The dynamics of voice processing revealed by direct cortical recordings suggests a top-down feedback-mediated response to famous voices in FG that may facilitate speaker identification.NEW & NOTEWORTHY Interactions between auditory and visual cortices play an important role in person identification, but the dynamics of these interactions remain poorly understood. We performed direct brain recordings of fusiform face cortex in human epilepsy patients performing a famous voice naming task, revealing the dynamics of famous voice processing in human fusiform face cortex. The findings support a model of top-down interactions from auditory to visual cortex to facilitate famous voice recognition.

Robust estimation of 1/f activity improves oscillatory burst detection

Neural oscillations often occur as transient bursts with variable amplitude and frequency dynamics. Quantifying these effects is important for understanding brain-behaviour relationships, especially in continuous datasets. To robustly measure bursts, rhythmical periods of oscillatory activity must be separated from arrhythmical background 1/f activity, which is ubiquitous in electrophysiological recordings. The Better OSCillation (BOSC) framework achieves this by defining a power threshold above the estimated background 1/f activity, combined with a duration threshold. Here we introduce a modification to this approach called fBOSC, which uses a spectral parametrisation tool to accurately model background 1/f activity in neural data. fBOSC (which is openly available as a MATLAB toolbox) is robust to power spectra with oscillatory peaks and can also model non-linear spectra. Through a series of simulations, we show that fBOSC more accurately models the 1/f power spectrum compared with existing methods. fBOSC was especially beneficial where power spectra contained a 'knee' below ~.5-10 Hz, which is typical in neural data. We also found that, unlike other methods, fBOSC was unaffected by oscillatory peaks in the neural power spectrum. Moreover, by robustly modelling background 1/f activity, the sensitivity for detecting oscillatory bursts was standardised across frequencies (e.g., theta- and alpha-bands). Finally, using openly available resting state magnetoencephalography and intracranial electrophysiology datasets, we demonstrate the application of fBOSC for oscillatory burst detection in the theta-band. These simulations and empirical analyses highlight the value of fBOSC in detecting oscillatory bursts, including in datasets that are long and continuous with no distinct experimental trials.

Low and high frequency intracranial neural signals match in the human associative cortex

In vivo intracranial recordings of neural activity offer a unique opportunity to understand human brain function. Intracranial electrophysiological (iEEG) activity related to sensory, cognitive or motor events manifests mostly in two types of signals: event-related local field potentials in lower frequency bands (<30 Hz, LF) and broadband activity in the higher end of the frequency spectrum (>30 Hz, High frequency, HF). While most current studies rely exclusively on HF, thought to be more focal and closely related to spiking activity, the relationship between HF and LF signals is unclear, especially in human associative cortex. Here, we provide a large-scale in-depth investigation of the spatial and functional relationship between these 2 signals based on intracranial recordings from 121 individual brains (8000 recording sites). We measure category-selective responses to complex ecologically salient visual stimuli - human faces - across a wide cortical territory in the ventral occipito-temporal cortex (VOTC), with a frequency-tagging method providing high signal-to-noise ratio (SNR) and the same objective quantification of signal and noise for the two frequency ranges. While LF face-selective activity has higher SNR across the VOTC, leading to a larger number of significant electrode contacts especially in the anterior temporal lobe, LF and HF display highly similar spatial, functional, and timing properties. Specifically, and contrary to a widespread assumption, our results point to nearly identical spatial distribution and local spatial extent of LF and HF activity at equal SNR. These observations go a long way towards clarifying the relationship between the two main iEEG signals and reestablish the informative value of LF iEEG to understand human brain function.

Broadband Dynamics Rather than Frequency-Specific Rhythms Underlie Prediction Error in the Primate Auditory Cortex

Detection of statistical irregularities, measured as a prediction error response, is fundamental to the perceptual monitoring of the environment. We studied whether prediction error response is associated with neural oscillations or asynchronous broadband activity. Electrocorticography was conducted in three male monkeys, who passively listened to the auditory roving oddball stimuli. Local field potentials (LFPs) recorded over the auditory cortex underwent spectral principal component analysis, which decoupled broadband and rhythmic components of the LFP signal. We found that the broadband component captured the prediction error response, whereas none of the rhythmic components were associated with statistical irregularities of sounds. The broadband component displayed more stochastic, asymmetrical multifractal properties than the rhythmic components, which revealed more self-similar dynamics. We thus conclude that the prediction error response is captured by neuronal populations generating asynchronous broadband activity, defined by irregular dynamic states, which, unlike oscillatory rhythms, appear to enable the neural representation of auditory prediction error response.SIGNIFICANCE STATEMENT This study aimed to examine the contribution of oscillatory and asynchronous components of auditory local field potentials in the generation of prediction error responses to sensory irregularities, as this has not been directly addressed in the previous studies. Here, we show that mismatch negativity-an auditory prediction error response-is driven by the asynchronous broadband component of potentials recorded in the auditory cortex. This finding highlights the importance of nonoscillatory neural processes in the predictive monitoring of the environment. At a more general level, the study demonstrates that stochastic neural processes, which are often disregarded as neural noise, do have a functional role in the processing of sensory information.

Basis profile curve identification to understand electrical stimulation effects in human brain networks

Brain networks can be explored by delivering brief pulses of electrical current in one area while measuring voltage responses in other areas. We propose a convergent paradigm to study brain dynamics, focusing on a single brain site to observe the average effect of stimulating each of many other brain sites. Viewed in this manner, visually-apparent motifs in the temporal response shape emerge from adjacent stimulation sites. This work constructs and illustrates a data-driven approach to determine characteristic spatiotemporal structure in these response shapes, summarized by a set of unique "basis profile curves" (BPCs). Each BPC may be mapped back to underlying anatomy in a natural way, quantifying projection strength from each stimulation site using simple metrics. Our technique is demonstrated for an array of implanted brain surface electrodes in a human patient. This framework enables straightforward interpretation of single-pulse brain stimulation data, and can be applied generically to explore the diverse milieu of interactions that comprise the connectome.

The current state of electrocorticography-based brain-computer interfaces

Brain-computer interfaces (BCIs) provide a way for the brain to interface directly with a computer. Many different brain signals can be used to control a device, varying in ease of recording, reliability, stability, temporal and spatial resolution, and noise. Electrocorticography (ECoG) electrodes provide a highly reliable signal from the human brain surface, and these signals have been used to decode movements, vision, and speech. ECoG-based BCIs are being developed to provide increased options for treatment and assistive devices for patients who have functional limitations. Decoding ECoG signals in real time provides direct feedback to the patient and can be used to control a cursor on a computer or an exoskeleton. In this review, the authors describe the current state of ECoG-based BCIs that are approaching clinical viability for restoring lost communication and motor function in patients with amyotrophic lateral sclerosis or tetraplegia. These studies provide a proof of principle and the possibility that ECoG-based BCI technology may also be useful in the future for assisting in the cortical rehabilitation of patients who have suffered a stroke.

Neural Mechanism of Noise Affecting Face Recognition

Face recognition is one of the most important cognitive functions for humans in social activities. The ability will be negatively affected when the face images deteriorate. However, the neural process of extracting facial information under challenging conditions is still poorly understood. Therefore, it is necessary to further understand the neurophysiological relevance of this effect. We examined patients with multiple subdural electrodes (ECoG) monitored for clinical purposes. During the experimental task, the patients were presented with face and house images with different noise levels and were asked to recognize the faces. We found a striking increase in high gamma band power (HGP; 60-160 Hz) when face images were shown. We localized the face-specific electrodes to the fusiform gyrus (FG) and surrounding cortices. For each subject, the behavioral performance and magnitudes of the HGP for the face-specific sites significantly both fit a sigmoid function and showed similar changes. Additionally, the curve profile of the average HGP magnitude across the face-specific sites was almost equal to the average behavior curve; the former could precisely track the behavioral performance. In general, these results suggest that the HGP in the FG is closely related to the performance of face image recognition.

Intracranial Recordings Reveal Unique Shape and Timing of Responses in Human Visual Cortex during Illusory Visual Events

During binocular rivalry, perception spontaneously changes without any alteration to the visual stimulus. What neural events bring about this illusion that a constant stimulus is changing? We recorded from intracranial electrodes placed on the occipital and posterior temporal cortex of two patients with epilepsy while they experienced illusory changes of a face-house binocular-rivalry stimulus or observed a control stimulus that physically changed. We performed within-patient comparisons of broadband high-frequency responses, focusing on single epochs recorded along the ventral processing stream. We found transient face- and house-selective responses localized to the same electrodes for illusory and physical changes, but the temporal characteristics of these responses markedly differed. In comparison with physical changes, responses to illusory changes were longer lasting, in particular exhibiting a characteristic slow rise. Furthermore, the temporal order of responses across the visual hierarchy was reversed for illusory as compared to physical changes: for illusory changes, higher order fusiform and parahippocampal regions responded before lower order occipital regions. Our tentative interpretation of these findings is that two stages underlie the initiation of illusory changes: a destabilization stage in which activity associated with the impending change gradually accumulates across the visual hierarchy, ultimately graduating in a top-down cascade of activity that may stabilize the new perceptual interpretation of the stimulus.

Functional Connectivity within and beyond the Face Network Is Related to Reduced Discrimination of Degraded Faces in Young and Older Adults

Degrading face stimuli reduces face discrimination in both young and older adults, but the brain correlates of this decline in performance are not fully understood. We used functional magnetic resonance imaging to examine the effects of degraded face stimuli on face and nonface brain networks and tested whether these changes would predict the linear declines seen in performance. We found decreased activity in the face network (FN) and a decrease in the similarity of functional connectivity (FC) in the FN across conditions as degradation increased but no effect of age. FC in whole-brain networks also changed with increasing degradation, including increasing FC between the visual network and cognitive control networks. Older adults showed reduced modulation of this whole-brain FC pattern. The strongest predictors of within-participant decline in accuracy were changes in whole-brain network FC and FC similarity of the FN. There was no influence of age on these brain-behavior relations. These results suggest that a systems-level approach beyond the FN is required to understand the brain correlates of performance decline when faces are obscured with noise. In addition, the association between brain and behavior changes was maintained into older age, despite the dampened FC response to face degradation seen in older adults.

Time-varying effective EEG source connectivity: the optimization of model parameters

Adaptive estimation methods based on general Kalman filter are powerful tools to investigate brain networks dynamics given the non-stationary nature of neural signals. These methods rely on two parameters, the model order p and adaptation constant c, which determine the resolution and smoothness of the time-varying multivariate autoregressive estimates. A sub-optimal filtering may present consistent biases in the frequency domain and temporal distortions, leading to fallacious interpretations. Thus, the performance of these methods heavily depends on the accurate choice of these two parameters in the filter design. In this work, we sought to define an objective criterion for the optimal choice of these parameters. Since residual- and information-based criteria are not guaranteed to reach an absolute minimum, we propose to study the partial derivatives of these functions to guide the choice of p and c. To validate the performance of our method, we used a dataset of human visual evoked potentials during face perception where the generation and propagation of information in the brain is well understood and a set of simulated data where the ground truth is available.

A library of human electrocorticographic data and analyses

Electrophysiological data from implanted electrodes in the human brain are rare, and therefore scientific access to such data has remained somewhat exclusive. Here we present a freely available curated library of implanted electrocorticographic data and analyses for 16 behavioural experiments, with 204 individual datasets from 34 patients recorded with the same amplifiers and at the same settings. For each dataset, electrode positions were carefully registered to brain anatomy. A large set of fully annotated analysis scripts with which to interpret these data is embedded in the library alongside them. All data, anatomical locations and analysis files (MATLAB code) are provided in a shared file structure at https://searchworks.stanford.edu/view/zk881ps0522.

Estimating EEG Source Dipole Orientation Based on Singular-value Decomposition for Connectivity Analysis

In the last decade, the use of high-density electrode arrays for EEG recordings combined with the improvements of source reconstruction algorithms has allowed the investigation of brain networks dynamics at a sub-second scale. One powerful tool for investigating large-scale functional brain networks with EEG is time-varying effective connectivity applied to source signals obtained from electric source imaging. Due to computational and interpretation limitations, the brain is usually parcelled into a limited number of regions of interests (ROIs) before computing EEG connectivity. One specific need and still open problem is how to represent the time- and frequency-content carried by hundreds of dipoles with diverging orientation in each ROI with one unique representative time-series. The main aim of this paper is to provide a method to compute a signal that explains most of the variability of the data contained in each ROI before computing, for instance, time-varying connectivity. As the representative time-series for a ROI, we propose to use the first singular vector computed by a singular-value decomposition of all dipoles belonging to the same ROI. We applied this method to two real datasets (visual evoked potentials and epileptic spikes) and evaluated the time-course and the frequency content of the obtained signals. For each ROI, both the time-course and the frequency content of the proposed method reflected the expected time-course and the scalp-EEG frequency content, representing most of the variability of the sources (~ 80%) and improving connectivity results in comparison to other procedures used so far. We also confirm these results in a simulated dataset with a known ground truth.

The inferior occipital gyrus is a major cortical source of the face-evoked N170: Evidence from simultaneous scalp and intracerebral human recordings

The sudden onset of a face image leads to a prominent face-selective response in human scalp electroencephalographic (EEG) recordings, peaking 170 ms after stimulus onset at occipito-temporal (OT) scalp sites: the N170 (or M170 in magnetoencephalography). According to a widely held view, the main cortical source of the N170 lies in the fusiform gyrus (FG), whereas the posteriorly located inferior occipital gyrus (IOG) would rather generate earlier face-selective responses. Here, we report neural responses to upright and inverted faces recorded in a unique patient using multicontact intracerebral electrodes implanted in the right IOG and in the OT sulcus above the right lateral FG (LFG). Simultaneous EEG recordings on the scalp identified the N170 over the right OT scalp region. The latency and amplitude of this scalp N170 were correlated at the single-trial level with the N170 recorded in the lateral IOG, close to the scalp lateral occipital surface. In addition, a positive component maximal around the latency of the N170 (a P170) was prominent above the internal LFG, whereas this region typically generates an N170 (or "N200") over its external/ventral surface. This suggests that electrophysiological responses in the LFG manifest as an equivalent dipole oriented mostly along the vertical axis with likely minimal projection to the lateral OT scalp region. Altogether, these observations provide evidence that the IOG is a major cortical generator of the face-selective scalp N170, qualifying the potential contribution of the FG and questioning a strict serial spatiotemporal organization of the human cortical face network.