Spontaneous Decoding of the Timing and Content of Human Object Perception from Cortical Surface Recordings Reveals Complementary Information in the Event-Related Potential and Broadband Spectral Change

Interacting ventral temporal gradients of timescales and functional connectivity and their relationships to visual behavior

Cortical gradients in endogenous and stimulus-evoked neurodynamic timescales, and long-range cortical interactions, provide organizational constraints to the brain and influence neural populations' roles in cognition. It is unclear how these functional gradients interrelate and which influence behavior. Here, intracranial recordings from 4,090 electrode contacts in 35 individuals map gradients of neural timescales and functional connectivity to assess their interactions along category-selective ventral temporal cortex. Endogenous and stimulus-evoked information processing timescales were not significantly correlated with one another suggesting that local neural timescales are context dependent and may arise through distinct neurophysiological mechanisms. Endogenous neural timescales correlated with functional connectivity even after removing the effects of shared anatomical gradients. Neural timescales and functional connectivity correlated with how strongly a population's activity predicted behavior in a simple visual task. These results suggest both interrelated and distinct neurophysiological processes give rise to different functional connectivity and neural timescale gradients, which together influence behavior.

Flexible multi-step hypothesis testing of human ECoG data using cluster-based permutation tests with GLMEs

BACKGROUND

Time series analysis is critical for understanding brain signals and their relationship to behavior and cognition. Cluster-based permutation tests (CBPT) are commonly used to analyze a variety of electrophysiological signals including EEG, MEG, ECoG, and sEEG data without a priori assumptions about specific temporal effects. However, two major limitations of CBPT include the inability to directly analyze experiments with multiple fixed effects and the inability to account for random effects (e.g. variability across subjects). Here, we propose a flexible multi-step hypothesis testing strategy using CBPT with Linear Mixed Effects Models (LMEs) and Generalized Linear Mixed Effects Models (GLMEs) that can be applied to a wide range of experimental designs and data types.

METHODS

We first evaluate the statistical robustness of LMEs and GLMEs using simulated data distributions. Second, we apply a multi-step hypothesis testing strategy to analyze ERPs and broadband power signals extracted from human ECoG recordings collected during a simple image viewing experiment with image category and novelty as fixed effects. Third, we assess the statistical power differences between analyzing signals with CBPT using LMEs compared to CBPT using separate t-tests run on each fixed effect through simulations that emulate broadband power signals. Finally, we apply CBPT using GLMEs to high-gamma burst data to demonstrate the extension of the proposed method to the analysis of nonlinear data.

RESULTS

First, we found that LMEs and GLMEs are robust statistical models. In simple simulations LMEs produced highly congruent results with other appropriately applied linear statistical models, but LMEs outperformed many linear statistical models in the analysis of "suboptimal" data and maintained power better than analyzing individual fixed effects with separate t-tests. GLMEs also performed similarly to other nonlinear statistical models. Second, in real world human ECoG data, LMEs performed at least as well as separate t-tests when applied to predefined time windows or when used in conjunction with CBPT. Additionally, fixed effects time courses extracted with CBPT using LMEs from group-level models of pseudo-populations replicated latency effects found in individual category-selective channels. Third, analysis of simulated broadband power signals demonstrated that CBPT using LMEs was superior to CBPT using separate t-tests in identifying time windows with significant fixed effects especially for small effect sizes. Lastly, the analysis of high-gamma burst data using CBPT with GLMEs produced results consistent with CBPT using LMEs applied to broadband power data.

CONCLUSIONS

We propose a general approach for statistical analysis of electrophysiological data using CBPT in conjunction with LMEs and GLMEs. We demonstrate that this method is robust for experiments with multiple fixed effects and applicable to the analysis of linear and nonlinear data. Our methodology maximizes the statistical power available in a dataset across multiple experimental variables while accounting for hierarchical random effects and controlling FWER across fixed effects. This approach substantially improves power leading to better reproducibility. Additionally, CBPT using LMEs and GLMEs can be used to analyze individual channels or pseudo-population data for the comparison of functional or anatomical groups of data.

Common Sequential Organization of Face Processing in the Human Brain and Convolutional Neural Networks

Face processing includes two crucial processing levels - face detection and face recognition. However, it remains unclear how human brains organize the two processing levels sequentially. While some studies found that faces are recognized as fast as they are detected, others have reported that faces are detected first, followed by recognition. We discriminated the two processing levels on a fine time scale by combining human intracranial EEG (two females, three males, and three subjects without reported sex information) and representation similarity analysis. Our results demonstrate that the human brain exhibits a "detection-first, recognition-later" pattern during face processing. In addition, we used convolutional neural networks to test the hypothesis that the sequential organization of the two face processing levels in the brain reflects computational optimization. Our findings showed that the networks trained on face recognition also exhibited the "detection-first, recognition-later" pattern. Moreover, this sequential organization mechanism developed gradually during the training of the networks and was observed only for correctly predicted images. These findings collectively support the computational account as to why the brain organizes them in this way.

Face-Specific Activity in the Ventral Stream Visual Cortex Linked to Conscious Face Perception

When presented with visual stimuli of face images, the ventral stream visual cortex of the human brain exhibits face-specific activity that is modulated by the physical properties of the input images. However, it is still unclear whether this activity relates to conscious face perception. We explored this issue by using the human intracranial electroencephalography technique. Our results showed that face-specific activity in the ventral stream visual cortex was significantly higher when the subjects subjectively saw faces than when they did not, even when face stimuli were presented in both conditions. In addition, the face-specific neural activity exhibited a more reliable neural response and increased posterior-anterior direction information transfer in the "seen" condition than the "unseen" condition. Furthermore, the face-specific neural activity was significantly correlated with performance. These findings support the view that face-specific activity in the ventral stream visual cortex is linked to conscious face perception.

Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.

Characterization of High-Gamma Activity in Electrocorticographic Signals

Introduction

Electrocorticographic (ECoG) high-gamma activity (HGA) is a widely recognized and robust neural correlate of cognition and behavior. However, fundamental signal properties of HGA, such as the high-gamma frequency band or temporal dynamics of HGA, have never been systematically characterized. As a result, HGA estimators are often poorly adjusted, such that they miss valuable physiological information.

Methods

To address these issues, we conducted a thorough qualitative and quantitative characterization of HGA in ECoG signals. Our study is based on ECoG signals recorded from 18 epilepsy patients while performing motor control, listening, and visual perception tasks. In this study, we first categorize HGA into HGA types based on the cognitive/behavioral task. For each HGA type, we then systematically quantify three fundamental signal properties of HGA: the high-gamma frequency band, the HGA bandwidth, and the temporal dynamics of HGA.

Results

The high-gamma frequency band strongly varies across subjects and across cognitive/behavioral tasks. In addition, HGA time courses have lowpass character, with transients limited to 10 Hz. The task-related rise time and duration of these HGA time courses depend on the individual subject and cognitive/behavioral task. Task-related HGA amplitudes are comparable across the investigated tasks.

Discussion

This study is of high practical relevance because it provides a systematic basis for optimizing experiment design, ECoG acquisition and processing, and HGA estimation. Our results reveal previously unknown characteristics of HGA, the physiological principles of which need to be investigated in further studies.

Canonical Response Parameterization: Quantifying the structure of responses to single-pulse intracranial electrical brain stimulation

Single-pulse electrical stimulation in the nervous system, often called cortico-cortical evoked potential (CCEP) measurement, is an important technique to understand how brain regions interact with one another. Voltages are measured from implanted electrodes in one brain area while stimulating another with brief current impulses separated by several seconds. Historically, researchers have tried to understand the significance of evoked voltage polyphasic deflections by visual inspection, but no general-purpose tool has emerged to understand their shapes or describe them mathematically. We describe and illustrate a new technique to parameterize brain stimulation data, where voltage response traces are projected into one another using a semi-normalized dot product. The length of timepoints from stimulation included in the dot product is varied to obtain a temporal profile of structural significance, and the peak of the profile uniquely identifies the duration of the response. Using linear kernel PCA, a canonical response shape is obtained over this duration, and then single-trial traces are parameterized as a projection of this canonical shape with a residual term. Such parameterization allows for dissimilar trace shapes from different brain areas to be directly compared by quantifying cross-projection magnitudes, response duration, canonical shape projection amplitudes, signal-to-noise ratios, explained variance, and statistical significance. Artifactual trials are automatically identified by outliers in sub-distributions of cross-projection magnitude, and rejected. This technique, which we call "Canonical Response Parameterization" (CRP) dramatically simplifies the study of CCEP shapes, and may also be applied in a wide range of other settings involving event-triggered data.

Flexible Multi-Step Hypothesis Testing of Human ECoG Data using Cluster-based Permutation Tests with GLMEs

Background

Time series analysis is critical for understanding brain signals and their relationship to behavior and cognition. Cluster-based permutation tests (CBPT) are commonly used to analyze a variety of electrophysiological signals including EEG, MEG, ECoG, and sEEG data without a priori assumptions about specific temporal effects. However, two major limitations of CBPT include the inability to directly analyze experiments with multiple fixed effects and the inability to account for random effects (e.g. variability across subjects). Here, we propose a flexible multi-step hypothesis testing strategy using CBPT with Linear Mixed Effects Models (LMEs) and Generalized Linear Mixed Effects Models (GLMEs) that can be applied to a wide range of experimental designs and data types.

Methods

We first evaluate the statistical robustness of LMEs and GLMEs using simulated data distributions. Second, we apply a multi-step hypothesis testing strategy to analyze ERPs and broadband power signals extracted from human ECoG recordings collected during a simple image viewing experiment with image category and novelty as fixed effects. Third, we assess the statistical power differences between analyzing signals with CBPT using LMEs compared to CBPT using separate t-tests run on each fixed effect through simulations that emulate broadband power signals. Finally, we apply CBPT using GLMEs to high-gamma burst data to demonstrate the extension of the proposed method to the analysis of nonlinear data.

Results

First, we found that LMEs and GLMEs are robust statistical models. In simple simulations LMEs produced highly congruent results with other appropriately applied linear statistical models, but LMEs outperformed many linear statistical models in the analysis of "suboptimal" data and maintained power better than analyzing individual fixed effects with separate t-tests. GLMEs also performed similarly to other nonlinear statistical models. Second, in real world human ECoG data, LMEs performed at least as well as separate t-tests when applied to predefined time windows or when used in conjunction with CBPT. Additionally, fixed effects time courses extracted with CBPT using LMEs from group-level models of pseudo-populations replicated latency effects found in individual category-selective channels. Third, analysis of simulated broadband power signals demonstrated that CBPT using LMEs was superior to CBPT using separate t-tests in identifying time windows with significant fixed effects especially for small effect sizes. Lastly, the analysis of high-gamma burst data using CBPT with GLMEs produced results consistent with CBPT using LMEs applied to broadband power data.

Conclusions

We propose a general approach for statistical analysis of electrophysiological data using CBPT in conjunction with LMEs and GLMEs. We demonstrate that this method is robust for experiments with multiple fixed effects and applicable to the analysis of linear and nonlinear data. Our methodology maximizes the statistical power available in a dataset across multiple experimental variables while accounting for hierarchical random effects and controlling FWER across fixed effects. This approach substantially improves power and accuracy leading to better reproducibility. Additionally, CBPT using LMEs and GLMEs can be used to analyze individual channels or pseudo-population data for the comparison of functional or anatomical groups of data.

Intracerebral Electrophysiological Recordings to Understand the Neural Basis of Human Face Recognition

Understanding how the human brain recognizes faces is a primary scientific goal in cognitive neuroscience. Given the limitations of the monkey model of human face recognition, a key approach in this endeavor is the recording of electrophysiological activity with electrodes implanted inside the brain of human epileptic patients. However, this approach faces a number of challenges that must be overcome for meaningful scientific knowledge to emerge. Here we synthesize a 10 year research program combining the recording of intracerebral activity (StereoElectroEncephaloGraphy, SEEG) in the ventral occipito-temporal cortex (VOTC) of large samples of participants and fast periodic visual stimulation (FPVS), to objectively define, quantify, and characterize the neural basis of human face recognition. These large-scale studies reconcile the wide distribution of neural face recognition activity with its (right) hemispheric and regional specialization and extend face-selectivity to anterior regions of the VOTC, including the ventral anterior temporal lobe (VATL) typically affected by magnetic susceptibility artifacts in functional magnetic resonance imaging (fMRI). Clear spatial dissociations in category-selectivity between faces and other meaningful stimuli such as landmarks (houses, medial VOTC regions) or written words (left lateralized VOTC) are found, confirming and extending neuroimaging observations while supporting the validity of the clinical population tested to inform about normal brain function. The recognition of face identity - arguably the ultimate form of recognition for the human brain - beyond mere differences in physical features is essentially supported by selective populations of neurons in the right inferior occipital gyrus and the lateral portion of the middle and anterior fusiform gyrus. In addition, low-frequency and high-frequency broadband iEEG signals of face recognition appear to be largely concordant in the human association cortex. We conclude by outlining the challenges of this research program to understand the neural basis of human face recognition in the next 10 years.

Multimodal Electrocorticogram Active Electrode Array Based on Zinc Oxide-Thin Film Transistors

Active electrocorticogram (ECoG) electrodes can amplify weak electrophysiological signals and improve anti-interference ability; however, traditional active electrodes are opaque and cannot realize photoelectric collaborative observation. In this study, an active and fully transparent ECoG array based on zinc oxide thin-film transistors (ZnO TFTs) is developed as a local neural signal amplifier for electrophysiological monitoring. The transparency of the proposed ECoG array is up to 85%, which is superior to that of the previously reported active electrode arrays. Various electrical characterizations have demonstrated its ability to record electrophysiological signals with a higher signal-to-noise ratio of 19.9 dB compared to the Au grid (13.2 dB). The high transparency of the ZnO-TFT electrode array allows the concurrent collection of high-quality electrophysiological signals (32.2 dB) under direct optical stimulation of the optogenetic mice brain. The ECoG array can also work under 7-Tesla magnetic resonance imaging to record local brain signals without affecting brain tissue imaging. As the most transparent active ECoG array to date, it provides a powerful multimodal tool for brain observation, including recording brain activity under synchronized optical modulation and 7-Tesla magnetic resonance imaging.

Phase-amplitude coupling-based adaptive filters for neural signal decoding

Bandpass filters play a core role in ECoG signal processing. Commonly used frequency bands such as alpha, beta, and gamma bands can reflect the normal rhythm of the brain. However, the universally predefined bands might not be optimal for a specific task. Especially the gamma band usually covers a wide frequency span (i.e., 30-200 Hz) which can be too coarse to capture features that appear in narrow bands. An ideal option is to find the optimal frequency bands for specific tasks in real-time and dynamically. To tackle this problem, we propose an adaptive band filter that selects the useful frequency band in a data-driven way. Specifically, we leverage the phase-amplitude coupling (PAC) of the coupled working mechanism of synchronizing neuron and pyramidal neurons in neuronal oscillations, in which the phase of slower oscillations modulates the amplitude of faster ones, to help locate the fine frequency bands from the gamma range, in a task-specific and individual-specific way. Thus, the information can be more precisely extracted from ECoG signals to improve neural decoding performance. Based on this, an end-to-end decoder (PACNet) is proposed to construct a neural decoding application with adaptive filter banks in a uniform framework. Experiments show that PACNet can improve neural decoding performance universally with different tasks.

Low and high frequency intracranial neural signals match in the human associative cortex

In vivo intracranial recordings of neural activity offer a unique opportunity to understand human brain function. Intracranial electrophysiological (iEEG) activity related to sensory, cognitive or motor events manifests mostly in two types of signals: event-related local field potentials in lower frequency bands (<30 Hz, LF) and broadband activity in the higher end of the frequency spectrum (>30 Hz, High frequency, HF). While most current studies rely exclusively on HF, thought to be more focal and closely related to spiking activity, the relationship between HF and LF signals is unclear, especially in human associative cortex. Here, we provide a large-scale in-depth investigation of the spatial and functional relationship between these 2 signals based on intracranial recordings from 121 individual brains (8000 recording sites). We measure category-selective responses to complex ecologically salient visual stimuli - human faces - across a wide cortical territory in the ventral occipito-temporal cortex (VOTC), with a frequency-tagging method providing high signal-to-noise ratio (SNR) and the same objective quantification of signal and noise for the two frequency ranges. While LF face-selective activity has higher SNR across the VOTC, leading to a larger number of significant electrode contacts especially in the anterior temporal lobe, LF and HF display highly similar spatial, functional, and timing properties. Specifically, and contrary to a widespread assumption, our results point to nearly identical spatial distribution and local spatial extent of LF and HF activity at equal SNR. These observations go a long way towards clarifying the relationship between the two main iEEG signals and reestablish the informative value of LF iEEG to understand human brain function.

Neural decoding of semantic concepts: A systematic literature review

Objective Semantic concepts are coherent entities within our minds. They underpin our thought processes and are a part of the basis for our understanding of the world. Modern neuroscience research is increasingly exploring how individual semantic concepts are encoded within our brains and a number of studies are beginning to reveal key patterns of neural activity that underpin specific concepts. Building upon this basic understanding of the process of semantic neural encoding, neural engineers are beginning to explore tools and methods for semantic decoding: identifying which semantic concepts an individual is focused on at a given moment in time from recordings of their neural activity. In this paper we review the current literature on semantic neural decoding. Approach We conducted this review according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. Specifically, we assess the eligibility of published peer-reviewed reports via a search of PubMed and Google Scholar. We identify a total of 74 studies in which semantic neural decoding is used to attempt to identify individual semantic concepts from neural activity. Results Our review reveals how modern neuroscientific tools have been developed to allow decoding of individual concepts from a range of neuroimaging modalities. We discuss specific neuroimaging methods, experimental designs, and machine learning pipelines that are employed to aid the decoding of semantic concepts. We quantify the efficacy of semantic decoders by measuring information transfer rates. We also discuss current challenges presented by this research area and present some possible solutions. Finally, we discuss some possible emerging and speculative future directions for this research area. Significance Semantic decoding is a rapidly growing area of research. However, despite its increasingly widespread popularity and use in neuroscientific research this is the first literature review focusing on this topic across neuroimaging modalities and with a focus on quantifying the efficacy of semantic decoders.

Spontaneous State Detection Using Time-Frequency and Time-Domain Features Extracted From Stereo-Electroencephalography Traces

As a minimally invasive recording technique, stereo-electroencephalography (SEEG) measures intracranial signals directly by inserting depth electrodes shafts into the human brain, and thus can capture neural activities in both cortical layers and subcortical structures. Despite gradually increasing SEEG-based brain-computer interface (BCI) studies, the features utilized were usually confined to the amplitude of the event-related potential (ERP) or band power, and the decoding capabilities of other time-frequency and time-domain features have not been demonstrated for SEEG recordings yet. In this study, we aimed to verify the validity of time-domain and time-frequency features of SEEG, where classification performances served as evaluating indicators. To do this, using SEEG signals under intermittent auditory stimuli, we extracted features including the average amplitude, root mean square, slope of linear regression, and line-length from the ERP trace and three traces of band power activities (high-gamma, beta, and alpha). These features were used to detect the active state (including activations to two types of names) against the idle state. Results suggested that valid time-domain and time-frequency features distributed across multiple regions, including the temporal lobe, parietal lobe, and deeper structures such as the insula. Among all feature types, the average amplitude, root mean square, and line-length extracted from high-gamma (60–140 Hz) power and the line-length extracted from ERP were the most informative. Using a hidden Markov model (HMM), we could precisely detect the onset and the end of the active state with a sensitivity of 95.7 ± 1.3% and a precision of 91.7 ± 1.6%. The valid features derived from high-gamma power and ERP in this work provided new insights into the feature selection procedure for further SEEG-based BCI applications.

Workshops of the Eighth International Brain-Computer Interface Meeting: BCIs: The Next Frontier

The Eighth International Brain-Computer Interface (BCI) Meeting was held June 7-9th, 2021 in a virtual format. The conference continued the BCI Meeting series' interactive nature with 21 workshops covering topics in BCI (also called brain-machine interface) research. As in the past, workshops covered the breadth of topics in BCI. Some workshops provided detailed examinations of specific methods, hardware, or processes. Others focused on specific BCI applications or user groups. Several workshops continued consensus building efforts designed to create BCI standards and increase the ease of comparisons between studies and the potential for meta-analysis and large multi-site clinical trials. Ethical and translational considerations were both the primary topic for some workshops or an important secondary consideration for others. The range of BCI applications continues to expand, with more workshops focusing on approaches that can extend beyond the needs of those with physical impairments. This paper summarizes each workshop, provides background information and references for further study, presents an overview of the discussion topics, and describes the conclusion, challenges, or initiatives that resulted from the interactions and discussion at the workshop.

Cortical correlation structure of aperiodic neuronal population activity

Electrophysiological population signals contain oscillatory and non-oscillatory aperiodic (1/frequency-like) components. So far research has largely focused on oscillatory activity, and only recently, interest in aperiodic population activity has gained momentum. Accordingly, while the cortical correlation structure of oscillatory population activity has been characterized, little is known about the correlation of aperiodic neuronal activity. To address this, we investigated aperiodic neuronal population activity in the human brain using resting-state magnetoencephalography (MEG). We combined source-analysis, signal orthogonalization and irregular-resampling auto-spectral analysis (IRASA) to systematically characterize the cortical distribution and correlation of aperiodic neuronal activity. We found that aperiodic population activity is robustly correlated across the cortex and that this correlation is spatially well structured. Furthermore, we found that the cortical correlation structure of aperiodic activity is similar but distinct from the correlation structure of oscillatory neuronal activity. Anterior cortical regions showed the strongest differences between oscillatory and aperiodic correlation patterns. Our results suggest that correlations of aperiodic population activity serve as robust markers of cortical network interactions. Furthermore, our results show that aperiodic and oscillatory signal components provide non-redundant information about large-scale neuronal correlations. This may reflect at least partly distinct neuronal mechanisms underlying and reflected by oscillatory and aperiodic neuronal population activity.

A canonical visualization tool for SEEG electrodes

Stereoencephalographic (SEEG) electrodes are clinically implanted into the brains of patients with refractory epilepsy to locate foci of seizure onset. They are increasingly used in neurophysiology research to determine focal human brain activity in response to tasks or stimuli. Clear visualization of SEEG electrode location with respect to patient anatomy on magnetic resonance image (MRI) scan is vital to neuroscientific understanding. An intuitive way to accomplish this is to plot brain activity and labels at electrode locations on closest MRI slices along the canonical axial, coronal, and sagittal planes. Therefore, we've developed an open-source software tool in Matlab for visualizing SEEG electrode positions, determined from computed tomography (CT), onto canonical planes of resliced brain MRI. The code and graphical user interface are available at: https://github.com/MultimodalNeuroimagingLab/mnl_seegviewClinical Relevance- This tool enables precise communication of SEEG electrode activity and location by visualization on slices of MRI in canonical axial, coronal, and sagittal planes.

Basis profile curve identification to understand electrical stimulation effects in human brain networks

Brain networks can be explored by delivering brief pulses of electrical current in one area while measuring voltage responses in other areas. We propose a convergent paradigm to study brain dynamics, focusing on a single brain site to observe the average effect of stimulating each of many other brain sites. Viewed in this manner, visually-apparent motifs in the temporal response shape emerge from adjacent stimulation sites. This work constructs and illustrates a data-driven approach to determine characteristic spatiotemporal structure in these response shapes, summarized by a set of unique "basis profile curves" (BPCs). Each BPC may be mapped back to underlying anatomy in a natural way, quantifying projection strength from each stimulation site using simple metrics. Our technique is demonstrated for an array of implanted brain surface electrodes in a human patient. This framework enables straightforward interpretation of single-pulse brain stimulation data, and can be applied generically to explore the diverse milieu of interactions that comprise the connectome.

Brain-Computer Interface: Advancement and Challenges

Brain-Computer Interface (BCI) is an advanced and multidisciplinary active research domain based on neuroscience, signal processing, biomedical sensors, hardware, etc. Since the last decades, several groundbreaking research has been conducted in this domain. Still, no comprehensive review that covers the BCI domain completely has been conducted yet. Hence, a comprehensive overview of the BCI domain is presented in this study. This study covers several applications of BCI and upholds the significance of this domain. Then, each element of BCI systems, including techniques, datasets, feature extraction methods, evaluation measurement matrices, existing BCI algorithms, and classifiers, are explained concisely. In addition, a brief overview of the technologies or hardware, mostly sensors used in BCI, is appended. Finally, the paper investigates several unsolved challenges of the BCI and explains them with possible solutions.

Enhancing oscillations in intracranial electrophysiological recordings with data-driven spatial filters

In invasive electrophysiological recordings, a variety of neural oscillations can be detected across the cortex, with overlap in space and time. This overlap complicates measurement of neural oscillations using standard referencing schemes, like common average or bipolar referencing. Here, we illustrate the effects of spatial mixing on measuring neural oscillations in invasive electrophysiological recordings and demonstrate the benefits of using data-driven referencing schemes in order to improve measurement of neural oscillations. We discuss referencing as the application of a spatial filter. Spatio-spectral decomposition is used to estimate data-driven spatial filters, a computationally fast method which specifically enhances signal-to-noise ratio for oscillations in a frequency band of interest. We show that application of these data-driven spatial filters has benefits for data exploration, investigation of temporal dynamics and assessment of peak frequencies of neural oscillations. We demonstrate multiple use cases, exploring between-participant variability in presence of oscillations, spatial spread and waveform shape of different rhythms as well as narrowband noise removal with the aid of spatial filters. We find high between-participant variability in the presence of neural oscillations, a large variation in spatial spread of individual rhythms and many non-sinusoidal rhythms across the cortex. Improved measurement of cortical rhythms will yield better conditions for establishing links between cortical activity and behavior, as well as bridging scales between the invasive intracranial measurements and noninvasive macroscale scalp measurements.

Multiple Adjoining Word- and Face-Selective Regions in Ventral Temporal Cortex Exhibit Distinct Dynamics

The map of category-selectivity in human ventral temporal cortex (VTC) provides organizational constraints to models of object recognition. One important principle is lateral-medial response biases to stimuli that are typically viewed in the center or periphery of the visual field. However, little is known about the relative temporal dynamics and location of regions that respond preferentially to stimulus classes that are centrally viewed, such as the face- and word-processing networks. Here, word- and face-selective regions within VTC were mapped using intracranial recordings from 36 patients. Partially overlapping, but also anatomically dissociable patches of face- and word-selectivity, were found in VTC. In addition to canonical word-selective regions along the left posterior occipitotemporal sulcus, selectivity was also located medial and anterior to face-selective regions on the fusiform gyrus at the group level and within individual male and female subjects. These regions were replicated using 7 Tesla fMRI in healthy subjects. Left hemisphere word-selective regions preceded right hemisphere responses by 125 ms, potentially reflecting the left hemisphere bias for language, with no hemispheric difference in face-selective response latency. Word-selective regions along the posterior fusiform responded first, then spread medially and laterally, then anteriorally. Face-selective responses were first seen in posterior fusiform regions bilaterally, then proceeded anteriorally from there. For both words and faces, the relative delay between regions was longer than would be predicted by purely feedforward models of visual processing. The distinct time courses of responses across these regions, and between hemispheres, suggest that a complex and dynamic functional circuit supports face and word perception.SIGNIFICANCE STATEMENT Representations of visual objects in the human brain have been shown to be organized by several principles, including whether those objects tend to be viewed centrally or peripherally in the visual field. However, it remains unclear how regions that process objects that are viewed centrally, such as words and faces, are organized relative to one another. Here, invasive and noninvasive neuroimaging suggests that there is a mosaic of regions in ventral temporal cortex that respond selectively to either words or faces. These regions display differences in the strength and timing of their responses, both within and between brain hemispheres, suggesting that they play different roles in perception. These results illuminate extended, bilateral, and dynamic brain pathways that support face perception and reading.

The current state of electrocorticography-based brain-computer interfaces

Brain-computer interfaces (BCIs) provide a way for the brain to interface directly with a computer. Many different brain signals can be used to control a device, varying in ease of recording, reliability, stability, temporal and spatial resolution, and noise. Electrocorticography (ECoG) electrodes provide a highly reliable signal from the human brain surface, and these signals have been used to decode movements, vision, and speech. ECoG-based BCIs are being developed to provide increased options for treatment and assistive devices for patients who have functional limitations. Decoding ECoG signals in real time provides direct feedback to the patient and can be used to control a cursor on a computer or an exoskeleton. In this review, the authors describe the current state of ECoG-based BCIs that are approaching clinical viability for restoring lost communication and motor function in patients with amyotrophic lateral sclerosis or tetraplegia. These studies provide a proof of principle and the possibility that ECoG-based BCI technology may also be useful in the future for assisting in the cortical rehabilitation of patients who have suffered a stroke.

Self-Related Stimuli Decoding With Auditory and Visual Modalities Using Stereo-Electroencephalography

Name recognition plays important role in self-related cognitive processes and also contributes to a variety of clinical applications, such as autism spectrum disorder diagnosis and consciousness disorder analysis. However, most previous name-related studies usually adopted noninvasive EEG or fMRI recordings, which were limited by low spatial resolution and temporal resolution, respectively, and thus millisecond-level response latencies in precise brain regions could not be measured using these noninvasive recordings. By invasive stereo-electroencephalography (SEEG) recordings that have high resolution in both the spatial and temporal domain, the current study distinguished the neural response to one's own name or a stranger's name, and explored common active brain regions in both auditory and visual modalities. The neural activities were classified using spatiotemporal features of high-gamma, beta, and alpha band. Results showed that different names could be decoded using multi-region SEEG signals, and the best classification performance was achieved at high gamma (60–145 Hz) band. In this case, auditory and visual modality-based name classification accuracies were 84.5 ± 8.3 and 79.9 ± 4.6%, respectively. Additionally, some single regions such as the supramarginal gyrus, middle temporal gyrus, and insula could also achieve remarkable accuracies for both modalities, supporting their roles in the processing of self-related information. The average latency of the difference between the two responses in these precise regions was 354 ± 63 and 285 ± 59 ms in the auditory and visual modality, respectively. This study suggested that name recognition was attributed to a distributed brain network, and the subsets with decoding capabilities might be potential implanted regions for awareness detection and cognition evaluation.

Cognitive and Emotional Mapping With SEEG

Mapping of cortical functions is critical for the best clinical care of patients undergoing epilepsy and tumor surgery, but also to better understand human brain function and connectivity. The purpose of this review is to explore existing and potential means of mapping higher cortical functions, including stimulation mapping, passive mapping, and connectivity analyses. We examine the history of mapping, differences between subdural and stereoelectroencephalographic approaches, and some risks and safety aspects, before examining different types of functional mapping. Much of this review explores the prospects for new mapping approaches to better understand other components of language, memory, spatial skills, executive, and socio-emotional functions. We also touch on brain-machine interfaces, philosophical aspects of aligning tasks to brain circuits, and the study of consciousness. We end by discussing multi-modal testing and virtual reality approaches to mapping higher cortical functions.

Posterior Fusiform and Midfusiform Contribute to Distinct Stages of Facial Expression Processing

Though the fusiform is well-established as a key node in the face perception network, its role in facial expression processing remains unclear, due to competing models and discrepant findings. To help resolve this debate, we recorded from 17 subjects with intracranial electrodes implanted in face sensitive patches of the fusiform. Multivariate classification analysis showed that facial expression information is represented in fusiform activity and in the same regions that represent identity, though with a smaller effect size. Examination of the spatiotemporal dynamics revealed a functional distinction between posterior fusiform and midfusiform expression coding, with posterior fusiform showing an early peak of facial expression sensitivity at around 180 ms after subjects viewed a face and midfusiform showing a later and extended peak between 230 and 460 ms. These results support the hypothesis that the fusiform plays a role in facial expression perception and highlight a qualitative functional distinction between processing in posterior fusiform and midfusiform, with each contributing to temporally segregated stages of expression perception.

Endogenous activity modulates stimulus and circuit-specific neural tuning and predicts perceptual behavior

Perception reflects not only sensory inputs, but also the endogenous state when these inputs enter the brain. Prior studies show that endogenous neural states influence stimulus processing through non-specific, global mechanisms, such as spontaneous fluctuations of arousal. It is unclear if endogenous activity influences circuit and stimulus-specific processing and behavior as well. Here we use intracranial recordings from 30 pre-surgical epilepsy patients to show that patterns of endogenous activity are related to the strength of trial-by-trial neural tuning in different visual category-selective neural circuits. The same aspects of the endogenous activity that relate to tuning in a particular neural circuit also correlate to behavioral reaction times only for stimuli from the category that circuit is selective for. These results suggest that endogenous activity can modulate neural tuning and influence behavior in a circuit- and stimulus-specific manner, reflecting a potential mechanism by which endogenous neural states facilitate and bias perception.

Intracranial Recordings Reveal Unique Shape and Timing of Responses in Human Visual Cortex during Illusory Visual Events

During binocular rivalry, perception spontaneously changes without any alteration to the visual stimulus. What neural events bring about this illusion that a constant stimulus is changing? We recorded from intracranial electrodes placed on the occipital and posterior temporal cortex of two patients with epilepsy while they experienced illusory changes of a face-house binocular-rivalry stimulus or observed a control stimulus that physically changed. We performed within-patient comparisons of broadband high-frequency responses, focusing on single epochs recorded along the ventral processing stream. We found transient face- and house-selective responses localized to the same electrodes for illusory and physical changes, but the temporal characteristics of these responses markedly differed. In comparison with physical changes, responses to illusory changes were longer lasting, in particular exhibiting a characteristic slow rise. Furthermore, the temporal order of responses across the visual hierarchy was reversed for illusory as compared to physical changes: for illusory changes, higher order fusiform and parahippocampal regions responded before lower order occipital regions. Our tentative interpretation of these findings is that two stages underlie the initiation of illusory changes: a destabilization stage in which activity associated with the impending change gradually accumulates across the visual hierarchy, ultimately graduating in a top-down cascade of activity that may stabilize the new perceptual interpretation of the stimulus.

Identifying task-relevant spectral signatures of perceptual categorization in the human cortex

Human brain has developed mechanisms to efficiently decode sensory information according to perceptual categories of high prevalence in the environment, such as faces, symbols, objects. Neural activity produced within localized brain networks has been associated with the process that integrates both sensory bottom-up and cognitive top-down information processing. Yet, how specifically the different types and components of neural responses reflect the local networks’ selectivity for categorical information processing is still unknown. In this work we train Random Forest classification models to decode eight perceptual categories from broad spectrum of human intracranial signals (4–150 Hz, 100 subjects) obtained during a visual perception task. We then analyze which of the spectral features the algorithm deemed relevant to the perceptual decoding and gain the insights into which parts of the recorded activity are actually characteristic of the visual categorization process in the human brain. We show that network selectivity for a single or multiple categories in sensory and non-sensory cortices is related to specific patterns of power increases and decreases in both low (4–50 Hz) and high (50–150 Hz) frequency bands. By focusing on task-relevant neural activity and separating it into dissociated anatomical and spectrotemporal groups we uncover spectral signatures that characterize neural mechanisms of visual category perception in human brain that have not yet been reported in the literature.

iEEG: Dura-lining electrodes

Intracranial electroencephalography (iEEG) is measured from electrodes placed in or on the brain. These measurements have an excellent signal-to-noise ratio and iEEG signals have often been used to decode brain activity or drive brain-computer interfaces (BCIs). iEEG recordings are typically done for seizure monitoring in epilepsy patients who have these electrodes placed for a clinical purpose: to localize both brain regions that are essential for function and others where seizures start. Brain regions not involved in epilepsy are thought to function normally and provide a unique opportunity to learn about human neurophysiology. Intracranial electrodes measure the aggregate activity of large neuronal populations and recorded signals contain many features. Different features are extracted by analyzing these signals in the time and frequency domain. The time domain may reveal an evoked potential at a particular time after the onset of an event. Decomposition into the frequency domain may show narrowband peaks in the spectrum at specific frequencies or broadband signal changes that span a wide range of frequencies. Broadband power increases are generally observed when a brain region is active while most other features are highly specific to brain regions, inputs, and tasks. Here we describe the spatiotemporal dynamics of several iEEG signals that have often been used to decode brain activity and drive BCIs.

Biomarkers and neuromodulation techniques in substance use disorders

Addictive disorders are a severe health concern. Conventional therapies have just moderate success and the probability of relapse after treatment remains high. Brain stimulation techniques, such as transcranial Direct Current Stimulation (tDCS) and Deep Brain Stimulation (DBS), have been shown to be effective in reducing subjectively rated substance craving. However, there are few objective and measurable parameters that reflect neural mechanisms of addictive disorders and relapse. Key electrophysiological features that characterize substance related changes in neural processing are Event-Related Potentials (ERP). These high temporal resolution measurements of brain activity are able to identify neurocognitive correlates of addictive behaviours. Moreover, ERP have shown utility as biomarkers to predict treatment outcome and relapse probability. A future direction for the treatment of addiction might include neural interfaces able to detect addiction-related neurophysiological parameters and deploy neuromodulation adapted to the identified pathological features in a closed-loop fashion. Such systems may go beyond electrical recording and stimulation to employ sensing and neuromodulation in the pharmacological domain as well as advanced signal analysis and machine learning algorithms. In this review, we describe the state-of-the-art in the treatment of addictive disorders with electrical brain stimulation and its effect on addiction-related neurophysiological markers. We discuss advanced signal processing approaches and multi-modal neural interfaces as building blocks in future bioelectronics systems for treatment of addictive disorders.

Fast temporal dynamics and causal relevance of face processing in the human temporal cortex

We measured the fast temporal dynamics of face processing simultaneously across the human temporal cortex (TC) using intracranial recordings in eight participants. We found sites with selective responses to faces clustered in the ventral TC, which responded increasingly strongly to marine animal, bird, mammal, and human faces. Both face-selective and face-active but non-selective sites showed a posterior to anterior gradient in response time and selectivity. A sparse model focusing on information from the human face-selective sites performed as well as, or better than, anatomically distributed models when discriminating faces from non-faces stimuli. Additionally, we identified the posterior fusiform site (pFUS) as causally the most relevant node for inducing distortion of conscious face processing by direct electrical stimulation. These findings support anatomically discrete but temporally distributed response profiles in the human brain and provide a new common ground for unifying the seemingly contradictory modular and distributed modes of face processing.

Spectral Encoding of Seen and Attended Object Categories in the Human Brain

Local field potentials (LFPs) encode visual information via variations in power at many frequencies. These variations are complex and depend on stimulus and cognitive state in ways that have yet to be fully characterized. Specifically, the frequencies (or combinations of frequencies) that most robustly encode specific types of visual information are not fully known. To address this knowledge gap, we used intracranial EEG to record LFPs at 858 widely distributed recording sites as human subjects (six males, five females) indicated whether briefly presented natural scenes depicted one of three attended object categories. Principal component analysis applied to power spectra of the LFPs near stimulus onset revealed a broadband component (1-100 Hz) and two narrowband components (1-8 and 8-30 Hz, respectively) that encoded information about both seen and attended categories. Interestingly, we found that seen and attended categories were not encoded with the same fidelity by these distinct spectral components. Model-based tuning and decoding analyses revealed that power variations along the broadband component were most sharply tuned and offered more accurate decoding for seen than for attended categories. Power along the narrowband delta-theta (1-8 Hz) component robustly decoded information about both seen and attended categories, while the alpha-beta (8-30 Hz) component was specialized for attention. We conclude that, when viewing natural scenes, information about the seen category is encoded via broadband and sub-gamma (<30 Hz) power variations, while the attended category is most robustly encoded in the sub-gamma range. More generally, these results suggest that power variation along different spectral components can encode qualitatively different kinds of visual information.SIGNIFICANCE STATEMENT In this article, we characterize how changes in visual stimuli depicting specific objects (cars, faces, and buildings) and changes in attention to those objects affect the frequency content of local field potentials in the human brain. In contrast to many previous studies that have investigated encoding by variations in power at high (>30 Hz) frequencies, we find that the most important variation patterns are broadband (i.e., distributed across multiple frequencies) and narrowband, but in lower frequencies (<30 Hz). Interestingly, we find that seen and attended categories are not encoded with the same fidelity by these distinct spectral encoding patterns, suggesting that power at different frequencies can encode qualitatively different kinds of information.

Workshops of the Seventh International Brain-Computer Interface Meeting: Not Getting Lost in Translation

The Seventh International Brain-Computer Interface (BCI) Meeting was held May 21-25th, 2018 at the Asilomar Conference Grounds, Pacific Grove, California, United States. The interactive nature of this conference was embodied by 25 workshops covering topics in BCI (also called brain-machine interface) research. Workshops covered foundational topics such as hardware development and signal analysis algorithms, new and imaginative topics such as BCI for virtual reality and multi-brain BCIs, and translational topics such as clinical applications and ethical assumptions of BCI development. BCI research is expanding in the diversity of applications and populations for whom those applications are being developed. BCI applications are moving toward clinical readiness as researchers struggle with the practical considerations to make sure that BCI translational efforts will be successful. This paper summarizes each workshop, providing an overview of the topic of discussion, references for additional information, and identifying future issues for research and development that resulted from the interactions and discussion at the workshop.

A library of human electrocorticographic data and analyses

Electrophysiological data from implanted electrodes in the human brain are rare, and therefore scientific access to such data has remained somewhat exclusive. Here we present a freely available curated library of implanted electrocorticographic data and analyses for 16 behavioural experiments, with 204 individual datasets from 34 patients recorded with the same amplifiers and at the same settings. For each dataset, electrode positions were carefully registered to brain anatomy. A large set of fully annotated analysis scripts with which to interpret these data is embedded in the library alongside them. All data, anatomical locations and analysis files (MATLAB code) are provided in a shared file structure at https://searchworks.stanford.edu/view/zk881ps0522.

Visual epidural field potentials possess high functional specificity in single trials

Recordings of epidural field potentials (EFPs) allow neuronal activity to be acquired over a large region of cortical tissue with minimal invasiveness. Because electrodes are placed on top of the dura and do not enter the neuronal tissue, EFPs offer intriguing options for both clinical and basic science research. On the other hand, EFPs represent the integrated activity of larger neuronal populations and possess a higher trial-by-trial variability and a reduced signal-to-noise ratio due the additional barrier of the dura. It is thus unclear whether and to what extent EFPs have sufficient spatial selectivity to allow for conclusions about the underlying functional cortical architecture, and whether single EFP trials provide enough information on the short timescales relevant for many clinical and basic neuroscience purposes. We used the high spatial resolution of primary visual cortex to address these issues and investigated the extent to which very short EFP traces allow reliable decoding of spatial information. We briefly presented different visual objects at one of nine closely adjacent locations and recorded neuronal activity with a high-density epidural multielectrode array in three macaque monkeys. With the use of receiver operating characteristics (ROC) to identify the most informative data, machine-learning algorithms provided close-to-perfect classification rates for all 27 stimulus conditions. A binary classifier applying a simple max function on ROC-selected data further showed that single trials might be classified with 100% performance even without advanced offline classifiers. Thus, although highly variable, EFPs constitute an extremely valuable source of information and offer new perspectives for minimally invasive recording of large-scale networks.NEW & NOTEWORTHY Epidural field potential (EFP) recordings provide a minimally invasive approach to investigate large-scale neural networks, but little is known about whether they possess the required specificity for basic and clinical neuroscience. By making use of the spatial selectivity of primary visual cortex, we show that single-trial information can be decoded with close-to-perfect performance, even without using advanced classifiers and based on very few data. This labels EFPs as a highly attractive and widely usable signal.

Spectral fingerprints or spectral tilt? Evidence for distinct oscillatory signatures of memory formation

Decreases in low-frequency power (2–30 Hz) alongside high-frequency power increases (>40 Hz) have been demonstrated to predict successful memory formation. Parsimoniously, this change in the frequency spectrum can be explained by one factor, a change in the tilt of the power spectrum (from steep to flat) indicating engaged brain regions. A competing view is that the change in the power spectrum contains several distinct brain oscillatory fingerprints, each serving different computations. Here, we contrast these two theories in a parallel magnetoencephalography (MEG)–intracranial electroencephalography (iEEG) study in which healthy participants and epilepsy patients, respectively, studied either familiar verbal material or unfamiliar faces. We investigated whether modulations in specific frequency bands can be dissociated in time and space and by experimental manipulation. Both MEG and iEEG data show that decreases in alpha/beta power specifically predicted the encoding of words but not faces, whereas increases in gamma power and decreases in theta power predicted memory formation irrespective of material. Critically, these different oscillatory signatures of memory encoding were evident in different brain regions. Moreover, high-frequency gamma power increases occurred significantly earlier compared to low-frequency theta power decreases. These results show that simple “spectral tilt” cannot explain common oscillatory changes and demonstrate that brain oscillations in different frequency bands serve different functions for memory encoding.

There are two competing explanations for electrophysiological signatures during cognitive processes. One assumes simultaneous increases in high frequencies paired with decreases in low frequencies, whereas the other suggests that different frequencies index separate oscillatory processes. This study reports data that support the latter view.

The inferior occipital gyrus is a major cortical source of the face-evoked N170: Evidence from simultaneous scalp and intracerebral human recordings

The sudden onset of a face image leads to a prominent face-selective response in human scalp electroencephalographic (EEG) recordings, peaking 170 ms after stimulus onset at occipito-temporal (OT) scalp sites: the N170 (or M170 in magnetoencephalography). According to a widely held view, the main cortical source of the N170 lies in the fusiform gyrus (FG), whereas the posteriorly located inferior occipital gyrus (IOG) would rather generate earlier face-selective responses. Here, we report neural responses to upright and inverted faces recorded in a unique patient using multicontact intracerebral electrodes implanted in the right IOG and in the OT sulcus above the right lateral FG (LFG). Simultaneous EEG recordings on the scalp identified the N170 over the right OT scalp region. The latency and amplitude of this scalp N170 were correlated at the single-trial level with the N170 recorded in the lateral IOG, close to the scalp lateral occipital surface. In addition, a positive component maximal around the latency of the N170 (a P170) was prominent above the internal LFG, whereas this region typically generates an N170 (or "N200") over its external/ventral surface. This suggests that electrophysiological responses in the LFG manifest as an equivalent dipole oriented mostly along the vertical axis with likely minimal projection to the lateral OT scalp region. Altogether, these observations provide evidence that the IOG is a major cortical generator of the face-selective scalp N170, qualifying the potential contribution of the FG and questioning a strict serial spatiotemporal organization of the human cortical face network.

Measuring transient phase-amplitude coupling using local mutual information

Here we demonstrate the suitability of a local mutual information measure for estimating the temporal dynamics of cross-frequency coupling (CFC) in brain electrophysiological signals. In CFC, concurrent activity streams in different frequency ranges interact and transiently couple. A particular form of CFC, phase-amplitude coupling (PAC), has raised interest given the growing amount of evidence of its possible role in healthy and pathological brain information processing. Although several methods have been proposed for PAC estimation, only a few have addressed the estimation of the temporal evolution of PAC, and these typically require a large number of experimental trials to return a reliable estimate. Here we explore the use of mutual information to estimate a PAC measure (MIPAC) in both continuous and event-related multi-trial data. To validate these two applications of the proposed method, we first apply it to a set of simulated phase-amplitude modulated signals and show that MIPAC can successfully recover the temporal dynamics of the simulated coupling in either continuous or multi-trial data. Finally, to explore the use of MIPAC to analyze data from human event-related paradigms, we apply it to an actual event-related human electrocorticographic (ECoG) data set that exhibits strong PAC, demonstrating that the MIPAC estimator can be used to successfully characterize amplitude-modulation dynamics in electrophysiological data.

Temporal coupling of field potentials and action potentials in the neocortex

The local field potential (LFP) is an aggregate measure of group neuronal activity and is often correlated with the action potentials of single neurons. In recent years, investigators have found that action potential firing rates increase during elevations in power high-frequency band oscillations (50-200 Hz range). However, action potentials also contribute to the LFP signal itself, making the spike-LFP relationship complex. Here, we examine the relationship between spike rates and LFP in varying frequency bands in rat neocortical recordings. We find that 50-180 Hz oscillations correlate most consistently with high firing rates, but that other LFP bands also carry information relating to spiking, including in some cases anti-correlations. Relatedly, we find that spiking itself and electromyographic activity contribute to LFP power in these bands. The relationship between spike rates and LFP power varies between brain states and between individual cells. Finally, we create an improved oscillation-based predictor of action potential activity by specifically utilizing information from across the entire recorded frequency spectrum of LFP. The findings illustrate both caveats and improvements to be taken into account in attempts to infer spiking activity from LFP.

Precision of Classification of Odorant Value by the Power of Olfactory Bulb Oscillations Is Altered by Optogenetic Silencing of Local Adrenergic Innervation

Neuromodulators such as noradrenaline appear to play a crucial role in learning and memory. The goal of this study was to determine the role of norepinephrine in representation of odorant identity and value by olfactory bulb oscillations in an olfactory learning task. We wanted to determine whether the different bandwidths of olfactory bulb oscillations encode information involved in associating the odor with the value, and whether norepinephrine is involved in modulating this association. To this end mice expressing halorhodopsin under the dopamine-beta-hydrolase (DBH) promoter received an optetrode implant targeted to the olfactory bulb. Mice learned to differentiate odorants in a go-no-go task. A receiver operating characteristic (ROC) analysis showed that there was development of a broadband differential rewarded vs. unrewarded odorant-induced change in the power of local field potential oscillations as the mice became proficient in discriminating between two odorants. In addition, the change in power reflected the value of the odorant rather than the identity. Furthermore, optogenetic silencing of local noradrenergic axons in the olfactory bulb altered the differential oscillatory power response to the odorants for the theta, beta, and gamma bandwidths.

Mapping face categorization in the human ventral occipitotemporal cortex with direct neural intracranial recordings

The neural basis of face categorization has been widely investigated with functional magnetic resonance imaging (fMRI), identifying a set of face-selective local regions in the ventral occipitotemporal cortex (VOTC). However, indirect recording of neural activity with fMRI is associated with large fluctuations of signal across regions, often underestimating face-selective responses in the anterior VOTC. While direct recording of neural activity with subdural grids of electrodes (electrocorticography, ECoG) or depth electrodes (stereotactic electroencephalography, SEEG) offers a unique opportunity to fill this gap in knowledge, these studies rather reveal widely distributed face-selective responses. Moreover, intracranial recordings are complicated by interindividual variability in neuroanatomy, ambiguity in definition, and quantification of responses of interest, as well as limited access to sulci with ECoG. Here, we propose to combine SEEG in large samples of individuals with fast periodic visual stimulation to objectively define, quantify, and characterize face categorization across the whole VOTC. This approach reconciles the wide distribution of neural face categorization responses with their (right) hemispheric and regional specialization, and reveals several face-selective regions in anterior VOTC sulci. We outline the challenges of this research program to understand the neural basis of face categorization and high-level visual recognition in general.

Real-time detection and discrimination of visual perception using electrocorticographic signals

OBJECTIVE

Several neuroimaging studies have demonstrated that the ventral temporal cortex contains specialized regions that process visual stimuli. This study investigated the spatial and temporal dynamics of electrocorticographic (ECoG) responses to different types and colors of visual stimulation that were presented to four human participants, and demonstrated a real-time decoder that detects and discriminates responses to untrained natural images.

APPROACH

ECoG signals from the participants were recorded while they were shown colored and greyscale versions of seven types of visual stimuli (images of faces, objects, bodies, line drawings, digits, and kanji and hiragana characters), resulting in 14 classes for discrimination (experiment I). Additionally, a real-time system asynchronously classified ECoG responses to faces, kanji and black screens presented via a monitor (experiment II), or to natural scenes (i.e. the face of an experimenter, natural images of faces and kanji, and a mirror) (experiment III). Outcome measures in all experiments included the discrimination performance across types based on broadband γ activity.

MAIN RESULTS

Experiment I demonstrated an offline classification accuracy of 72.9% when discriminating among the seven types (without color separation). Further discrimination of grey versus colored images reached an accuracy of 67.1%. Discriminating all colors and types (14 classes) yielded an accuracy of 52.1%. In experiment II and III, the real-time decoder correctly detected 73.7% responses to face, kanji and black computer stimuli and 74.8% responses to presented natural scenes.

SIGNIFICANCE

Seven different types and their color information (either grey or color) could be detected and discriminated using broadband γ activity. Discrimination performance maximized for combined spatial-temporal information. The discrimination of stimulus color information provided the first ECoG-based evidence for color-related population-level cortical broadband γ responses in humans. Stimulus categories can be detected by their ECoG responses in real time within 500 ms with respect to stimulus onset.

Reading Minds: Brain-Decoding Scientists Move Closer to Discovering the Keys to Unlock the Brain

When you see or think about an object, your brain engages in a unique pattern of activity tied specifically to that object. That's how you know a cat is a cat, and not a dog or a house or a cloud. Using functional magnetic resonance imaging (fMRI) and other techniques, scientists are not only able to measure those activity patterns but are also deciphering what each pattern means. Essentially, they are beginning to read minds.

Intracranial markers of conscious face perception in humans

Investigations of the neural basis of consciousness have greatly benefited from protocols that involve the presentation of stimuli at perceptual threshold, enabling the assessment of the patterns of brain activity that correlate with conscious perception, independently of any changes in sensory input. However, the comparison between perceived and unperceived trials would be expected to reveal not only the core neural substrate of a particular conscious perception, but also aspects of brain activity that facilitate, hinder or tend to follow conscious perception. We take a step towards the resolution of these confounds by combining an analysis of neural responses observed during the presentation of faces partially masked by Continuous Flash Suppression, and those responses observed during the unmasked presentation of faces and other images in the same subjects. We employed multidimensional classifiers to decode physical properties of stimuli or perceptual states from spectrotemporal representations of electrocorticographic signals (1071 channels in 5 subjects). Neural activity in certain face responsive areas located in both the fusiform gyrus and in the lateral-temporal/inferior-parietal cortex discriminated seen vs. unseen faces in the masked paradigm and upright faces vs. other categories in the unmasked paradigm. However, only the former discriminated upright vs. inverted faces in the unmasked paradigm. Our results suggest a prominent role for the fusiform gyrus in the configural perception of faces, and possibly other objects that are holistically processed. More generally, we advocate comparative analysis of neural recordings obtained during different, but related, experimental protocols as a promising direction towards elucidating the functional specificities of the patterns of neural activation that accompany our conscious experiences.

Face percept formation in human ventral temporal cortex

Loci in ventral temporal cortex are selectively active during viewing of faces and other objects, but it remains unclear whether these areas represent accumulation of simple visual information or processing of intact percept. We measured broadband electrocorticographic changes from implanted electrodes on the ventral temporal brain surface while showing patients noise-degraded images of faces and houses. In a subset of posterior fusiform gyrus face-selective regions, cortical activity decreased parametrically with noise increase, until the perceptual threshold was surpassed. At noise levels higher than the perceptual threshold, and for house stimuli, activity remained at baseline. We propose that this convergence of proportional and thresholded response may identify active areas where face percepts are extracted from simple visual features. These loci exist within a topological structure of face percept formation in the human ventral visual stream, preceded by category-nonselective activity in pericalcarine early visual areas and in concert with all-or-nothing activity in postperceptual subregions of the ventral temporal lobe. This topological organization suggests a physiological basis for the anatomy of face perception, explaining different perceptual deficits following temporal lobe injury.NEW & NOTEWORTHY Philosophers have puzzled for millennia about how humans build abstract conceptual objects (house/face/tool) from the simple features of the world they see around them (line/patch/lighting). Understanding the biological foundation of this process requires detailed knowledge of the spatial-temporal characteristics of cerebral cortex. By examining the physiology of the human temporal lobe via implanted electrodes while showing subjects noise-degraded images, we find that face percept formation happens in specific subregions within known face-processing areas.

Decoding and disrupting left midfusiform gyrus activity during word reading

The nature of the visual representation for words has been fiercely debated for over 150 y. We used direct brain stimulation, pre- and postsurgical behavioral measures, and intracranial electroencephalography to provide support for, and elaborate upon, the visual word form hypothesis. This hypothesis states that activity in the left midfusiform gyrus (lmFG) reflects visually organized information about words and word parts. In patients with electrodes placed directly in their lmFG, we found that disrupting lmFG activity through stimulation, and later surgical resection in one of the patients, led to impaired perception of whole words and letters. Furthermore, using machine-learning methods to analyze the electrophysiological data from these electrodes, we found that information contained in early lmFG activity was consistent with an orthographic similarity space. Finally, the lmFG contributed to at least two distinguishable stages of word processing, an early stage that reflects gist-level visual representation sensitive to orthographic statistics, and a later stage that reflects more precise representation sufficient for the individuation of orthographic word forms. These results provide strong support for the visual word form hypothesis and demonstrate that across time the lmFG is involved in multiple stages of orthographic representation.