Human intracranial recordings link suppressed transients rather than 'filling-in' to perceptual continuity across blinks

Taking consciousness for real: Increasing the ecological validity of the study of conscious vs. unconscious processes

The study of consciousness has developed well-controlled, rigorous methods for manipulating and measuring consciousness. Yet, in the process, experimental paradigms grew farther away from everyday conscious and unconscious processes, which raises the concern of ecological validity. In this review, we suggest that the field can benefit from adopting a more ecological approach, akin to other fields of cognitive science. There, this approach challenged some existing hypotheses, yielded stronger effects, and enabled new research questions. We argue that such a move is critical for studying consciousness, where experimental paradigms tend to be artificial and small effect sizes are relatively prevalent. We identify three paths for doing so-changing the stimuli and experimental settings, changing the measures, and changing the research questions themselves-and review works that have already started implementing such approaches. While acknowledging the inherent challenges, we call for increasing ecological validity in consciousness studies.

Eye blinks as a visual processing stage

Humans blink their eyes frequently during normal viewing, more often than it seems necessary for keeping the cornea well lubricated. Since the closure of the eyelid disrupts the image on the retina, eye blinks are commonly assumed to be detrimental to visual processing. However, blinks also provide luminance transients rich in spatial information to neural pathways highly sensitive to temporal changes. Here, we report that the luminance modulations from blinks enhance visual sensitivity. By coupling high-resolution eye tracking in human observers with modeling of blink transients and spectral analysis of visual input signals, we show that blinking increases the power of retinal stimulation and that this effect significantly enhances visibility despite the time lost in exposure to the external scene. We further show that, as predicted from the spectral content of input signals, this enhancement is selective for stimuli at low spatial frequencies and occurs irrespective of whether the luminance transients are actively generated or passively experienced. These findings indicate that, like eye movements, blinking acts as a computational component of a visual processing strategy that uses motor behavior to reformat spatial information into the temporal domain.

Electrophysiological signatures of inequity-dependent reward encoding in the human OFC

Social decision making requires the integration of reward valuation and social cognition systems, both dependent on the orbitofrontal cortex (OFC). How these two OFC functions interact is largely unknown. We recorded intracranial activity from the OFC of ten patients making choices in a social context where reward inequity with a social counterpart varied and could be either advantageous or disadvantageous. We find that OFC high-frequency activity (HFA; 70-150 Hz) encodes self-reward, consistent with previous reports. We also observe encoding of the social counterpart's reward, as well as the type of inequity being experienced. Additionally, we find evidence of inequity-dependent reward encoding: depending on the type of inequity, electrodes rapidly and reversibly switch between different reward-encoding profiles. These results provide direct evidence for encoding of self- and other rewards in the human OFC and highlight the dynamic nature of encoding in the OFC as a function of social context.

Neuronal correlates of eyeblinks are an expression of primary consciousness phenomena

The blinking rate far exceeds that required for moistening the cornea and changes depending on whether a person is resting or engaged in cognitive tasks. During ecological cognitive tasks (such as speaking, reading, and watching videos), blinks occur at breakpoints of attention suggesting a role in information segmentation, but the close relationship between cognition dynamics and blink timing still escapes a full understanding. The aim of the present study is to seek (1) if there is a temporal relationship between blink events and the consecutive steps of cognitive processing, and (2) if blink timing and the intensity of blink-related EEG responses are affected by task-relevance of stimuli. Our results show that, in a classical visual oddball task, (i) the occurrence of blinks is influenced by stimuli, irrespective of their relevance, (ii) blinks following relevant stimuli are only apparently delayed due to the need of finalizing a behavioural response, and (iii) stimulus relevance does not affect the intensity of the blink-related EEG response. This evidence reinforce the idea that blinks are not emitted until the last step of the processing sequence has been completed and suggests that blink-related EEG responses are generated by primary consciousness phenomena which are considered by their nature non-modulable (all-or-nothing) phenomena.

Perceptual stability reflected in neuronal pattern similarities in human visual cortex

The magnitude of neuronal activation is commonly considered a critical factor for conscious perception of visual content. However, this dogma contrasts with the phenomenon of rapid adaptation, in which the magnitude of neuronal activation drops dramatically in a rapid manner while the visual stimulus and the conscious experience it elicits remain stable. Here, we report that the profiles of multi-site activation patterns and their relational geometry-i.e., the similarity distances between activation patterns, as revealed using intracranial electroencephalographic (iEEG) recordings-are sustained during extended visual stimulation despite the major magnitude decrease. These results are compatible with the hypothesis that conscious perceptual content is associated with the neuronal pattern profiles and their similarity distances, rather than the overall activation magnitude, in human visual cortex.

Relating Pupil Diameter and Blinking to Cortical Activity and Hemodynamics across Arousal States

Arousal state affects neural activity and vascular dynamics in the cortex, with sleep associated with large changes in the local field potential and increases in cortical blood flow. We investigated the relationship between pupil diameter and blink rate with neural activity and blood volume in the somatosensory cortex in male and female unanesthetized, head-fixed mice. We monitored these variables while the mice were awake, during periods of rapid eye movement (REM), and non-rapid eye movement (NREM) sleep. Pupil diameter was smaller during sleep than in the awake state. Changes in pupil diameter were coherent with both gamma-band power and blood volume in the somatosensory cortex, but the strength and sign of this relationship varied with arousal state. We observed a strong negative correlation between pupil diameter and both gamma-band power and blood volume during periods of awake rest and NREM sleep, although the correlations between pupil diameter and these signals became positive during periods of alertness, active whisking, and REM. Blinking was associated with increases in arousal and decreases in blood volume when the mouse was asleep. Bilateral coherence in gamma-band power and in blood volume dropped following awake blinking, indicating a reset of neural and vascular activity. Using only eye metrics (pupil diameter and eye motion), we could determine the arousal state of the mouse ('Awake,' 'NREM,' 'REM') with >90% accuracy with a 5 s resolution. There is a strong relationship between pupil diameter and hemodynamics signals in mice, reflecting the pronounced effects of arousal on cerebrovascular dynamics.SIGNIFICANCE STATEMENT Determining arousal state is a critical component of any neuroscience experiment. Pupil diameter and blinking are influenced by arousal state, as are hemodynamics signals in the cortex. We investigated the relationship between cortical hemodynamics and pupil diameter and found that pupil diameter was strongly related to the blood volume in the cortex. Mice were more likely to be awake after blinking than before, and blinking resets neural activity. Pupil diameter and eye motion can be used as a reliable, noninvasive indicator of arousal state. As mice transition from wake to sleep and back again over a timescale of seconds, monitoring pupil diameter and eye motion permits the noninvasive detection of sleep events during behavioral or resting-state experiments.

Activation of brain arousal networks coincident with eye blinks during resting state

Eye-blinking has been implicated in arousal and attention. Here we test the hypothesis that blinking-moments represent arousal surges associated with activation of the ascending arousal network (AAN) and its thalamic projections. For this purpose, we explored the temporal relationship between eye-blinks and fMRI BOLD activity in AAN and thalamic nuclei, as well as whole brain cluster corrected activations during eyes-open, resting-state fMRI scanning. We show that BOLD activations in the AAN nuclei peaked prior to the eye blinks and in thalamic nuclei peaked prior to and during the blink, consistent with the role of eye blinking in arousal surges. Additionally, we showed visual cortex peak activation prior to the eye blinks, providing further evidence of the visual cortex's role in arousal, and document cerebellar peak activation post eye blinks, which might reflect downstream engagement from arousal surges.

Advances in human intracranial electroencephalography research, guidelines and good practices

Since the second-half of the twentieth century, intracranial electroencephalography (iEEG), including both electrocorticography (ECoG) and stereo-electroencephalography (sEEG), has provided an intimate view into the human brain. At the interface between fundamental research and the clinic, iEEG provides both high temporal resolution and high spatial specificity but comes with constraints, such as the individual's tailored sparsity of electrode sampling. Over the years, researchers in neuroscience developed their practices to make the most of the iEEG approach. Here we offer a critical review of iEEG research practices in a didactic framework for newcomers, as well addressing issues encountered by proficient researchers. The scope is threefold: (i) review common practices in iEEG research, (ii) suggest potential guidelines for working with iEEG data and answer frequently asked questions based on the most widespread practices, and (iii) based on current neurophysiological knowledge and methodologies, pave the way to good practice standards in iEEG research. The organization of this paper follows the steps of iEEG data processing. The first section contextualizes iEEG data collection. The second section focuses on localization of intracranial electrodes. The third section highlights the main pre-processing steps. The fourth section presents iEEG signal analysis methods. The fifth section discusses statistical approaches. The sixth section draws some unique perspectives on iEEG research. Finally, to ensure a consistent nomenclature throughout the manuscript and to align with other guidelines, e.g., Brain Imaging Data Structure (BIDS) and the OHBM Committee on Best Practices in Data Analysis and Sharing (COBIDAS), we provide a glossary to disambiguate terms related to iEEG research.

From non-conscious processing to conscious events: a minimalist approach

The minimalist approach that we develop here is a framework that allows to appreciate how non-conscious processing and conscious contents shape human cognition, broadly defined. It is composed of three simple principles. First, cognitive processes are inherently non-conscious, while their inputs and (interim) outputs may be consciously experienced. Second, non-conscious processes and elements of the cognitive architecture prioritize information for conscious experiences. Third, conscious events are composed of series of conscious contents and non-conscious processes, with increased duration leading to more opportunity for processing. The narrowness of conscious experiences is conceptualized here as a solution to the problem of channeling the plethora of non-conscious processes into action and communication processes that are largely serial. The framework highlights the importance of prioritization for consciousness, and we provide an illustrative review of three main factors that shape prioritization-stimulus strength, motivational relevance and mental accessibility. We further discuss when and how this framework (i) is compatible with previous theories, (ii) enables new understandings of established findings and models, and (iii) generates new predictions and understandings.

Resolving visual motion through perceptual gaps

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.

Local neuronal relational structures underlying the contents of human conscious experience

While most theories of consciousness posit some kind of dependence on global network activities, I consider here an alternative, localist perspective-in which localized cortical regions each underlie the emergence of a unique category of conscious experience. Under this perspective, the large-scale activation often found in the cortex is a consequence of the complexity of typical conscious experiences rather than an obligatory condition for the emergence of conscious awareness-which can flexibly shift, depending on the richness of its contents, from local to more global activation patterns. This perspective fits a massive body of human imaging, recordings, lesions and stimulation data but opens a fundamental problem: how can the information, defining each content, be derived locally in each cortical region. Here, I will discuss a solution echoing pioneering structuralist ideas in which the content of a conscious experience is defined by its relationship to all other contents within an experiential category. In neuronal terms, this relationship structure between contents is embodied by the local geometry of similarity distances between cortical activation patterns generated during each conscious experience, likely mediated via networks of local neuronal connections. Thus, in order for any conscious experience to appear in an individual's mind, two central conditions must be met. First, a specific configural pattern ("bar-code") of neuronal activity must appear within a local relational geometry, i.e. a cortical area. Second, the individual neurons underlying the activated pattern must be bound into a unified functional ensemble through a burst of recurrent neuronal firing: local "ignitions".

Neural activity in the human anterior thalamus during natural vision

In natural vision humans and other primates explore environment by active sensing, using saccadic eye movements to relocate the fovea and sample different bits of information multiple times per second. Saccades induce a phase reset of ongoing neuronal oscillations in primary and higher-order visual cortices and in the medial temporal lobe. As a result, neuron ensembles are shifted to a common state at the time visual input propagates through the system (i.e., just after fixation). The extent of the brain’s circuitry that is modulated by saccades is not yet known. Here, we evaluate the possibility that saccadic phase reset impacts the anterior nuclei of the thalamus (ANT). Using recordings in the human thalamus of three surgical patients during natural vision, we found that saccades and visual stimulus onset both modulate neural activity, but with distinct field potential morphologies. Specifically, we found that fixation-locked field potentials had a component that preceded saccade onset. It was followed by an early negativity around 50 ms after fixation onset which is significantly faster than any response to visual stimulus presentation. The timing of these events suggests that the ANT is predictively modulated before the saccadic eye movement. We also found oscillatory phase concentration, peaking at 3–4 Hz, coincident with suppression of Broadband High-frequency Activity (BHA; 80–180 Hz), both locked to fixation onset supporting the idea that neural oscillations in these nuclei are reorganized to a low excitability state right after fixation onset. These findings show that during real-world natural visual exploration neural dynamics in the human ANT is influenced by visual and oculomotor events, which supports the idea that ANT, apart from their contribution to episodic memory, also play a role in natural vision.

Mind-wandering Is Accompanied by Both Local Sleep and Enhanced Processes of Spatial Attention Allocation

Mind-wandering (MW) is a subjective, cognitive phenomenon, in which thoughts move away from the task toward an internal train of thoughts, possibly during phases of neuronal sleep-like activity (local sleep, LS). MW decreases cortical processing of external stimuli and is assumed to decouple attention from the external world. Here, we directly tested how indicators of LS, cortical processing, and attentional selection change in a pop-out visual search task during phases of MW. Participants’ brain activity was recorded using magnetoencephalography, MW was assessed via self-report using randomly interspersed probes. As expected, the performance decreased under MW. Consistent with the occurrence of LS, MW was accompanied by a decrease in high-frequency activity (HFA, 80–150 Hz) and an increase in slow wave activity (SWA, 1–6 Hz). In contrast, visual attentional selection as indexed by the N2pc component was enhanced during MW with the N2pc amplitude being directly linked to participants’ performance. This observation clearly contradicts accounts of attentional decoupling that would predict a decrease in attention-related responses to external stimuli during MW. Together, our results suggest that MW occurs during phases of LS with processes of attentional target selection being upregulated, potentially to compensate for the mental distraction during MW.

Hippocampal ripples and their coordinated dialogue with the default mode network during recent and remote recollection

Hippocampal ripples are prominent synchronization events generated by hippocampal neuronal assemblies. To date, ripples have been primarily associated with navigational memory in rodents and short-term episodic recollections in humans. Here, we uncover different profiles of ripple activity in the human hippocampus during the retrieval of recent and remote autobiographical events and semantic facts. We found that the ripple rate increased significantly before reported recall compared to control conditions. Patterns of ripple activity across multiple hippocampal sites demonstrated remarkable specificity for memory type. Intriguingly, these ripple patterns revealed a semantization dimension, in which patterns associated with autobiographical contents become similar to those of semantic memory as a function of memory age. Finally, widely distributed sites across the neocortex exhibited ripple-coupled activations during recollection, with the strongest activation found within the default mode network. Our results thus reveal a key role for hippocampal ripples in orchestrating hippocampal-cortical communication across large-scale networks involved in conscious recollection.

Blink- and saccade-related suppression effects in early visual areas of the human brain: Intracranial EEG investigations during natural viewing conditions

Blinks and saccades, both ubiquitous in natural viewing conditions, cause rapid changes of visual inputs that are hardly consciously perceived. The neural dynamics in early visual areas of the human brain underlying this remarkable visual stability are still incompletely understood. We used electrocorticography (ECoG) from electrodes directly implanted on the human early visual areas V1, V2, V3d/v, V4d/v and the fusiform gyrus to investigate blink- and saccade-related neuronal suppression effects during non-experimental, free viewing conditions. We found a characteristic, biphasic, broadband gamma power decrease-increase pattern in all investigated visual areas. During saccades, a decrease in gamma power clearly preceded eye movement onset, at least in V1. This may indicate that cortical information processing is actively suppressed in human early visual areas before and during saccades, which then possibly mediates perceptual visual suppression. The following eye movement offset-related increase in gamma power may indicate the recovery of visual perception and the resumption of visual processing.

Measuring spectrally-resolved information transfer

Information transfer, measured by transfer entropy, is a key component of distributed computation. It is therefore important to understand the pattern of information transfer in order to unravel the distributed computational algorithms of a system. Since in many natural systems distributed computation is thought to rely on rhythmic processes a frequency resolved measure of information transfer is highly desirable. Here, we present a novel algorithm, and its efficient implementation, to identify separately frequencies sending and receiving information in a network. Our approach relies on the invertible maximum overlap discrete wavelet transform (MODWT) for the creation of surrogate data in the computation of transfer entropy and entirely avoids filtering of the original signals. The approach thereby avoids well-known problems due to phase shifts or the ineffectiveness of filtering in the information theoretic setting. We also show that measuring frequency-resolved information transfer is a partial information decomposition problem that cannot be fully resolved to date and discuss the implications of this issue. Last, we evaluate the performance of our algorithm on simulated data and apply it to human magnetoencephalography (MEG) recordings and to local field potential recordings in the ferret. In human MEG we demonstrate top-down information flow in temporal cortex from very high frequencies (above 100Hz) to both similarly high frequencies and to frequencies around 20Hz, i.e. a complex spectral configuration of cortical information transmission that has not been described before. In the ferret we show that the prefrontal cortex sends information at low frequencies (4-8 Hz) to early visual cortex (V1), while V1 receives the information at high frequencies (> 125 Hz).

The phase of cortical oscillations determines the perceptual fate of visual cues in naturalistic audiovisual speech

When we see our interlocutor, our brain seamlessly extracts visual cues from their face and processes them along with the sound of their voice, making speech an intrinsically multimodal signal. Visual cues are especially important in noisy environments, when the auditory signal is less reliable. Neuronal oscillations might be involved in the cortical processing of audiovisual speech by selecting which sensory channel contributes more to perception. To test this, we designed computer-generated naturalistic audiovisual speech stimuli where one mismatched phoneme-viseme pair in a key word of sentences created bistable perception. Neurophysiological recordings (high-density scalp and intracranial electroencephalography) revealed that the precise phase angle of theta-band oscillations in posterior temporal and occipital cortex of the right hemisphere was crucial to select whether the auditory or the visual speech cue drove perception. We demonstrate that the phase of cortical oscillations acts as an instrument for sensory selection in audiovisual speech processing.

Crossmodal Phase Reset and Evoked Responses Provide Complementary Mechanisms for the Influence of Visual Speech in Auditory Cortex

Natural conversation is multisensory: when we can see the speaker's face, visual speech cues improve our comprehension. The neuronal mechanisms underlying this phenomenon remain unclear. The two main alternatives are visually mediated phase modulation of neuronal oscillations (excitability fluctuations) in auditory neurons and visual input-evoked responses in auditory neurons. Investigating this question using naturalistic audiovisual speech with intracranial recordings in humans of both sexes, we find evidence for both mechanisms. Remarkably, auditory cortical neurons track the temporal dynamics of purely visual speech using the phase of their slow oscillations and phase-related modulations in broadband high-frequency activity. Consistent with known perceptual enhancement effects, the visual phase reset amplifies the cortical representation of concomitant auditory speech. In contrast to this, and in line with earlier reports, visual input reduces the amplitude of evoked responses to concomitant auditory input. We interpret the combination of improved phase tracking and reduced response amplitude as evidence for more efficient and reliable stimulus processing in the presence of congruent auditory and visual speech inputs.SIGNIFICANCE STATEMENT Watching the speaker can facilitate our understanding of what is being said. The mechanisms responsible for this influence of visual cues on the processing of speech remain incompletely understood. We studied these mechanisms by recording the electrical activity of the human brain through electrodes implanted surgically inside the brain. We found that visual inputs can operate by directly activating auditory cortical areas, and also indirectly by modulating the strength of cortical responses to auditory input. Our results help to understand the mechanisms by which the brain merges auditory and visual speech into a unitary perception.

Spike Timing in the Attention Network Predicts Behavioral Outcome Prior to Target Selection

There has been little evidence linking changes in spiking activity that occur prior to a spatially predictable target (i.e., prior to target selection) to behavioral outcomes, despite such preparatory changes being widely assumed to enhance the sensitivity of sensory processing. We simultaneously recorded from frontal and parietal nodes of the attention network while macaques performed a spatial cueing task. When anticipating a spatially predictable target, different patterns of coupling between spike timing and the oscillatory phase in local field potentials-but not changes in spike rate-were predictive of different behavioral outcomes. These behaviorally relevant differences in local and between-region synchronization occurred among specific cell types that were defined based on their sensory and motor properties, providing insight into the mechanisms underlying enhanced sensory processing prior to target selection. We propose that these changes in neural synchronization reflect differential anticipatory engagement of the network nodes and functional units that shape attention-related sampling.

Resting-State Activity in High-Order Visual Areas as a Window into Natural Human Brain Activations

A major limitation of conventional human brain research has been its basis in highly artificial laboratory experiments. Due to technical constraints, little is known about the nature of cortical activations during ecological real life. We have previously proposed the "spontaneous trait reactivation (STR)" hypothesis arguing that resting-state patterns, which emerge spontaneously in the absence of external stimulus, reflect the statistics of habitual cortical activations during real life. Therefore, these patterns can serve as a window into daily life cortical activity. A straightforward prediction of this hypothesis is that spontaneous patterns should preferentially correlate to patterns generated by naturalistic stimuli compared with artificial ones. Here we targeted high-level category-selective visual areas and tested this prediction by comparing BOLD functional connectivity patterns formed during rest to patterns formed in response to naturalistic stimuli, as well as to more artificial category-selective, dynamic stimuli. Our results revealed a significant correlation between the resting-state patterns and functional connectivity patterns generated by naturalistic stimuli. Furthermore, the correlations to naturalistic stimuli were significantly higher than those found between resting-state patterns and those generated by artificial control stimuli. These findings provide evidence of a stringent link between spontaneous patterns and the activation patterns during natural vision.

Boosted visual performance after eye blinks

We blink more often than required for maintaining the corneal tear film. Whether there are perceptual or cognitive consequences of blinks that may justify their high frequency is unclear. Previous findings showed that blinks may indicate switches between large-scale cortical networks, such as dorsal attention and default-mode networks. Thus, blinks may trigger a refresh of visual attention. Yet, this has so far not been confirmed behaviorally. Here, we tested the effect of blinks on visual performance in a series of rapid serial visual presentation tasks. In Experiment 1, participants had to identify a target digit embedded in a random stream of letter distractors, presented foveally for 60 ms each. Participants blinked once during the presentation stream. In a separate condition, blinks were simulated by shutter glasses. Detection performance was enhanced (up to 13% point increase in accuracy) for targets appearing up to 300 ms after eye blinks. Performance boosts were stronger for voluntary blinks than artificial blinks. This performance boost was also replicated with more naturalistic stimuli (Experiment 2). We conclude that eye blinks lead to attentional benefits for object recognition in the period after reopening of the eyelids and may be used strategically for temporarily boosting visual performance.

Responses to Visual Speech in Human Posterior Superior Temporal Gyrus Examined with iEEG Deconvolution

Experimentalists studying multisensory integration compare neural responses to multisensory stimuli with responses to the component modalities presented in isolation. This procedure is problematic for multisensory speech perception since audiovisual speech and auditory-only speech are easily intelligible but visual-only speech is not. To overcome this confound, we developed intracranial encephalography (iEEG) deconvolution. Individual stimuli always contained both auditory and visual speech, but jittering the onset asynchrony between modalities allowed for the time course of the unisensory responses and the interaction between them to be independently estimated. We applied this procedure to electrodes implanted in human epilepsy patients (both male and female) over the posterior superior temporal gyrus (pSTG), a brain area known to be important for speech perception. iEEG deconvolution revealed sustained positive responses to visual-only speech and larger, phasic responses to auditory-only speech. Confirming results from scalp EEG, responses to audiovisual speech were weaker than responses to auditory-only speech, demonstrating a subadditive multisensory neural computation. Leveraging the spatial resolution of iEEG, we extended these results to show that subadditivity is most pronounced in more posterior aspects of the pSTG. Across electrodes, subadditivity correlated with visual responsiveness, supporting a model in which visual speech enhances the efficiency of auditory speech processing in pSTG. The ability to separate neural processes may make iEEG deconvolution useful for studying a variety of complex cognitive and perceptual tasks.SIGNIFICANCE STATEMENT Understanding speech is one of the most important human abilities. Speech perception uses information from both the auditory and visual modalities. It has been difficult to study neural responses to visual speech because visual-only speech is difficult or impossible to comprehend, unlike auditory-only and audiovisual speech. We used intracranial encephalography deconvolution to overcome this obstacle. We found that visual speech evokes a positive response in the human posterior superior temporal gyrus, enhancing the efficiency of auditory speech processing.

Why Does Space Feel the Way it Does? Towards a Principled Account of Spatial Experience

There must be a reason why an experience feels the way it does. A good place to begin addressing this question is spatial experience, because it may be more penetrable by introspection than other qualities of consciousness such as color or pain. Moreover, much of experience is spatial, from that of our body to the visual world, which appears as if painted on an extended canvas in front of our eyes. Because it is ‘right there’, we usually take space for granted and overlook its qualitative properties. However, we should realize that a great number of phenomenal distinctions and relations are required for the canvas of space to feel ‘extended’. Here we argue that, to be experienced as extended, the canvas of space must be composed of countless spots, here and there, small and large, and these spots must be related to each other in a characteristic manner through connection, fusion, and inclusion. Other aspects of the structure of spatial experience follow from extendedness: every spot can be experienced as enclosing a particular region, with its particular location, size, boundary, and distance from other spots. We then propose an account of the phenomenal properties of spatial experiences based on integrated information theory (IIT). The theory provides a principled approach for characterizing both the quantity and quality of experience by unfolding the cause-effect structure of a physical substrate. Specifically, we show that a simple simulated substrate of units connected in a grid-like manner yields a cause-effect structure whose properties can account for the main properties of spatial experience. These results uphold the hypothesis that our experience of space is supported by brain areas whose units are linked by a grid-like connectivity. They also predict that changes in connectivity, even in the absence of changes in activity, should lead to a warping of experienced space. To the extent that this approach provides an initial account of phenomenal space, it may also serve as a starting point for investigating other aspects of the quality of experience and their physical correspondents.

Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks

The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception.

Deep convolutional neural networks (DCNNs) are able to identify faces on par with humans. Here, the authors record neuronal activity from higher visual areas in humans and show that face-selective responses in the brain show similarity to those in the intermediate layers of the DCNN.

Hippocampal sharp-wave ripples linked to visual episodic recollection in humans

Hippocampal sharp-wave ripples (SWRs) constitute one of the most synchronized activation events in the brain and play a critical role in offline memory consolidation. Yet their cognitive content and function during awake, conscious behavior remains unclear. We directly examined this question using intracranial recordings in human patients engaged in episodic free recall of previously viewed photographs. Our results reveal a content-selective increase in hippocampal ripple rate emerging 1 to 2 seconds prior to recall events. During recollection, high-order visual areas showed pronounced SWR-coupled reemergence of activation patterns associated with recalled content. Finally, the SWR rate during encoding predicted subsequent free-recall performance. These results point to a role for hippocampal SWRs in triggering spontaneous recollections and orchestrating the reinstatement of cortical representations during free episodic memory retrieval.

Twitches, Blinks, and Fidgets: Important Generators of Ongoing Neural Activity

Animals and humans continuously engage in small, spontaneous motor actions, such as blinking, whisking, and postural adjustments ("fidgeting"). These movements are accompanied by changes in neural activity in sensory and motor regions of the brain. The frequency of these motions varies in time, is affected by sensory stimuli, arousal levels, and pathology. These fidgeting behaviors can be entrained by sensory stimuli. Fidgeting behaviors will cause distributed, bilateral functional activation in the 0.01 to 0.1 Hz frequency range that will show up in functional magnetic resonance imaging and wide-field calcium neuroimaging studies, and will contribute to the observed functional connectivity among brain regions. However, despite the large potential of these behaviors to drive brain-wide activity, these fidget-like behaviors are rarely monitored. We argue that studies of spontaneous and evoked brain dynamics in awake animals and humans should closely monitor these fidgeting behaviors. Differences in these fidgeting behaviors due to arousal or pathology will "contaminate" ongoing neural activity, and lead to apparent differences in functional connectivity. Monitoring and accounting for the brain-wide activations by these behaviors is essential during experiments to differentiate fidget-driven activity from internally driven neural dynamics.

Where Does Time Go When You Blink?

Retinal input is frequently lost because of eye blinks, yet humans rarely notice these gaps in visual input. Although previous studies focused on the perceptual and neural correlates of diminished awareness to blinks, the impact of these correlates on the perceived time of concurrent events is unknown. Here, we investigated whether the subjective sense of time is altered by spontaneous blinks. We found that participants ( N = 22) significantly underestimated the duration of a visual stimulus when a spontaneous blink occurred during stimulus presentation and that this underestimation was correlated with the blink duration of individual participants. Importantly, the effect was not present when durations of an auditory stimulus were judged ( N = 23). The results point to a link between spontaneous blinks, previously demonstrated to induce activity suppression in the visual cortex, and a compression of subjective time. They suggest that ongoing encoding within modality-specific sensory cortices, independent of conscious awareness, informs the subjective sense of time.

Neuronal baseline shifts underlying boundary setting during free recall

Asked to freely recall items from a predefined set (e.g., animals), we rarely recall a wrong exemplar (e.g., a vegetable). This capability is so powerful and effortless that it is essentially taken for granted, yet, surprisingly, the underlying neuronal mechanisms are unknown. Here we investigate this boundary setting mechanism using intracranial recordings (ECoG), in 12 patients undergoing epilepsy monitoring engaged in episodic free recall. After viewing vivid photographs from two categories (famous faces and places), patients were asked to freely recall these items, targeting each category in separate blocks. Our results reveal a rapid and sustained rise in neuronal activity (“baseline shift”) in high-order visual areas that persists throughout the free recall period and reflects the targeted category. We further show a more transient reactivation linked to individual recall events. The results point to baseline shift as a flexible top−down mechanism that biases spontaneous recall to remain within the required categorical boundaries.

Though people are easily able to recall items in a category without mentioning a wrong exemplar, the mechanism underlying this ability is unknown. Here, authors use intracranial recordings to show that this ability is likely due to a selective increase in baseline neuronal activity in category-specific regions.

Increasing suppression of saccade-related transients along the human visual hierarchy

A key hallmark of visual perceptual awareness is robustness to instabilities arising from unnoticeable eye and eyelid movements. In previous human intracranial (iEEG) work (Golan et al., 2016) we found that excitatory broadband high-frequency activity transients, driven by eye blinks, are suppressed in higher-level but not early visual cortex. Here, we utilized the broad anatomical coverage of iEEG recordings in 12 eye-tracked neurosurgical patients to test whether a similar stabilizing mechanism operates following small saccades. We compared saccades (1.3°−3.7°) initiated during inspection of large individual visual objects with similarly-sized external stimulus displacements. Early visual cortex sites responded with positive transients to both conditions. In contrast, in both dorsal and ventral higher-level sites the response to saccades (but not to external displacements) was suppressed. These findings indicate that early visual cortex is highly unstable compared to higher-level visual regions which apparently constitute the main target of stabilizing extra-retinal oculomotor influences.