Intracranial Recordings Reveal Unique Shape and Timing of Responses in Human Visual Cortex during Illusory Visual Events

Divergent mechanisms of perceptual reversals in spinning and wobbling structure-from-motion stimuli

This study explores the visual phenomenon of random dot structure-from-motion (SFM), where the brain perceives 3D shapes from the coordinated 2D motion of dots. Observing SFM may lead to ambiguous depth relations that reverse back and forth during prolonged viewing. I demonstrate that different processes are involved in triggering perceived reversals for identical SFM shapes involved in spinning and wobbling motion. Durations of stable percepts were measured while human participants viewed the two SFM stimuli, and also a static Necker figure, and a wobbling Necker figure for two sets of 2.5 minutes each. The results showed that wobbling SFM resulted in much longer stable durations compared to the other stimuli. The durations for the wobbling SFM stimuli was not correlated with the spinning SFM, or the two Necker stimuli. Yet, such correlations were obtained between the other stimuli. It is known that reversals obtained while viewing spinning SFM stimuli involves bottom-up driven adaptation and recovery cycles between neural populations. This result suggests that wobbling SFM efficiently deactivates this process and targets other contributions to the reversals, such as top-down processes. In addition, biases observed in the first set disappeared in the second set implying influences of learning between the sets. Imagery vividness, which measures intrinsic top-down processes, was also scored but no correlation between scores in visual imagery and reversal rates were obtained. This research provides insight into the complex interplay between bottom-up driven adaptation-recovery cycles, and top-down processes in ambiguous perception.

An Accumulating Neural Signal Underlying Binocular Rivalry Dynamics

During binocular rivalry, conflicting images are presented one to each eye and perception alternates stochastically between them. Despite stable percepts between alternations, modeling suggests that neural signals representing the two images change gradually, and that the duration of stable percepts are determined by the time required for these signals to reach a threshold that triggers an alternation. However, direct physiological evidence for such signals has been lacking. Here, we identify a neural signal in the human visual cortex that shows these predicted properties. We measured steady-state visual evoked potentials (SSVEPs) in 84 human participants (62 females, 22 males) who were presented with orthogonal gratings, one to each eye, flickering at different frequencies. Participants indicated their percept while EEG data were collected. The time courses of the SSVEP amplitudes at the two frequencies were then compared across different percept durations, within participants. For all durations, the amplitude of signals corresponding to the suppressed stimulus increased and the amplitude corresponding to the dominant stimulus decreased throughout the percept. Critically, longer percepts were characterized by more gradual increases in the suppressed signal and more gradual decreases of the dominant signal. Changes in signals were similar and rapid at the end of all percepts, presumably reflecting perceptual transitions. These features of the SSVEP time courses are well predicted by a model in which perceptual transitions are produced by the accumulation of noisy signals. Identification of this signal underlying binocular rivalry should allow strong tests of neural models of rivalry, bistable perception, and neural suppression.SIGNIFICANCE STATEMENT During binocular rivalry, two conflicting images are presented to the two eyes and perception alternates between them, with switches occurring at seemingly random times. Rivalry is an important and longstanding model system in neuroscience, used for understanding neural suppression, intrinsic neural dynamics, and even the neural correlates of consciousness. All models of rivalry propose that it depends on gradually changing neural activity that on reaching some threshold triggers the perceptual switches. This manuscript reports the first physiological measurement of neural signals with that set of properties in human participants. The signals, measured with EEG in human observers, closely match the predictions of recent models of rivalry, and should pave the way for much future work.

Does V1 response suppression initiate binocular rivalry?

During binocular rivalry (BR) only one eye's view is perceived. Neural underpinnings of BR are debated. Recent studies suggest that primary visual cortex (V1) initiates BR. One trigger might be response suppression across most V1 neurons at the onset of BR. Here, we utilize a variant of BR called binocular rivalry flash suppression (BRFS) to test this hypothesis. BRFS is identical to BR, except stimuli are shown with a ∼1s delay. If V1 response suppression was required to initiate BR, it should occur during BRFS as well. To test this, we compared V1 spiking in two macaques observing BRFS. We found that BRFS resulted in response facilitation rather than response suppression across V1 neurons. However, BRFS still reduces responses in a subset of V1 neurons due to the adaptive effects of asynchronous stimulus presentation. We argue that this selective response suppression could serve as an alternate initiator of BR.

Spontaneous Necker-cube reversals may not be that spontaneous

Introduction

During observation of the ambiguous Necker cube, our perception suddenly reverses between two about equally possible 3D interpretations. During passive observation, perceptual reversals seem to be sudden and spontaneous. A number of theoretical approaches postulate destabilization of neural representations as a pre-condition for reversals of ambiguous figures. In the current study, we focused on possible Electroencephalogram (EEG) correlates of perceptual destabilization, that may allow prediction of an upcoming perceptual reversal.

Methods

We presented ambiguous Necker cube stimuli in an onset-paradigm and investigated the neural processes underlying endogenous reversals as compared to perceptual stability across two consecutive stimulus presentations. In a separate experimental condition, disambiguated cube variants were alternated randomly, to exogenously induce perceptual reversals. We compared the EEG immediately before and during endogenous Necker cube reversals with corresponding time windows during exogenously induced perceptual reversals of disambiguated cube variants.

Results

For the ambiguous Necker cube stimuli, we found the earliest differences in the EEG between reversal trials and stability trials already 1 s before a reversal occurred, at bilateral parietal electrodes. The traces remained similar until approximately 1100 ms before a perceived reversal, became maximally different at around 890 ms (p = 7.59 × 10^-6, Cohen's d = 1.35) and remained different until shortly before offset of the stimulus preceding the reversal. No such patterns were found in the case of disambiguated cube variants.

Discussion

The identified EEG effects may reflect destabilized states of neural representations, related to destabilized perceptual states preceding a perceptual reversal. They further indicate that spontaneous Necker cube reversals are most probably not as spontaneous as generally thought. Rather, the destabilization may occur over a longer time scale, at least 1 s before a reversal event, despite the reversal event as such being perceived as spontaneous by the viewer.

Feedback information sharing in the human brain reflects bistable perception in the absence of report

In the search for the neural basis of conscious experience, perception and the cognitive processes associated with reporting perception are typically confounded as neural activity is recorded while participants explicitly report what they experience. Here, we present a novel way to disentangle perception from report using eye movement analysis techniques based on convolutional neural networks and neurodynamical analyses based on information theory. We use a bistable visual stimulus that instantiates two well-known properties of conscious perception: integration and differentiation. At any given moment, observers either perceive the stimulus as one integrated unitary object or as two differentiated objects that are clearly distinct from each other. Using electroencephalography, we show that measures of integration and differentiation based on information theory closely follow participants' perceptual experience of those contents when switches were reported. We observed increased information integration between anterior to posterior electrodes (front to back) prior to a switch to the integrated percept, and higher information differentiation of anterior signals leading up to reporting the differentiated percept. Crucially, information integration was closely linked to perception and even observed in a no-report condition when perceptual transitions were inferred from eye movements alone. In contrast, the link between neural differentiation and perception was observed solely in the active report condition. Our results, therefore, suggest that perception and the processes associated with report require distinct amounts of anterior-posterior network communication and anterior information differentiation. While front-to-back directed information is associated with changes in the content of perception when viewing bistable visual stimuli, regardless of report, frontal information differentiation was absent in the no-report condition and therefore is not directly linked to perception per se.

Neural representations of competing stimuli along the dorsal and ventral visual pathways during binocular rivalry

Binocular rivalry arises when two discrepant stimuli are simultaneously presented to different eyes, during which observers consciously experience vivid perceptual alternations without physical changes in visual inputs. Neural dynamics tracking such perceptual alternations have been identified at both early and late visual areas, leading to the fundamental debate concerning the primary neural substrate underlying binocular rivalry. One promising hypothesis that might reconcile these seemingly paradoxical findings is a gradual shift from interocular competition between monocular neurons to pattern competition among binocular neurons. Here, we examined this hypothesis by investigating how neural representations of rivalrous stimuli evolved along the visual pathway. We found that representations of the dominant and the suppressed stimuli initially co-existed in V1, which were enhanced and attenuated respectively in extrastriate visual areas. Notably, neural activity in V4 was dictated by the representation of the dominant stimulus, while the representation of the suppressed stimulus was only partially inhibited in dorsal areas V3A and MT+. Our findings revealed a progressive transition from the co-existing representations of the rivalrous inputs to the dictatorial representation of the dominant stimulus in the ventral pathway, and advocated different cortical evolutionary patterns of visual representations between the dorsal and the ventral pathways.

Priming and reversals of the perceived ambiguous orientation of a structure-from-motion shape and relation to personality traits

We demonstrate contributions of top-down and bottom-up influences in perception as explored by priming and counts of perceived reversals and mixed percepts, as probed by an ambiguously slanted structure-from-motion (SFM) test-cylinder. We included three different disambiguated primes: a SFM cylinder, a still image of a cylinder, and an imagined cylinder. In Experiment 1 where the prime and test sequentially occupied the same location, we also administered questionnaires with the Big-5 trait openness and vividness of visual imagery to probe possible relations to top-down driven priming. Since influences of gaze or position in the prime conditions in Experiment 1 could not be ruled out completely, Experiment 2 was conducted where the test cylinder appeared at a randomly chosen position after the prime. In Experiment 2 we also measured the number of perceptual reversals and mixed percepts during prolonged viewing of our ambiguous SFM-cylinder, and administered questionnaires to measure all Big-5 traits, autism, spatial and object imagery, and rational or experiential cognitive styles, associated with bottom-up and top-down processes. The results revealed contributions of position-invariant and cue-invariant priming. In addition, residual contributions of low-level priming was found when the prime and test were both defined by SFM, and were presented at the same location, and the correlation between the SFM priming and the other two priming conditions were weaker than between the pictorial and imagery priming. As previously found with ambiguous binocular rivalry stimuli, we found positive correlations between mixed percepts and the Big-5 dimension openness to experience, and between reversals, mixed percepts and neuroticism. Surprisingly, no correlations between the scores from the vividness of imagery questionnaires and influence from any of the primes were obtained. An intriguing finding was the significant differences between the positive correlation from the experiential cognitive style scores, and the negative correlation between rational style and the cue invariant priming. Among other results, negative correlations between agreeableness and all priming conditions were obtained. These results not only support the notion of multiple processes involved in the perception of ambiguous SFM, but also link these processes in perception to specific personality traits.

Multistability, perceptual value, and internal foraging

Substantial experimental, theoretical, and computational insights into sensory processing have been derived from the phenomena of perceptual multistability-when two or more percepts alternate or switch in response to a single sensory input. Here, we review a range of findings suggesting that alternations can be seen as internal choices by the brain responding to values. We discuss how elements of external, experimenter-controlled values and internal, uncertainty- and aesthetics-dependent values influence multistability. We then consider the implications for the involvement in switching of regions, such as the anterior cingulate cortex, which are more conventionally tied to value-dependent operations such as cognitive control and foraging.

The Perceptual Magic of Binocular Rivalry

Binocular rivalry (BR) refers to the spontaneous, unpredictable fluctuations in visual awareness provoked by dissimilar stimulation of the two eyes. Reports of the phenomenon date back several centuries, but interest in BR has exploded in recent years as researchers in diverse disciplines—psychology, neuroscience, medicine, philosophy—have found reasons to study it. New ideas about BR have emerged, sparking controversies about its neural bases, which may be resolved thanks to new methodological developments. This essay provides a synopsis of some key empirically determined aspects of BR as well as an overview of theoretical developments in this field. Work published during the past decade or so is emphasized (and explicitly referenced); earlier key findings are mentioned and referenced in the annotated bibliography included in the Supplemental Material.

Change not State: Perceptual coupling in multistable displays reflects transient bias induced by perceptual change

We investigated how changes in dynamic spatial context influence visual perception. Specifically, we reexamined the perceptual coupling phenomenon when two multistable displays viewed simultaneously tend to be in the same dominant state and switch in accord. Current models assume this interaction reflecting mutual bias produced by a dominant perceptual state. In contrast, we demonstrate that influence of spatial context is strongest when perception changes. First, we replicated earlier work using bistable kinetic-depth effect displays, then extended it by employing asynchronous presentation to show that perceptual coupling cannot be accounted for by the static context provided by perceptually dominant states. Next, we demonstrated that perceptual coupling reflects transient bias induced by perceptual change, both in ambiguous and disambiguated displays. We used a hierarchical Bayesian model to characterize its timing, demonstrating that the transient bias is induced 50-70 ms after the exogenous trigger event and decays within ~200-300 ms. Both endogenous and exogenous switches led to quantitatively and qualitatively similar perceptual consequences, activating similar perceptual reevaluation mechanisms within a spatial surround. We explain how they can be understood within a transient selective visual attention framework or using local lateral connections within sensory representations. We suggest that observed perceptual effects reflect general mechanisms of perceptual inference for dynamic visual scene perception.

Bi-Stable Perception: Self-Coordinating Brain Regions to Make-Up the Mind

Bi-stable perception is a strong instance of cognitive self-organization, providing a research model for how ‘the brain makes up its mind.’ The complexity of perceptual bistability prevents a simple attribution of functions to areas, because many cognitive processes, recruiting multiple brain regions, are simultaneously involved. The functional magnetic resonance imaging (fMRI) evidence suggests the activation of a large network of distant brain areas. Concurrently, electroencephalographic and magnetoencephalographic (MEEG) literature shows sub second oscillatory activity and phase synchrony on several frequency bands. Strongly represented are beta and gamma bands, often associated with neural/cognitive integration processes. The spatial extension and short duration of brain activities suggests the need for a fast, large-scale neural coordination mechanism. To address the range of temporo-spatial scales involved, we systematize the current knowledge from mathematical models, cognitive sciences and neuroscience at large, from single-cell- to system-level research, including evidence from human and non-human primates. Surprisingly, despite evidence spanning through different organization levels, models, and experimental approaches, the scarcity of integrative studies is evident. In a final section of the review we dwell on the reasons behind such scarcity and on the need of integration in order to achieve a real understanding of the complexities underlying bi-stable perception processes.

Long-term priors influence visual perception through recruitment of long-range feedback

Perception results from the interplay of sensory input and prior knowledge. Despite behavioral evidence that long-term priors powerfully shape perception, the neural mechanisms underlying these interactions remain poorly understood. We obtained direct cortical recordings in neurosurgical patients as they viewed ambiguous images that elicit constant perceptual switching. We observe top-down influences from the temporal to occipital cortex, during the preferred percept that is congruent with the long-term prior. By contrast, stronger feedforward drive is observed during the non-preferred percept, consistent with a prediction error signal. A computational model based on hierarchical predictive coding and attractor networks reproduces all key experimental findings. These results suggest a pattern of large-scale information flow change underlying long-term priors' influence on perception and provide constraints on theories about long-term priors' influence on perception.

Bilingualism contributes to reserve and working memory efficiency: Evidence from structural and functional neuroimaging

This study compared brain and behavioral outcomes for monolingual and bilingual older adults who reported no cognitive or memory problems on three types of memory that typically decline in older age, namely, working memory (measured by n-back), item, and associative recognition. The results showed that bilinguals were faster on the two-back working memory task than monolinguals but used a set of frontostriatal regions less than monolinguals. There was no group difference on an item/associative recognition task. In brain structure, gray matter volume and white matter integrity (fractional anisotropy) were generally lower in bilinguals than in monolinguals, but bilinguals had better white matter integrity than monolinguals in the bilateral superior corona radiata and better gray matter density in the left inferior temporal gyrus. These regions may help preserve bilinguals' executive functions despite generally more significant atrophy throughout the brain than monolinguals in that these structures contribute to efficient communication between executive frontal regions and subcortical motor regions, and perceptual pathways. Reliable negative correlations between brain structure and age were only observed in bilinguals, and to the extent that bilinguals (but not monolinguals) had better brain structure, their performance was enhanced. Collectively, the findings provide evidence for reserve in bilingual older adults.

FMRI and intra-cranial electrocorticography recordings in the same human subjects reveals negative BOLD signal coupled with silenced neuronal activity

Positive blood oxygenation level-dependent (BOLD) responses (PBR), as measured by functional Magnetic Resonance Imaging (fMRI), are the most utilized measurements to non-invasively map activity in the brain. Recent studies have consistently shown that BOLD responses are not exclusively positive. Negative BOLD responses (NBR) have been reported in response to specific sensory stimulations and tasks. However, the exact relationship between NBR and the underlying metabolic and neuronal demand is still under debate. In this study, we investigated the neurophysiological basis of negative BOLD using fMRI and intra-cranial electrophysiology (electrocorticography, ECoG) measurements from the same human participants. We show that, for those electrodes that responded to visual stimulation, PBR are correlated with high-frequency band (HFB) responses. Crucially, NBR were associated with an absence of HFB power responses and an unpredicted decrease in the alpha power responses.

Visual Perception: Human Brain Cells Cause a Change of View

Some images spontaneously change in appearance. A new study has found that these changes are reflected in high-level visual cortical areas before they become apparent in early sensory cortex. This suggests that visual information not only flows towards interpretative areas of our brain, but also in the reverse direction.