SSVEP signatures of binocular rivalry during simultaneous EEG and fMRI

Tracking rivalry with neural rhythms: multivariate SSVEPs reveal perception during binocular rivalry

The contents of awareness can substantially change without any modification to the external world. Such effects are exemplified in binocular rivalry, where a different stimulus is presented to each eye causing instability in perception. This phenomenon has made binocular rivalry a quintessential method for studying consciousness and the necessary neural correlates for awareness. However, to conduct research on binocular rivalry usually requires self-reports of changes in percept, which can produce confounds and exclude states and contexts where self-reports are undesirable or unreliable. Here, we use a novel multivariate spatial filter dubbed 'Rhythmic Entrainment Source Separation' to extract steady state visual evoked potentials from electroencephalography data. We show that this method can be used to quantify the perceptual switch-rate of participants during binocular rivalry and therefore may be valuable in experimental contexts where self-reports are methodologically problematic or impossible, particularly as an adjunct. Our analyses also reveal that 'no-report' conditions may affect the deployment of attention and thereby neural correlates, another important consideration for consciousness research.

An Accumulating Neural Signal Underlying Binocular Rivalry Dynamics

During binocular rivalry, conflicting images are presented one to each eye and perception alternates stochastically between them. Despite stable percepts between alternations, modeling suggests that neural signals representing the two images change gradually, and that the duration of stable percepts are determined by the time required for these signals to reach a threshold that triggers an alternation. However, direct physiological evidence for such signals has been lacking. Here, we identify a neural signal in the human visual cortex that shows these predicted properties. We measured steady-state visual evoked potentials (SSVEPs) in 84 human participants (62 females, 22 males) who were presented with orthogonal gratings, one to each eye, flickering at different frequencies. Participants indicated their percept while EEG data were collected. The time courses of the SSVEP amplitudes at the two frequencies were then compared across different percept durations, within participants. For all durations, the amplitude of signals corresponding to the suppressed stimulus increased and the amplitude corresponding to the dominant stimulus decreased throughout the percept. Critically, longer percepts were characterized by more gradual increases in the suppressed signal and more gradual decreases of the dominant signal. Changes in signals were similar and rapid at the end of all percepts, presumably reflecting perceptual transitions. These features of the SSVEP time courses are well predicted by a model in which perceptual transitions are produced by the accumulation of noisy signals. Identification of this signal underlying binocular rivalry should allow strong tests of neural models of rivalry, bistable perception, and neural suppression.SIGNIFICANCE STATEMENT During binocular rivalry, two conflicting images are presented to the two eyes and perception alternates between them, with switches occurring at seemingly random times. Rivalry is an important and longstanding model system in neuroscience, used for understanding neural suppression, intrinsic neural dynamics, and even the neural correlates of consciousness. All models of rivalry propose that it depends on gradually changing neural activity that on reaching some threshold triggers the perceptual switches. This manuscript reports the first physiological measurement of neural signals with that set of properties in human participants. The signals, measured with EEG in human observers, closely match the predictions of recent models of rivalry, and should pave the way for much future work.

Effects of interocular grouping demands on binocular rivalry

Binocular rivalry (BR) is a visual phenomenon in which perception alternates between two non-fusible images presented to each eye. Transition periods between dominant and suppressed images are marked by mixed percepts, where participants report fragments of each image being dynamically perceived. Interestingly, BR remains robust even when typical images are subdivided and presented in complementary patches to each eye, a phenomenon termed interocular grouping (IOG). The objective of the present study was to determine if increasing grouping demand in the context of BR changes the perceptual experience of rivalry. In 48 subjects with normal vision, mean dominant and mixed percept durations were recorded for classic BR and IOG conditions with increasing grouping demands from two, four, and six patches. We found that, as grouping demands increased, the duration of mixed periods increased. Indeed, durations of dominant and mixed percepts, as well as percentage of time spent in dominant or mixed state, differed significantly across conditions. However, durations of global dominant percepts remained relatively stable and saturated at about 1.5 seconds, despite the exponential increase in possible mixed combinations. Evidence shows that this saturation followed a nonlinear trend. The data also indicate that grouping across the vertical meridian is slightly more stable than for the horizontal meridian. Finally, individual differences in speed of alternation identified during BR were maintained in all interocular grouping conditions. These results provide new information about binocular visual spatial integration and will be useful for future studies of the underlying neural substrates and models of binocular vision.

Intrinsic brain dynamics in the Default Mode Network predict involuntary fluctuations of visual awareness

Brain activity is intrinsically organised into spatiotemporal patterns, but it is still not clear whether these intrinsic patterns are functional or epiphenomenal. Using a simultaneous fMRI-EEG implementation of a well-known bistable visual task, we showed that the latent transient states in the intrinsic EEG oscillations can predict upcoming involuntarily perceptual transitions. The critical state predicting a dominant perceptual transition was characterised by the phase coupling between the precuneus (PCU), a key node of the Default Mode Network (DMN), and the primary visual cortex (V1). The interaction between the lifetime of this state and the PCU- > V1 Granger-causal effect is correlated with the perceptual fluctuation rate. Our study suggests that the brain's endogenous dynamics are phenomenologically relevant, as they can elicit a diversion between potential visual processing pathways, while external stimuli remain the same. In this sense, the intrinsic DMN dynamics pre-empt the content of consciousness.

Multi-objective optimization approach for channel selection and cross-subject generalization in RSVP-based BCIs

Objective.Achieving high precision rapid serial visual presentation (RSVP) task often requires many electrode channels to obtain more information. However, the more channels may contain more redundant information and also lead to its limited practical applications. Therefore, it is necessary to reduce the number of channels to enhance the classification performance and users experience. Furthermore, cross-subject generalization has always been one of major challenges in electroencephalography channel reduction, especially in the RSVP paradigm. Most search-based channel selection method presented in the literature are single-objective methods, the classification accuracy (ACC) is usually chosen as the only criterion.Approach.In this article, the idea of multi-objective optimization was introduced into the RSVP channel selection to minimize two objectives: classification error and the number of channels. By combining a multi-objective evolutionary algorithm for solving large-scale sparse problems and hierarchical discriminant component analysis (HDCA), a novel channel selection method for RSVP was proposed. After that, the cross-subject generalization validation through the proposed channel selection method.Main results.The proposed method achieved an average ACC of 95.41% in a public dataset, which is 3.49% higher than HDCA. The ACC was increased by 2.73% and 2.52%, respectively. Besides, the cross-subject generalization models in channel selection, namely special-16 and special-32, on untrained subjects show that the classification performance is better than the Hoffmann empirical channels.Significance.The proposed channel selection method could reduce the calibration time in the experimental preparation phase and obtain a better accuracy, which is promising application in the RSVP scenario that requires low-density electrodes.

The SSVEP tracks attention, not consciousness, during perceptual filling-in

Research on the neural basis of conscious perception has almost exclusively shown that becoming aware of a stimulus leads to increased neural responses. By designing a novel form of perceptual filling-in (PFI) overlaid with a dynamic texture display, we frequency-tagged multiple disappearing targets as well as their surroundings. We show that in a PFI paradigm, the disappearance of a stimulus and subjective invisibility is associated with increases in neural activity, as measured with steady-state visually evoked potentials (SSVEPs), in electroencephalography (EEG). We also find that this increase correlates with alpha-band activity, a well-established neural measure of attention. These findings cast doubt on the direct relationship previously reported between the strength of neural activity and conscious perception, at least when measured with current tools, such as the SSVEP. Instead, we conclude that SSVEP strength more closely measures changes in attention.

A multiple-response frequency-tagging paradigm measures graded changes in consciousness during perceptual filling-in

Perceptual filling-in (PFI) occurs when a physically present visual target disappears from conscious perception, with its location filled-in by the surrounding visual background. These perceptual changes are complete, near instantaneous, and can occur for multiple separate locations simultaneously. Here, we show that contrasting neural activity during the presence or absence of multi-target PFI can complement other findings from multistable phenomena to reveal the neural correlates of consciousness (NCC). We presented four peripheral targets over a background dynamically updating at 20 Hz. While participants reported on target disappearances/reappearances via button press/release, we tracked neural activity entrained by the background during PFI using steady-state visually evoked potentials (SSVEPs) recorded in the electroencephalogram. We found background SSVEPs closely correlated with subjective report, and increased with an increasing amount of PFI. Unexpectedly, we found that as the number of filled-in targets increased, the duration of target disappearances also increased, suggesting that facilitatory interactions exist between targets in separate visual quadrants. We also found distinct spatiotemporal correlates for the background SSVEP harmonics. Prior to genuine PFI, the response at the second harmonic (40 Hz) increased before the first (20 Hz), which we tentatively link to an attentional effect, while no such difference between harmonics was observed for physically removed stimuli. These results demonstrate that PFI can be used to study multi-object perceptual suppression when frequency-tagging the background of a visual display, and because there are distinct neural correlates for endogenously and exogenously induced changes in consciousness, that it is ideally suited to study the NCC.

Intrinsic Rivalry. Can White Bears Help Us With the Other Side of Consciousness?

Studies of consciousness have traditionally been based mainly upon the perceptual domains of consciousness. However, there is another side of consciousness, represented by various types of intrinsic conscious experiences. Even though intrinsic experiences can represent up to 50% of our conscious experiences, they are still largely neglected in conscious studies. We assume there are two reasons for this. First, the field of intrinsic conscious experiences is methodologically far more problematic than any other. Second, specific paradigms for capturing the correlates of intrinsic conscious experiences are almost nonexistent. Nevertheless, we expect the intrinsic side of consciousness to soon take its place in conscious studies, but first new experimental paradigms will have to be devised, which would be of a similar design to the paradigms used in studies of perceptual consciousness. In this hypothesis and theory article, we propose such a hypothetical paradigm, presenting the exploratory data of our proof-of-concept study, discussing its use, and addressing its shortcomings and their possible remediation.

Perceptual reversals in binocular rivalry: Improved detection from OKN

When binocular rivalry is induced by opponent motion displays, perceptual reversals are often associated with changed oculomotor behavior (Frässle, Sommer, Jansen, Naber, & Einhäuser, 2014; Fujiwara et al., 2017). Specifically, the direction of smooth pursuit phases in optokinetic nystagmus typically corresponds to the direction of motion that dominates perceptual appearance at any given time. Here we report an improved analysis that continuously estimates perceived motion in terms of "cumulative smooth pursuit." In essence, smooth pursuit segments are identified, interpolated where necessary, and joined probabilistically into a continuous record of cumulative smooth pursuit (i.e., probability of eye position disregarding blinks, saccades, signal losses, and artefacts). The analysis is fully automated and robust in healthy, developmental, and patient populations. To validate reliability, we compare volitional reports of perceptual reversals in rivalry displays, and of physical reversals in nonrivalrous control displays. Cumulative smooth pursuit detects physical reversals and estimates eye velocity more accurately than existing methods do (Frässle et al., 2014). It also appears to distinguish dominant and transitional perceptual states, detecting changes with a precision of ±100 ms. We conclude that cumulative smooth pursuit significantly improves the monitoring of binocular rivalry by means of recording optokinetic nystagmus.

Atypical intrinsic neural timescale in autism

How long neural information is stored in a local brain area reflects functions of that region and is often estimated by the magnitude of the autocorrelation of intrinsic neural signals in the area. Here, we investigated such intrinsic neural timescales in high-functioning adults with autism and examined whether local brain dynamics reflected their atypical behaviours. By analysing resting-state fMRI data, we identified shorter neural timescales in the sensory/visual cortices and a longer timescale in the right caudate in autism. The shorter intrinsic timescales in the sensory/visual areas were correlated with the severity of autism, whereas the longer timescale in the caudate was associated with cognitive rigidity. These observations were confirmed from neurodevelopmental perspectives and replicated in two independent cross-sectional datasets. Moreover, the intrinsic timescale was correlated with local grey matter volume. This study shows that functional and structural atypicality in local brain areas is linked to higher-order cognitive symptoms in autism.

Autism is a brain disorder that affects how people interact with others. It occupies a spectrum, with severe autism at one end and high-functioning autism at the other. People with severe autism usually have intellectual impairments and little spoken language. Those with high-functioning autism have average or above average IQ, but struggle with more subtle aspects of communication, such as body language. As well as social difficulties, many individuals with autism show repetitive behaviors and have narrow interests.

The brains of people with autism process information differently to those of people without autism. The brain as a whole shows less coordinated activity in autism, for example. But whether individual brain regions themselves also work differently in autism is unclear. Watanabe et al. set out to answer this question by using a brain scanner to compare the resting brain activity of high-functioning people with autism to that of people without autism.

In both groups, networks of brain regions increased and decreased their activity in predictable patterns. But in individuals with autism, sensory areas of the brain showed more random activity than in individuals without autism. The most random activity occurred in those with the most severe autism. This suggests that the brains of people with autism cannot hold onto and process sensory input for as long as those of neurotypical people. By contrast, a brain region called the caudate showed the opposite pattern, being more predictable in individuals with autism. The most predictable caudate activity occurred in those individuals with the most inflexible, repetitive behaviors. These differences in this neural randomness appear to result from changes in the structure of the individual brain regions.

The findings of Watanabe et al. suggest that changes in the structure and activity of small brain regions give rise to complex symptoms in autism. If these differences also exist in young children, they could help doctors diagnose autism earlier. Future studies should investigate whether the differences in brain activity cause the symptoms of autism. If so, it may be possible to treat the symptoms by changing brain activity, for example, by applying magnetic stimulation to the scalp.

Electrophysiological Source Imaging: A Noninvasive Window to Brain Dynamics

Brain activity and connectivity are distributed in the three-dimensional space and evolve in time. It is important to image brain dynamics with high spatial and temporal resolution. Electroencephalography (EEG) and magnetoencephalography (MEG) are noninvasive measurements associated with complex neural activations and interactions that encode brain functions. Electrophysiological source imaging estimates the underlying brain electrical sources from EEG and MEG measurements. It offers increasingly improved spatial resolution and intrinsically high temporal resolution for imaging large-scale brain activity and connectivity on a wide range of timescales. Integration of electrophysiological source imaging and functional magnetic resonance imaging could further enhance spatiotemporal resolution and specificity to an extent that is not attainable with either technique alone. We review methodological developments in electrophysiological source imaging over the past three decades and envision its future advancement into a powerful functional neuroimaging technology for basic and clinical neuroscience applications.

Deactivation in the posterior mid-cingulate cortex reflects perceptual transitions during binocular rivalry: Evidence from simultaneous EEG-fMRI

Binocular rivalry is a phenomenon in which perception spontaneously shifts between two different images that are dichoptically presented to the viewer. By elucidating the cortical networks responsible for these stochastic fluctuations in perception, we can potentially learn much about the neural correlates of visual awareness. We obtained concurrent EEG-fMRI data for a group of 20 healthy human subjects during the continuous presentation of dichoptic visual stimuli. The two eyes’ images were tagged with different temporal frequencies so that eye specific steady-state visual evoked potential (SSVEP) signals could be extracted from the EEG data for direct comparison with changes in fMRI BOLD activity associated with binocular rivalry. We additionally included a smooth replay condition that emulated the perceptual transitions experienced during binocular rivalry as a control stimulus. We evaluated a novel SSVEP-informed fMRI analysis in this study in order to delineate the temporal dynamics of rivalry-related BOLD activity from both an electrophysiological and behavioral perspective. In this manner, we assessed BOLD activity during rivalry that was directly correlated with peaks and crosses of the two rivaling, frequency-tagged SSVEP signals, for comparison with BOLD activity associated with subject reported perceptual transitions. Our findings point to a critical role of a right lateralized fronto-parietal network in the processing of bistable stimuli, given that BOLD activity in the right superior/inferior parietal lobules was significantly elevated throughout binocular rivalry and in particular during perceptual transitions, compared with the replay condition. Based on the SSVEP-informed analysis, rivalry was further associated with significantly enhanced BOLD suppression in the posterior mid-cingulate cortex during perceptual transitions, compared with SSVEP crosses. Overall, this work points to a careful interplay between early visual areas, the right posterior parietal cortex and the mid-cingulate cortex in mediating the spontaneous perceptual changes associated with binocular rivalry and has significant implications for future multimodal imaging studies of perception and awareness.