Top-down modulation in human visual cortex predicts the stability of a perceptual illusion

Skill and experience impact neural activity during global and local biological motion processing

During biological motion perception, individuals with perceptual experience learn to use more global processing, simultaneously extracting information from multiple body segments. Less experienced observers may use more local processing of individual body segments. The parietal lobe (e.g., alpha and beta power) has been shown to be critical to global and local static stimulus perception. Therefore, in this paper, we examined how skill impacts motion processing by assessing behavioral and neural responses to degrading global or local motion information for soccer penalty kicks. Skilled (N = 21) and less skilled (N = 19) soccer players anticipated temporally occluded videos of penalty kicks under normal, blurred (degraded local information), or spatially occluded (hips-only; degraded global information) viewing conditions. EEG was used to measure parietal alpha and beta power. Skilled players outperformed less skilled players, albeit both skill groups were less accurate in the blurred and hips-only conditions. Skilled performers showed significant decreases in bilateral parietal beta power in the hips-only condition, suggesting a greater reliance on global motion information under normal viewing conditions. Additionally, the hips-only condition elicited significantly greater beta relative to alpha power (beta - alpha), lower beta power, and lower alpha power than the control condition for both skill groups, suggesting spatial occlusion elicited a shift towards more local processing. Our novel findings demonstrate that skill and experience impact how motion is processed.

Beta oscillations in vision: a (preconscious) neural mechanism for the dorsal visual stream?

Neural oscillations in alpha (8-12 Hz) and beta (13-30 Hz) frequency bands are thought to reflect feedback/reentrant loops and large-scale cortical interactions. In the last decades a main effort has been made in linking perception with alpha-band oscillations, with converging evidence showing that alpha oscillations have a key role in the temporal and featural binding of visual input, configuring the alpha rhythm a key determinant of conscious visual experience. Less attention has been historically dedicated to link beta oscillations and visual processing. Nonetheless, increasing studies report that task conditions that require to segregate/integrate stimuli in space, to disentangle local/global shapes, to spatially reorganize visual inputs, and to achieve motion perception or form-motion integration, rely on the activity of beta oscillations, with a main hub in parietal areas. In the present review, we summarize the evidence linking oscillations within the beta band and visual perception. We propose that beta oscillations represent a neural code that supports the functionality of the magnocellular-dorsal (M-D) visual pathway, serving as a fast primary neural code to exert top-down influences on the slower parvocellular-ventral visual pathway activity. Such M-D-related beta activity is proposed to act mainly pre-consciously, providing the spatial coordinates of vision and guiding the conscious extraction of objects identity that are achieved with slower alpha rhythms in ventral areas. Finally, within this new theoretical framework, we discuss the potential role of M-D-related beta oscillations in visuo-spatial attention, oculo-motor behavior and reading (dis)abilities.

β Band Rhythms Influence Reaction Times

Despite their involvement in many cognitive functions, β oscillations are among the least understood brain rhythms. Reports on whether the functional role of β is primarily inhibitory or excitatory have been contradictory. Our framework attempts to reconcile these findings and proposes that several β rhythms co-exist at different frequencies. β Frequency shifts and their potential influence on behavior have thus far received little attention. In this human magnetoencephalography (MEG) experiment, we asked whether changes in β power or frequency in auditory cortex and motor cortex influence behavior (reaction times) during an auditory sweep discrimination task. We found that in motor cortex, increased β power slowed down responses, while in auditory cortex, increased β frequency slowed down responses. We further characterized β as transient burst events with distinct spectro-temporal profiles influencing reaction times. Finally, we found that increased motor-to-auditory β connectivity also slowed down responses. In sum, β power, frequency, bursting properties, cortical focus, and connectivity profile all influenced behavioral outcomes. Our results imply that the study of β oscillations requires caution as β dynamics are multifaceted phenomena, and that several dynamics must be taken into account to reconcile mixed findings in the literature.

Impact of Visual Game-Like Features on Cognitive Performance in a Virtual Reality Working Memory Task: Within-Subjects Experiment

BACKGROUND

Although the pursuit of improved cognitive function through working memory training has been the subject of decades of research, the recent growth in commercial adaptations of classic working memory tasks in the form of gamified apps warrants additional scrutiny. In particular, the emergence of virtual reality as a platform for cognitive training presents opportunities for the use of novel visual features.

OBJECTIVE

This study aimed to add to the body of knowledge regarding the use of game-like visual design elements by specifically examining the application of two particular visual features common to virtual reality environments: immersive, colorful backgrounds and the use of 3D depth. In addition, electroencephalography (EEG) data were collected to identify potential neural correlates of any observed changes in performance.

METHODS

A simple visual working memory task was presented to participants in several game-like adaptations, including the use of colorful, immersive backgrounds and 3D depth. The impact of each adaptation was separately assessed using both EEG and performance assessment outcomes and compared with an unmodified version of the task.

RESULTS

Results suggest that although accuracy and reaction time may be slightly affected by the introduction of such game elements, the effects were small and not statistically significant. Changes in EEG power, particularly in the beta and theta rhythms, were significant but failed to correlate with any corresponding changes in performance. Therefore, they may only reflect cognitive changes at the perceptual level.

CONCLUSIONS

Overall, the data suggest that the addition of these specific visual features to simple cognitive tasks does not appear to significantly affect performance or task-dependent cognitive load.

Neural oscillations promoting perceptual stability and perceptual memory during bistable perception

Ambiguous images elicit bistable perception, wherein periods of momentary perceptual stability are interrupted by sudden perceptual switches. When intermittently presented, ambiguous images trigger a perceptual memory trace in the intervening blank periods. Understanding the neural bases of perceptual stability and perceptual memory during bistable perception may hold clues for explaining the apparent stability of visual experience in the natural world, where ambiguous and fleeting images are prevalent. Motivated by recent work showing the involvement of the right inferior frontal gyrus (rIFG) in bistable perception, we conducted a transcranial direct-current stimulation (tDCS) study with a double-blind, within-subject cross-over design to test a potential causal role of rIFG in these processes. Subjects viewed ambiguous images presented continuously or intermittently while under EEG recording. We did not find any significant tDCS effect on perceptual behavior. However, the fluctuations of oscillatory power in the alpha and beta bands predicted perceptual stability, with higher power corresponding to longer percept durations. In addition, higher alpha and beta power predicted enhanced perceptual memory during intermittent viewing. These results reveal a unified neurophysiological mechanism sustaining perceptual stability and perceptual memory when the visual system is faced with ambiguous input.

Bi-Stable Perception: Self-Coordinating Brain Regions to Make-Up the Mind

Bi-stable perception is a strong instance of cognitive self-organization, providing a research model for how ‘the brain makes up its mind.’ The complexity of perceptual bistability prevents a simple attribution of functions to areas, because many cognitive processes, recruiting multiple brain regions, are simultaneously involved. The functional magnetic resonance imaging (fMRI) evidence suggests the activation of a large network of distant brain areas. Concurrently, electroencephalographic and magnetoencephalographic (MEEG) literature shows sub second oscillatory activity and phase synchrony on several frequency bands. Strongly represented are beta and gamma bands, often associated with neural/cognitive integration processes. The spatial extension and short duration of brain activities suggests the need for a fast, large-scale neural coordination mechanism. To address the range of temporo-spatial scales involved, we systematize the current knowledge from mathematical models, cognitive sciences and neuroscience at large, from single-cell- to system-level research, including evidence from human and non-human primates. Surprisingly, despite evidence spanning through different organization levels, models, and experimental approaches, the scarcity of integrative studies is evident. In a final section of the review we dwell on the reasons behind such scarcity and on the need of integration in order to achieve a real understanding of the complexities underlying bi-stable perception processes.

Separable pupillary signatures of perception and action during perceptual multistability

The pupil provides a rich, non-invasive measure of the neural bases of perception and cognition and has been of particular value in uncovering the role of arousal-linked neuromodulation, which alters both cortical processing and pupil size. But pupil size is subject to a multitude of influences, which complicates unique interpretation. We measured pupils of observers experiencing perceptual multistability-an ever-changing subjective percept in the face of unchanging but inconclusive sensory input. In separate conditions, the endogenously generated perceptual changes were either task-relevant or not, allowing a separation between perception-related and task-related pupil signals. Perceptual changes were marked by a complex pupil response that could be decomposed into two components: a dilation tied to task execution and plausibly indicative of an arousal-linked noradrenaline surge, and an overlapping constriction tied to the perceptual transient and plausibly a marker of altered visual cortical representation. Constriction, but not dilation, amplitude systematically depended on the time interval between perceptual changes, possibly providing an overt index of neural adaptation. These results show that the pupil provides a simultaneous reading on interacting but dissociable neural processes during perceptual multistability, and suggest that arousal-linked neuromodulator release shapes action but not perception in these circumstances.

Reduced alpha amplitudes predict perceptual suppression

The amplitude of prestimulus alpha oscillations over parieto-occipital cortex has been shown to predict visual detection of masked and threshold-level stimuli. Whether alpha activity similarly predicts target visibility in perceptual suppression paradigms, another type of illusion commonly used to investigate visual awareness, is presently unclear. Here, we examined prestimulus alpha activity in the electroencephalogram (EEG) of healthy participants in the context of a generalized flash suppression (GFS) task during which salient target stimuli are rendered subjectively invisible in a subset of trials following the onset of a full-field motion stimulus. Unlike for masking or threshold paradigms, alpha (8-12 Hz) amplitude prior to motion onset was significantly higher when targets remained subjectively visible compared to trials during which the targets became perceptually suppressed. Furthermore, individual prestimulus alpha amplitudes strongly correlated with the individual trial-to-trial variability quenching following motion stimulus onset, indicating that variability quenching in visual cortex is closely linked to prestimulus alpha activity. We conclude that predictive correlates of conscious perception derived from perceptual suppression paradigms differ substantially from those obtained with "near threshold paradigms", possibly reflecting the effectiveness of the suppressor stimulus.

Impact of screen size on cognitive training task performance: An HMD study

To better understand the impact of different screen sizes in cognitive training, study subjects performed an adaptive training task at three separate visual angles using a head-mounted display (HMD). Cognitive load was assessed using EEG and compared with task performance (accuracy and response time) for each condition. While previous studies found performance benefits corresponding to increased screen size in memory and learning tasks, our results suggest such benefits may only apply up to a visual angle of approximately 20°, after which increases in size become inversely correlated with task performance.

Top-down control of visual cortex by the frontal eye fields through oscillatory realignment

Voluntary allocation of visual attention is controlled by top-down signals generated within the Frontal Eye Fields (FEFs) that can change the excitability of lower-level visual areas. However, the mechanism through which this control is achieved remains elusive. Here, we emulated the generation of an attentional signal using single-pulse transcranial magnetic stimulation to activate the FEFs and tracked its consequences over the visual cortex. First, we documented changes to brain oscillations using electroencephalography and found evidence for a phase reset over occipital sites at beta frequency. We then probed for perceptual consequences of this top-down triggered phase reset and assessed its anatomical specificity. We show that FEF activation leads to cyclic modulation of visual perception and extrastriate but not primary visual cortex excitability, again at beta frequency. We conclude that top-down signals originating in FEF causally shape visual cortex activity and perception through mechanisms of oscillatory realignment.

Pre-stimulus alpha predicts inattentional blindness

Pre- and post-stimulus oscillatory activity between 8 and 12 hertz, referred to as the alpha-band, correlates with conscious visual awareness of stimuli across a variety of psychophysical tasks. Within an EEG-adapted inattentional blindness task, the current study sought to examine whether this relationship holds for conscious awareness of stimuli under conditions of inattentional blindness. Noticing rates of the task-irrelevant unexpected stimulus were correlated with a significant decrease in alpha power over bilateral parietal-occipital areas during the pre-stimulus interval, and a significant decrease in alpha power over parietal-occipital regions in the right hemisphere during the post-stimulus interval. Findings are taken to imply alpha-band neural activity represents a valid correlate of consciousness that is not confounded by task relevancy or the need for report.

Boosts in brain signal variability track liberal shifts in decision bias

Adopting particular decision biases allows organisms to tailor their choices to environmental demands. For example, a liberal response strategy pays off when target detection is crucial, whereas a conservative strategy is optimal for avoiding false alarms. Using conventional time-frequency analysis of human electroencephalographic (EEG) activity, we previously showed that bias setting entails adjustment of evidence accumulation in sensory regions (Kloosterman et al., 2019), but the presumed prefrontal signature of a conservative-to-liberal bias shift has remained elusive. Here, we show that a liberal bias shift is reflected in a more unconstrained neural regime (boosted entropy) in frontal regions that is suited to the detection of unpredictable events. Overall EEG variation, spectral power and event-related potentials could not explain this relationship, highlighting that moment-to-moment neural variability uniquely tracks bias shifts. Neural variability modulation through prefrontal cortex appears instrumental for permitting an organism to adapt its biases to environmental demands.

The 'Narcissus Effect': Top-down alpha-beta band modulation of face-related brain areas during self-face processing

Self-related information, such as one's own face, is prioritized by our cognitive system. Whilst recent theoretical developments suggest that this is achieved by an interplay between bottom-up and top-down attentional mechanisms, their underlying neural dynamics are still poorly understood. Furthermore, it is still matter of discussion as to whether these attentional mechanisms are truly self-specific or instead driven by face familiarity. To address these questions, we used EEG to record the brain activity of twenty-five healthy participants whilst identifying their own face, a friend's face and a stranger's face. Time-frequency analysis revealed a greater sustained power decrease in the alpha and beta frequency bands for the self-face, which emerged at late latencies and was maintained even when the face was no longer present. Critically, source analysis showed that this activity was generated in key brain regions for self-face recognition, such as the fusiform gyrus. As in the Myth of Narcissus, our results indicate that one's own face might have the potential to hijack attention. We suggest that this effect is specific to the self and driven by a top-down attentional control mechanism, which might facilitate further processing of personally relevant events.

Action information contributes to metacognitive decision-making

Metacognitive abilities allow us to adjust ongoing behavior and modify future decisions in the absence of external feedback. Although metacognition is critical in many daily life settings, it remains unclear what information is actually being monitored and what kind of information is being used for metacognitive decisions. In the present study, we investigated whether response information connected to perceptual events contribute to metacognitive decision-making. Therefore, we recorded EEG signals during a perceptual color discrimination task while participants were asked to provide an estimate about the quality of their decision on each trial. Critically, the moment participants provided their confidence judgments varied across conditions, thereby changing the amount of action information (e.g., response competition or response fluency) available for metacognitive decisions. Results from three experiments demonstrate that metacognitive performance improved when first-order action information was available at the moment metacognitive decisions about the perceptual task had to be provided. This behavioral effect was accompanied by enhanced functional connectivity (beta phase synchrony) between motor areas and prefrontal regions, exclusively observed during metacognitive decision-making. Our findings demonstrate that action information contributes to metacognitive decision-making, thereby painting a picture of metacognition as a process that integrates sensory evidence and information about our interactions with the world.

Top-down and bottom-up mechanisms as reflected by beta and gamma oscillations in speech perception: An individual-difference approach

Recent neurophysiological studies have proposed distinct roles of β and γ oscillations in implementing top-down and bottom-up processes. The present study aims to test this hypothesis in the domain of speech perception. We examined β and γ oscillations elicited to a tone contrast in a passive oddball paradigm, and their relationships with discrimination sensitivity d' and RT from two groups of healthy adults who showed high and low discrimination sensitivity to the contrast. The low-sensitivity group showed a significant reduction in β, which was further related to d'. Individual differences in RT were related to different frequency bands in the two groups, with a RT-β correlation in the low-sensitivity group, and a RT-γ relation in the high-sensitivity group. Based on these findings, we suggest that β, implicated in top-down processing, reflects individual differences in phonological representations, and that γ, involved in bottom-up processing, reflects individual differences in acoustic encoding.

Neonatal EEG linked to individual differences in socioemotional outcomes and autism risk in toddlers

Research using electroencephalography (EEG) as a measure of brain function and maturation has demonstrated links between cortical activity and cognitive processes during infancy and early childhood. The current study examines whether neonatal EEG is correlated with later atypical socioemotional behaviors or neurocognitive delays. Parental report developmental assessments were administered to families with children ages 24 to 36 months who had previously participated in a neonatal EEG study (N = 129). Significant associations were found between neonatal EEG (higher frequencies in the frontal polar, temporal, and parietal brain regions) and BITSEA ASD risk scores. Infants with lower EEG power in these brain areas were more likely to have higher risk of socioemotional problems. When examining sex differences, significant links were found for males but not for females. These results demonstrate some promising associations between early neural biomarkers and later risk for atypical behaviors, which may shape early neurobehavioral development and could lead to earlier identification and intervention.

The Relationship between Trial-by-Trial Variability and Oscillations of Cortical Population Activity

Neural activity fluctuates over time, creating considerable variability across trials. This trial-by-trial neural variability is dramatically reduced (“quenched”) after the presentation of sensory stimuli. Likewise, the power of neural oscillations, primarily in the alpha-beta band, is also reduced after stimulus onset. Despite their similarity, these phenomena have so far been studied and discussed independently. We hypothesized that the two phenomena are tightly coupled in electrophysiological recordings of large cortical neural populations. To test this, we examined magnetoencephalography (MEG) recordings of healthy subjects viewing repeated presentations of a visual stimulus. The timing, amplitude, and spatial topography of variability-quenching and power-suppression were remarkably similar. Neural variability quenching was eliminated by excluding the alpha-beta band from the recordings, but not by excluding other frequency-bands. Moreover, individual magnitudes of alpha-beta band-power explained 86% of between-subject differences in variability quenching. An alternative mechanism that may generate variability quenching is increased phase alignment across trials. However, changes in inter-trial-phase-coherence (ITPC) exhibited distinct timing and no correlations with the magnitude of variability quenching in individual participants. These results reveal that neural variability quenching is tightly coupled with stimulus-induced changes in the power of alpha-beta band oscillations, associating two phenomena that have so far been studied in isolation.

Switch or stay? Automatic classification of internal mental states in bistable perception

The human brain goes through numerous cognitive states, most of these being hidden or implicit while performing a task, and understanding them is of great practical importance. However, identifying internal mental states is quite challenging as these states are difficult to label, usually short-lived, and generally, overlap with other tasks. One such problem pertains to bistable perception, which we consider to consist of two internal mental states, namely, transition and maintenance. The transition state is short-lived and represents a change in perception while the maintenance state is comparatively longer and represents a stable perception. In this study, we proposed a novel approach for characterizing the duration of transition and maintenance states and classified them from the neuromagnetic brain responses. Participants were presented with various types of ambiguous visual stimuli on which they indicated the moments of perceptual switches, while their magnetoencephalogram (MEG) data were recorded. We extracted different spatio-temporal features based on wavelet transform, and classified transition and maintenance states on a trial-by-trial basis. We obtained a classification accuracy of 79.58% and 78.40% using SVM and ANN classifiers, respectively. Next, we investigated the temporal fluctuations of these internal mental representations as captured by our classifier model and found that the accuracy showed a decreasing trend as the maintenance state was moved towards the next transition state. Further, to identify the neural sources corresponding to these internal mental states, we performed source analysis on MEG signals. We observed the involvement of sources from the parietal lobe, occipital lobe, and cerebellum in distinguishing transition and maintenance states. Cross-conditional classification analysis established generalization potential of wavelet features. Altogether, this study presents an automatic classification of endogenous mental states involved in bistable perception by establishing brain-behavior relationships at the single-trial level.

Scene Regularity Interacts With Individual Biases to Modulate Perceptual Stability

Sensory input is inherently ambiguous but our brains achieve remarkable perceptual stability. Prior experience and knowledge of the statistical properties of the world are thought to play a key role in the stabilization process. Individual differences in responses to ambiguous input and biases toward one or the other interpretation could modulate the decision mechanism for perception. However, the role of perceptual bias and its interaction with stimulus spatial properties such as regularity and element density remain to be understood. To this end, we developed novel bi-stable moving visual stimuli in which perception could be parametrically manipulated between two possible mutually exclusive interpretations: transparently or coherently moving. We probed perceptual stability across three composite stimulus element density levels with normal or degraded regularity using a factorial design. We found that increased density led to the amplification of individual biases and consequently to a stabilization of one interpretation over the alternative. This effect was reduced for degraded regularity, demonstrating an interaction between density and regularity. To understand how prior knowledge could be used by the brain in this task, we compared the data with simulations coming from four different hierarchical models of causal inference. These models made different assumptions about the use of prior information by including conditional priors that either facilitated or inhibited motion direction integration. An architecture that included a prior inhibiting motion direction integration consistently outperformed the others. Our results support the hypothesis that direction integration based on sensory likelihoods maybe the default processing mode with conditional priors inhibiting integration employed in order to help motion segmentation and transparency perception.

Humans strategically shift decision bias by flexibly adjusting sensory evidence accumulation

Decision bias is traditionally conceptualized as an internal reference against which sensory evidence is compared. Instead, we show that individuals implement decision bias by shifting the rate of sensory evidence accumulation toward a decision bound. Participants performed a target detection task while we recorded EEG. We experimentally manipulated participants' decision criterion for reporting targets using different stimulus-response reward contingencies, inducing either a liberal or a conservative bias. Drift diffusion modeling revealed that a liberal strategy biased sensory evidence accumulation toward target-present choices. Moreover, a liberal bias resulted in stronger midfrontal pre-stimulus 2-6 Hz (theta) power and suppression of pre-stimulus 8-12 Hz (alpha) power in posterior cortex. Alpha suppression in turn was linked to the output activity in visual cortex, as expressed through 59-100 Hz (gamma) power. These findings show that observers can intentionally control cortical excitability to strategically bias evidence accumulation toward the decision bound that maximizes reward.

The Degree of Modulation of Beta Band Activity During Motor Planning Is Related to Trait Impulsivity

Impulsivity is a prominent personality trait, and a key modulating component of neurologic and psychiatric disorders. How impulsivity is related to the brain mechanisms associated with action planning is poorly understood. Here, we investigated the relation between impulsivity and the modulation of beta band oscillatory activity associated with action planning and execution. Given that beta power decreases during action planning and decreases further during action execution, we hypothesized that during planning the level of beta band power of more impulsive individuals would be closer to the level reached during execution than that of less impulsive individuals. This could explain the tendency to "jump the gun" (commission errors) in high impulsivity. To test this hypothesis, we recruited healthy volunteers (50 participants analyzed) and used the Barratt Impulsiveness Scale questionnaire to evaluate their impulsivity as high or low. We then recorded their brain neuromagnetic signals while they performed an instructed-delay task that induced different levels of action planning by varying the number of spatial cues, hence the uncertainty, about the location of the upcoming target. During the early cue period of the task, we found a posterior (source localized in the occipito-parietal areas) and a left fronto-central group of channels (source localized in the left sensorimotor areas) where beta power was modulated by number of cues, whereas during the late cue period only the left fronto-central group was modulated. We found that the decrease of relative beta band power during action planning in the left fronto-central group of channels was more pronounced in the high impulsivity group than in the low impulsivity group. In addition, we found that the beta band-mediated functional connectivity between the posterior and the left fronto-central groups of channels was weaker in the high impulsivity group than in the low impulsivity group during the early cue period. Furthermore, high impulsives made more commission and movement errors in the task than low impulsives. These results reveal neural mechanisms through which impulsivity affects action planning and open the way for further study of the role of beta band activity in impulsivity, especially in the context of disease and therapeutics.

Beta and Theta Oscillations Differentially Support Free Versus Forced Control over Multiple-Target Search

Many important situations require human observers to simultaneously search for more than one object. Despite a long history of research into visual search, the behavioral and neural mechanisms associated with multiple-target search are poorly understood. Here we test the novel theory that the efficiency of looking for multiple targets critically depends on the mode of cognitive control the environment affords to the observer. We used an innovative combination of electroencephalogram (EEG) and eye tracking while participants searched for two targets, within two different contexts: either both targets were present in the search display and observers were free to prioritize either one of them, thus enabling proactive control over selection; or only one of the two targets would be present in each search display, which requires reactive control to reconfigure selection when the wrong target has been prioritized. During proactive control, both univariate and multivariate signals of beta-band (15-35 Hz) power suppression before display onset predicted switches between target selections. This signal originated over midfrontal and sensorimotor regions and has previously been associated with endogenous state changes. In contrast, imposed target selections requiring reactive control elicited prefrontal power enhancements in the delta/theta band (2-8 Hz), but only after display onset. This signal predicted individual differences in associated oculomotor switch costs, reflecting reactive reconfiguration of target selection. The results provide compelling evidence that multiple target representations are differentially prioritized during visual search, and for the first time reveal distinct neural mechanisms underlying proactive and reactive control over multiple-target search.SIGNIFICANCE STATEMENT Searching for more than one object in complex visual scenes can be detrimental for search performance. Although perhaps annoying in daily life, this can have severe consequences in professional settings such as medical and security screening. Previous research has not yet resolved whether multiple-target search involves changing priorities in what people attend to, and how such changes are controlled. We approached these questions by concurrently measuring cortical activity and eye movements using EEG and eye tracking while observers searched for multiple possible targets. Our findings provide the first unequivocal support for the existence of two modes of control during multiple-target search, which are expressed in qualitatively distinct time-frequency signatures of the EEG both before and after visual selection.

Pupillary responses to a cognitive effort task in schizophrenia

Effort-based decision making paradigms are increasingly utilized to gain insight into the nature of motivation deficits. Although these tasks are being used to assess effort and motivation in schizophrenia, little work has been done to confirm that effort-based decision making tasks validly manipulate effort. In the current study, we adapted the effort component a cognitive effort-based decision making task (the Deck Choice Effort Task) for use with pupillometric assessment. We sought to confirm with psychophysiology that cognitive effort is manipulated. We also examined correlations between physiological indicators of effort exertion and cognition and negative symptoms. The results confirmed manipulation of cognitive effort: there was a significant difference in pupillary responses between easy and difficult task conditions. Pupillary responses were also correlated with cognitive ability, and with negative symptoms when controlling for cognition. Thus, our findings offer physiological validation of an effort manipulation included in a cognitive effort-based decision making task for schizophrenia.

Surprise About Sensory Event Timing Drives Cortical Transients in the Beta Frequency Band

Learning the statistical structure of the environment is crucial for adaptive behavior. Humans and nonhuman decision-makers seem to track such structure through a process of probabilistic inference, which enables predictions about behaviorally relevant events. Deviations from such predictions cause surprise, which in turn helps improve inference. Surprise about the timing of behaviorally relevant sensory events drives phasic responses of neuromodulatory brainstem systems, which project to the cerebral cortex. Here, we developed a computational model-based magnetoencephalography (MEG) approach for mapping the resulting cortical transients across space, time, and frequency, in the human brain (N = 28, 17 female). We used a Bayesian ideal observer model to learn the statistics of the timing of changes in a simple visual detection task. This model yielded quantitative trial-by-trial estimates of temporal surprise. The model-based surprise variable predicted trial-by-trial variations in reaction time more strongly than the externally observable interval timings alone. Trial-by-trial variations in surprise were negatively correlated with the power of cortical population activity measured with MEG. This surprise-related power suppression occurred transiently around the behavioral response, specifically in the beta frequency band. It peaked in parietal and prefrontal cortices, remote from the motor cortical suppression of beta power related to overt report (button press) of change detection. Our results indicate that surprise about sensory event timing transiently suppresses ongoing beta-band oscillations in association cortex. This transient suppression of frontal beta-band oscillations might reflect an active reset triggered by surprise, and is in line with the idea that beta-oscillations help maintain cognitive sets.SIGNIFICANCE STATEMENT The brain continuously tracks the statistical structure of the environment to anticipate behaviorally relevant events. Deviations from such predictions cause surprise, which in turn drives neural activity in subcortical brain regions that project to the cerebral cortex. We used magnetoencephalography in humans to map out surprise-related modulations of cortical population activity across space, time, and frequency. Surprise was elicited by variable timing of visual stimulus changes requiring a behavioral response. Surprise was quantified by means of an ideal observer model. Surprise predicted behavior as well as a transient suppression of beta frequency-band oscillations in frontal cortical regions. Our results are in line with conceptual accounts that have linked neural oscillations in the beta-band to the maintenance of cognitive sets.

From ERPs to MVPA Using the Amsterdam Decoding and Modeling Toolbox (ADAM)

In recent years, time-resolved multivariate pattern analysis (MVPA) has gained much popularity in the analysis of electroencephalography (EEG) and magnetoencephalography (MEG) data. However, MVPA may appear daunting to those who have been applying traditional analyses using event-related potentials (ERPs) or event-related fields (ERFs). To ease this transition, we recently developed the Amsterdam Decoding and Modeling (ADAM) toolbox in MATLAB. ADAM is an entry-level toolbox that allows a direct comparison of ERP/ERF results to MVPA results using any dataset in standard EEGLAB or Fieldtrip format. The toolbox performs and visualizes multiple-comparison corrected group decoding and forward encoding results in a variety of ways, such as classifier performance across time, temporal generalization (time-by-time) matrices of classifier performance, channel tuning functions (CTFs) and topographical maps of (forward-transformed) classifier weights. All analyses can be performed directly on raw data or can be preceded by a time-frequency decomposition of the data in which case the analyses are performed separately on different frequency bands. The figures ADAM produces are publication-ready. In the current manuscript, we provide a cookbook in which we apply a decoding analysis to a publicly available MEG/EEG dataset involving the perception of famous, non-famous and scrambled faces. The manuscript covers the steps involved in single subject analysis and shows how to perform and visualize a subsequent group-level statistical analysis. The processing pipeline covers computation and visualization of group ERPs, ERP difference waves, as well as MVPA decoding results. It ends with a comparison of the differences and similarities between EEG and MEG decoding results. The manuscript has a level of description that allows application of these analyses to any dataset in EEGLAB or Fieldtrip format.

A Bayesian model comparison approach to test the specificity of visual integration impairment in schizophrenia or psychosis

Impaired visual integration is well documented in schizophrenia and related to functional outcomes. However, it is unclear if this deficit is specific to schizophrenia, or characteristic of psychosis more broadly. To address this question, this study used a Bayesian model comparison approach to examine the evidence of three grouping models of visual integration performance in 116 individuals with schizophrenia (SZ), schizoaffective disorder (SA), bipolar disorder (BD) with or without a history of prominent psychosis (BDP+ and BDP-, respectively), or no psychiatric diagnosis (healthy controls; HC). We compared: (1) Psychosis Model (psychosis, non-psychosis), where the psychosis group included SZ, SA, and BDP+, and the non-psychosis group included BDP- and HC; (2) Schizophrenia Model (SZ, non-SZ); and (3) DSM Model (SZ, SA, BD, HC). The relationship between visual integration and general cognition was also explored. The Psychosis Model showed the strongest evidence, and visual integration was associated with general cognition in participants with psychosis. The results were consistent with the Research Domain Criteria (RDoC) framework, indicating that visual integration impairment is characteristic of psychosis and not specific to SZ or DSM categories, and may share similar disease pathways with observed neurocognitive deficits in psychotic disorders.

Heading representations in primates are compressed by saccades

Perceptual illusions help to understand how sensory signals are decoded in the brain. Here we report that the opposite approach is also applicable, i.e., results from decoding neural activity from monkey extrastriate visual cortex correctly predict a hitherto unknown perceptual illusion in humans. We record neural activity from monkey medial superior temporal (MST) and ventral intraparietal (VIP) area during presentation of self-motion stimuli and concurrent reflexive eye movements. A heading-decoder performs veridically during slow eye movements. During fast eye movements (saccades), however, the decoder erroneously reports compression of heading toward straight ahead. Functional equivalents of macaque areas MST and VIP have been identified in humans, implying a perceptual correlate (illusion) of this perisaccadic decoding error. Indeed, a behavioral experiment in humans shows that perceived heading is perisaccadically compressed toward the direction of gaze. Response properties of primate areas MST and VIP are consistent with being the substrate of the newly described visual illusion.Macaque higher visual areas MST and VIP encode heading direction based on self-motion stimuli. Here the authors show that, while making saccades, the heading direction decoded from the neural responses is compressed toward straight-ahead, and independently demonstrate a perceptual illusion in humans based on this perisaccadic decoding error.

Decoupling of BOLD amplitude and pattern classification of orientation-selective activity in human visual cortex

Multivariate pattern analysis (MVPA) of fMRI data has allowed the investigation of neural representations of stimuli on the basis of distributed patterns of activity within a brain region, independently from overall brain activity. For instance, several studies on early visual cortex have reported reliable MVPA decoding of the identity of a stimulus representation that was kept in working memory or internally generated, despite the fact that the overall BOLD response was low or even at baseline levels. Here we ask how it is possible that reliable stimulus information can be decoded from early visual cortex even when the overall BOLD signal remains low. We reanalyzed a data set in which human participants (N = 24) imagined or kept in working memory an oriented visual grating. We divided voxels from V1, V2, and V3 into groups based on orientation preference, and compared the time course of mean BOLD responses to preferred and non-preferred orientations with the time course of the multivariate decoding performance. Decoding accuracy related to a numerically small, but reliable univariate difference in the mean BOLD response to preferred and non-preferred stimuli. The time course of the difference in BOLD responses to preferred and non-preferred orientations was highly similar to the time course of the multivariate pattern classification accuracy. The reliability of the classification strongly correlated with the magnitude of differences in BOLD signal between preferred and non-preferred stimuli. These activity differences were small compared to the large overall BOLD modulations. This suggests that a substantial part of the task-related BOLD response to visual stimulation might not be stimulus-specific. Rather, stimulus-evoked BOLD signals in early visual cortex during a task context may be an amalgam of small stimulus-specific responses and large task-related but non-stimulus-specific responses. The latter are not evident during the maintenance or internal generation of stimulus representations, but provide an explanation of how reliable stimulus information can be decoded from early visual cortex even though its overall BOLD signal remains low.

Multistable Perception and the Role of the Frontoparietal Cortex in Perceptual Inference

A given pattern of optical stimulation can arise from countless possible real-world sources, creating a dilemma for vision: What in the world actually gives rise to the current pattern? This dilemma was pointed out centuries ago by the astronomer and mathematician Ibn Al-Haytham and was forcefully restated 150 years ago when von Helmholtz characterized perception as unconscious inference. To buttress his contention, von Helmholtz cited multistable perception: recurring changes in perception despite unchanging sensory input. Recent neuroscientific studies have exploited multistable perception to identify brain areas uniquely activated in association with these perceptual changes, but the specific roles of those activations remain controversial. This article provides an overview of theoretical models of multistable perception, a review of recent neuroimaging and brain stimulation studies focused on mechanisms associated with these perceptual changes, and a synthesis of available evidence within the context of current notions about Bayesian inference that find their historical roots in von Helmholtz's work.

Simulated forward and backward self motion, based on realistic parameters, causes motion induced blindness

Motion Induced Blindness (MIB) is a well-established visual phenomenon whereby highly salient targets disappear when viewed against a moving background mask. No research has yet explored whether contracting and expanding optic flow can also trigger target disappearance. We explored MIB using mask speeds corresponding to driving at 35, 50, 65 and 80 km/h in simulated forward (expansion) and backward (contraction) motion as well as 2-D radial movement, random, and static mask motion types. Participants (n = 18) viewed MIB targets against masks with different movement types, speed, and target locations. To understand the relationship between saccades, pupil response and perceptual disappearance, we ran two additional eye-tracking experiments (n = 19). Target disappearance increased significantly with faster mask speeds and upper visual field target presentation. Simulated optic flow and 2-D radial movement caused comparable disappearance, and all moving masks caused significantly more disappearance than a static mask. Saccades could not entirely account for differences between conditions, suggesting that self-motion optic flow does cause MIB in an artificial setting. Pupil analyses implied that MIB disappearance induced by optic flow is not subjectively salient, potentially explaining why MIB is not noticed during driving. Potential implications of MIB for driving safety and Head-Up-Display (HUD) technologies are discussed.

Beyond the Status Quo: A Role for Beta Oscillations in Endogenous Content (Re)Activation

Among the rhythms of the brain, oscillations in the beta frequency range (∼13-30 Hz) have been considered the most enigmatic. Traditionally associated with sensorimotor functions, beta oscillations have recently become more broadly implicated in top-down processing, long-range communication, and preservation of the current brain state. Here, we extend and refine these views based on accumulating new findings of content-specific beta-synchronization during endogenous information processing in working memory (WM) and decision making. We characterize such content-specific beta activity as short-lived, flexible network dynamics supporting the endogenous (re)activation of cortical representations. Specifically, we suggest that beta-mediated ensemble formation within and between cortical areas may awake, rather than merely preserve, an endogenous cognitive set in the service of current task demands. This proposal accommodates key aspects of content-specific beta modulations in monkeys and humans, integrates with timely computational models, and outlines a functional role for beta that fits its transient temporal characteristics.

Sure I'm Sure: Prefrontal Oscillations Support Metacognitive Monitoring of Decision Making

Successful decision making critically involves metacognitive processes such as monitoring and control of our decision process. Metacognition enables agents to modify ongoing behavior adaptively and determine what to do next in situations in which external feedback is not (immediately) available. Despite the importance of metacognition for many aspects of life, little is known about how our metacognitive system operates or about what kind of information is used for metacognitive (second-order) judgments. In particular, it remains an open question whether metacognitive judgments are based on the same information as first-order decisions. Here, we investigated the relationship between metacognitive performance and first-order task performance by recording EEG signals while participants were asked to make a "diagnosis" after seeing a sample of fictitious patient data (a complex pattern of colored moving dots of different sizes). To assess metacognitive performance, participants provided an estimate about the quality of their diagnosis on each trial. Results demonstrate that the information that contributes to first-order decisions differs from the information that supports metacognitive judgments. Further, time-frequency analyses of EEG signals reveal that metacognitive performance is associated specifically with prefrontal theta-band activity. Together, our findings are consistent with a hierarchical model of metacognition and suggest a crucial role for prefrontal oscillations in metacognitive performance.

SIGNIFICANCE STATEMENT

Monitoring and control of our decision process (metacognition) is a crucial aspect of adaptive decision making. Crucially, metacognitive skills enable us to adjust ongoing behavior and determine future decision making when immediate feedback is not available. In the present study, we constructed a "diagnosis task" that allowed us to assess in what way first-order task performance and metacognition are related to each other. Results demonstrate that the contribution of sensory evidence (size, color, and motion direction) differs between first- and second-order decision making. Further, our results indicate that metacognitive performance specifically is orchestrated by means of prefrontal theta oscillations. Together, our findings suggest a hierarchical model of metacognition.

Alpha Power Predicts Persistence of Bistable Perception

Perception is strongly affected by the intrinsic state of the brain, which controls the propensity to either maintain a particular perceptual interpretation or switch to another. To understand the mechanisms underlying the spontaneous drive of the brain to explore alternative interpretations of unchanging stimuli, we repeatedly recorded high-density EEG after normal sleep and after sleep deprivation while participants observed a Necker cube image and reported the durations of the alternating representations of their bistable perception. We found that local alpha power around the parieto-occipital sulcus within the first second after the emergence of a perceptual representation predicted the fate of its duration. An experimentally induced increase in alpha power by means of sleep deprivation increased the average duration of individual representations. Taken together, these findings show that high alpha power promotes the stability of a perceptual representation and suppresses switching to the alternative. The observations support the hypothesis that synchronization of alpha oscillations across a wide neuronal network promotes the maintenance and stabilization of its current perceptual representation. Elevated alpha power could also be key to the poorly understood cognitive deficits, that typically accompany sleep deprivation, such as the loss of mental flexibility and lapses of responsiveness.

Multiple Transient Signals in Human Visual Cortex Associated with an Elementary Decision

The cerebral cortex continuously undergoes changes in its state, which are manifested in transient modulations of the cortical power spectrum. Cortical state changes also occur at full wakefulness and during rapid cognitive acts, such as perceptual decisions. Previous studies found a global modulation of beta-band (12–30 Hz) activity in human and monkey visual cortex during an elementary visual decision: reporting the appearance or disappearance of salient visual targets surrounded by a distractor. The previous studies disentangled neither the motor action associated with behavioral report nor other secondary processes, such as arousal, from perceptual decision processing per se. Here, we used magnetoencephalography in humans to pinpoint the factors underlying the beta-band modulation. We found that disappearances of a salient target were associated with beta-band suppression, and target reappearances with beta-band enhancement. This was true for both overt behavioral reports (immediate button presses) and silent counting of the perceptual events. This finding indicates that the beta-band modulation was unrelated to the execution of the motor act associated with a behavioral report of the perceptual decision. Further, changes in pupil-linked arousal, fixational eye movements, or gamma-band responses were not necessary for the beta-band modulation. Together, our results suggest that the beta-band modulation was a top-down signal associated with the process of converting graded perceptual signals into a categorical format underlying flexible behavior. This signal may have been fed back from brain regions involved in decision processing to visual cortex, thus enforcing a “decision-consistent” cortical state.

SIGNIFICANCE STATEMENT Elementary visual decisions are associated with a rapid state change in visual cortex, indexed by a modulation of neural activity in the beta-frequency range. Such decisions are also followed by other events that might affect the state of visual cortex, including the motor command associated with the report of the decision, an increase in pupil-linked arousal, fixational eye movements, and fluctuations in bottom-up sensory processing. Here, we ruled out the necessity of these events for the beta-band modulation of visual cortex. We propose that the modulation reflects a decision-related state change, which is induced by the conversion of graded perceptual signals into a categorical format underlying behavior. The resulting decision signal may be fed back to visual cortex.

Power modulation of electroencephalogram mu and beta frequency depends on perceived level of observed actions

INTRODUCTION

The ability to understand actions and intentions of others is of great importance to social relationships and is associated with the mirror neuron system of the human brain. Whether conscious perception of specific actions is necessary to trigger activity in this system, or alternatively whether this response is independent of conscious perception is not known.

METHODS

We addressed this issue by rendering videos of right hand movements invisible to conscious perception, and measuring electroencephalogram (EEG) power suppression in the mu (8-13 Hz) and beta (15-25 Hz) range as index corresponding to the magnitude of mirror neuron activity.

RESULTS

In the beta range over bilateral sensorimotor sites, we find that suppression indices follow the reported perceptual level of subjects with stronger suppression for consciously perceived trials. Furthermore, in the nonperceived trials, oscillation power is significantly suppressed relative to baseline. In the low mu range (8-10 Hz), oscillation power over the left sensorimotor site is significantly more suppressed in the consciously perceived versus nonperceived trials.

CONCLUSIONS

Our data suggest that the intensity of mirror system responses during action observation decreases with the observers' perception level yet remains significant during observation of invisible actions. Such subliminal activity could help explain phenomena such as covert imitation.

Aging into Perceptual Control: A Dynamic Causal Modeling for fMRI Study of Bistable Perception

Aging is accompanied by stereotyped changes in functional brain activations, for example a cortical shift in activity patterns from posterior to anterior regions is one hallmark revealed by functional magnetic resonance imaging (fMRI) of aging cognition. Whether these neuronal effects of aging could potentially contribute to an amelioration of or resistance to the cognitive symptoms associated with psychopathology remains to be explored. We used a visual illusion paradigm to address whether aging affects the cortical control of perceptual beliefs and biases. Our aim was to understand the effective connectivity associated with volitional control of ambiguous visual stimuli and to test whether greater top-down control of early visual networks emerged with advancing age. Using a bias training paradigm for ambiguous images we found that older participants (n = 16) resisted experimenter-induced visual bias compared to a younger cohort (n = 14) and that this resistance was associated with greater activity in prefrontal and temporal cortices. By applying Dynamic Causal Models for fMRI we uncovered a selective recruitment of top-down connections from the middle temporal to Lingual gyrus (LIN) by the older cohort during the perceptual switch decision following bias training. In contrast, our younger cohort did not exhibit any consistent connectivity effects but instead showed a loss of driving inputs to orbitofrontal sources following training. These findings suggest that perceptual beliefs are more readily controlled by top-down strategies in older adults and introduce age-dependent neural mechanisms that may be important for understanding aberrant belief states associated with psychopathology.

Negligible fronto-parietal BOLD activity accompanying unreportable switches in bistable perception

The human brain's executive systems play a vital role in deciding and selecting among actions. Selection among alternatives also occurs in the perceptual domain, for instance when perception switches between interpretations during perceptual bistability. Whether executive systems also underlie this functionality remains debated, with known fronto-parietal concomitants of perceptual switches being variously interpreted as reflecting the switches' cause, or as reflecting their consequences. We developed a paradigm where the two eyes receive different inputs and perception demonstrably switches between these inputs, yet where switches themselves are so inconspicuous as to become unreportable, minimizing their executive consequences. Fronto-parietal fMRI BOLD responses that accompany perceptual switches were similarly minimized in this paradigm, indicating that these reflect the switches' consequences rather than their cause. We conclude that perceptual switches do not always rely on executive brain areas, and that processes responsible for selection among alternatives may operate outside of the brain's executive systems.

Pupil size tracks perceptual content and surprise

Changes in pupil size at constant light levels reflect the activity of neuromodulatory brainstem centers that control global brain state. These endogenously driven pupil dynamics can be synchronized with cognitive acts. For example, the pupil dilates during the spontaneous switches of perception of a constant sensory input in bistable perceptual illusions. It is unknown whether this pupil dilation only indicates the occurrence of perceptual switches, or also their content. Here, we measured pupil diameter in human subjects reporting the subjective disappearance and re-appearance of a physically constant visual target surrounded by a moving pattern ('motion-induced blindness' illusion). We show that the pupil dilates during the perceptual switches in the illusion and a stimulus-evoked 'replay' of that illusion. Critically, the switch-related pupil dilation encodes perceptual content, with larger amplitude for disappearance than re-appearance. This difference in pupil response amplitude enables prediction of the type of report (disappearance vs. re-appearance) on individual switches (receiver-operating characteristic: 61%). The amplitude difference is independent of the relative durations of target-visible and target-invisible intervals and subjects' overt behavioral report of the perceptual switches. Further, we show that pupil dilation during the replay also scales with the level of surprise about the timing of switches, but there is no evidence for an interaction between the effects of surprise and perceptual content on the pupil response. Taken together, our results suggest that pupil-linked brain systems track both the content of, and surprise about, perceptual events.