Computational model links normalization to chemoarchitecture in the human visual system

A goal of cognitive neuroscience is to provide computational accounts of brain function. Canonical computations-mathematical operations used by the brain in many contexts-fulfill broad information-processing needs by varying their algorithmic parameters. A key question concerns the identification of biological substrates for these computations and their algorithms. Chemoarchitecture-the spatial distribution of neurotransmitter receptor densities-shapes brain function. Here, we propose that local variations in specific receptor densities implement algorithmic modulations of canonical computations. To test this hypothesis, we combine mathematical modeling of brain responses with chemoarchitecture data. We compare parameters of divisive normalization obtained from 7-tesla functional magnetic resonance imaging with receptor density maps obtained from positron emission tomography. We find evidence that serotonin and γ-aminobutyric acid receptor densities are the biological substrate for algorithmic modulations of divisive normalization in the human visual system. Our model links computational and biological levels of vision, explaining how canonical computations allow the brain to fulfill broad information-processing needs.

A selection and targeting framework of cortical locations for line-scanning fMRI

Depth-resolved functional magnetic resonance imaging (fMRI) is an emerging field growing in popularity given the potential of separating signals from different computational processes in cerebral cortex. Conventional acquisition schemes suffer from low spatial and temporal resolutions. Line-scanning methods allow depth-resolved fMRI by sacrificing spatial coverage to sample blood oxygenated level-dependent (BOLD) responses at ultra-high temporal and spatial resolution. For neuroscience applications, it is critical to be able to place the line accurately to (1) sample the right neural population and (2) target that neural population with tailored stimuli or tasks. To this end, we devised a multi-session framework where a target cortical location is selected based on anatomical and functional properties. The line is then positioned according to this information in a separate second session, and we tailor the experiment to focus on the target location. Anatomically, the precision of the line placement was confirmed by projecting a nominal representation of the acquired line back onto the surface. Functional estimates of neural selectivities in the line, as quantified by a visual population-receptive field model, resembled the target selectivities well for most subjects. This functional precision was quantified in detail by estimating the distance between the visual field location of the targeted vertex and the location in visual cortex (V1) that most closely resembled the line-scanning estimates; this distance was on average ~5.5 mm. Given the dimensions of the line, differences in acquisition, session, and stimulus design, this validates that line-scanning can be used to probe local neural sensitivities across sessions. In summary, we present an accurate framework for line-scanning MRI; we believe such a framework is required to harness the full potential of line-scanning and maximize its utility. Furthermore, this approach bridges canonical fMRI experiments with electrophysiological experiments, which in turn allows novel avenues for studying human physiology non-invasively.

Robust high spatio-temporal line-scanning fMRI in humans at 7T using multi-echo readouts, denoising and prospective motion correction

BACKGROUND

Functional magnetic resonance imaging (fMRI), typically using blood oxygenation level-dependent (BOLD) contrast weighted imaging, allows the study of brain function with millimeter spatial resolution and temporal resolution of one to a few seconds. At a mesoscopic scale, neurons in the human brain are spatially organized in structures with dimensions of hundreds of micrometers, while they communicate at the millisecond timescale. For this reason, it is important to develop an fMRI method with simultaneous high spatial and temporal resolution. Line-scanning promises to reach this goal at the cost of volume coverage.

NEW METHOD

Here, we release a comprehensive update to human line-scanning fMRI. First, we investigated multi-echo line-scanning with five different protocols varying the number of echoes and readout bandwidth while keeping the TR constant. In these, we compared different echo combination approaches in terms of BOLD activation (sensitivity) and temporal signal-to-noise ratio. Second, we implemented an adaptation of NOise reduction with DIstribution Corrected principal component analysis (NORDIC) thermal noise removal for line-scanning fMRI data. Finally, we tested three image-based navigators for motion correction and investigated different ways of performing fMRI analysis on the timecourses which were influenced by the insertion of the navigators themselves.

RESULTS

The presented improvements are relatively straightforward to implement; multi-echo readout and NORDIC denoising together, significantly improve data quality in terms of tSNR and t-statistical values, while motion correction makes line-scanning fMRI more robust.

COMPARISON WITH EXISTING METHODS

Multi-echo acquisitions and denoising have previously been applied in 3D magnetic resonance imaging. Their combination and application to 1D line-scanning is novel. The current proposed method greatly outperforms the previous line-scanning acquisitions with single-echo acquisition, in terms of tSNR (4.0 for single-echo line-scanning and 36.2 for NORDIC-denoised multi-echo) and t-statistical values (3.8 for single-echo line-scanning and 25.1 for NORDIC-denoised multi-echo line-scanning).

CONCLUSIONS

Line-scanning fMRI was advanced compared to its previous implementation in order to improve sensitivity and reliability. The improved line-scanning acquisition could be used, in the future, for neuroscientific and clinical applications.

Visual Working Memory Adapts to the Nature of Anticipated Interference

Visual working memory has been proven to be relatively robust against interference. However, little is known on whether such robust coding is obligatory, or can be flexibly recruited depending on its expected usefulness. To address this, participants remembered both the color and orientation of a grating. During the maintenance, we inserted a secondary color/orientation memory task, interfering with the primary task. Crucially, we varied the expectations of the type of interference by varying the probability of the two types of intervening task. Behavioral data indicate that to-be-remembered features for which interference is expected are bolstered, whereas to-be-remembered features for which no interference is expected are left vulnerable. This was further supported by fMRI data obtained from visual cortex. In conclusion, the flexibility of visual working memory allows it to strengthen memories for which it anticipates the highest risk of interference.

Visuospatial coding as ubiquitous scaffolding for human cognition

For more than 100 years we have known that the visual field is mapped onto the surface of visual cortex, imposing an inherently spatial reference frame on visual information processing. Recent studies highlight visuospatial coding not only throughout visual cortex, but also brain areas not typically considered visual. Such widespread access to visuospatial coding raises important questions about its role in wider cognitive functioning. Here, we synthesise these recent developments and propose that visuospatial coding scaffolds human cognition by providing a reference frame through which neural computations interface with environmental statistics and task demands via perception-action loops.

A line through the brain: implementation of human line-scanning at 7T for ultra-high spatiotemporal resolution fMRI

Functional magnetic resonance imaging (fMRI) is a widely used tool in neuroscience to detect neurally evoked responses, e.g. the blood oxygenation level-dependent (BOLD) signal. Typically, BOLD fMRI has millimeter spatial resolution and temporal resolution of one to few seconds. To study the sub-millimeter structures and activity of the cortical gray matter, the field needs an fMRI method with high spatial and temporal resolution. Line-scanning fMRI achieves very high spatial resolution and high sampling rate, at the cost of a sacrifice in volume coverage. Here, we present a human line-scanning implementation on a 7T MRI system. First, we investigate the quality of the saturation pulses that suppress MR signal outside the line. Second, we established the best coil combination for reconstruction. Finally, we applied the line-scanning method in the occipital lobe during a visual stimulation task, showing BOLD responses along cortical depth, every 250 µm with a 200 ms repetition time (TR). We found a good correspondence of t-statistics values with 2D gradient-echo echo planar imaging (GE-EPI) BOLD fMRI data with the same temporal resolution and voxel volume (R = 0.6 ± 0.2). In summary, we demonstrate the feasibility of line-scanning in humans and this opens line-scanning fMRI for applications in cognitive and clinical neuroscience.

Separable pupillary signatures of perception and action during perceptual multistability

The pupil provides a rich, non-invasive measure of the neural bases of perception and cognition and has been of particular value in uncovering the role of arousal-linked neuromodulation, which alters both cortical processing and pupil size. But pupil size is subject to a multitude of influences, which complicates unique interpretation. We measured pupils of observers experiencing perceptual multistability-an ever-changing subjective percept in the face of unchanging but inconclusive sensory input. In separate conditions, the endogenously generated perceptual changes were either task-relevant or not, allowing a separation between perception-related and task-related pupil signals. Perceptual changes were marked by a complex pupil response that could be decomposed into two components: a dilation tied to task execution and plausibly indicative of an arousal-linked noradrenaline surge, and an overlapping constriction tied to the perceptual transient and plausibly a marker of altered visual cortical representation. Constriction, but not dilation, amplitude systematically depended on the time interval between perceptual changes, possibly providing an overt index of neural adaptation. These results show that the pupil provides a simultaneous reading on interacting but dissociable neural processes during perceptual multistability, and suggest that arousal-linked neuromodulator release shapes action but not perception in these circumstances.

Representation of Contralateral Visual Space in the Human Hippocampus

The initial encoding of visual information primarily from the contralateral visual field is a fundamental organizing principle of the primate visual system. Recently, the presence of such retinotopic sensitivity has been shown to extend well beyond early visual cortex to regions not historically considered retinotopically sensitive. In particular, human scene-selective regions in parahippocampal and medial parietal cortex exhibit prominent biases for the contralateral visual field. Here, we used fMRI to test the hypothesis that the human hippocampus, which is thought to be anatomically connected with these scene-selective regions, would also exhibit a biased representation of contralateral visual space. First, population receptive field (pRF) mapping with scene stimuli revealed strong biases for the contralateral visual field in bilateral hippocampus. Second, the distribution of retinotopic sensitivity suggested a more prominent representation in anterior medial portions of the hippocampus. Finally, the contralateral bias was confirmed in independent data taken from the Human Connectome Project (HCP) initiative. The presence of contralateral biases in the hippocampus, a structure considered by many as the apex of the visual hierarchy, highlights the truly pervasive influence of retinotopy. Moreover, this finding has important implications for understanding how visual information relates to the allocentric global spatial representations known to be encoded therein.SIGNIFICANCE STATEMENT Retinotopic encoding of visual information is an organizing principle of visual cortex. Recent work demonstrates this sensitivity in structures far beyond early visual cortex, including those anatomically connected to the hippocampus. Here, using population receptive field (pRF) modeling in two independent sets of data we demonstrate a consistent bias for the contralateral visual field in bilateral hippocampus. Such a bias highlights the truly pervasive influence of retinotopy, with important implications for understanding how the presence of retinotopy relates to more allocentric spatial representations.

Topographic connectivity reveals task-dependent retinotopic processing throughout the human brain

The human visual system is organized as a hierarchy of maps that share the topography of the retina. Known retinotopic maps have been identified using simple visual stimuli under strict fixation, conditions different from everyday vision which is active, dynamic, and complex. This means that it remains unknown how much of the brain is truly visually organized. Here I demonstrate widespread stable visual organization beyond the traditional visual system, in default-mode network and hippocampus. Detailed topographic connectivity with primary visual cortex during movie-watching, resting-state, and retinotopic-mapping experiments revealed that visual-spatial representations throughout the brain are warped by cognitive state. Specifically, traditionally visual regions alternate with default-mode network and hippocampus in preferentially representing the center of the visual field. This visual role of default-mode network and hippocampus would allow these regions to interface between abstract memories and concrete sensory impressions. Together, these results indicate that visual-spatial organization is a fundamental coding principle that structures the communication between distant brain regions.

Ultra-high field fMRI reveals origins of feedforward and feedback activity within laminae of human ocular dominance columns

Ultra-high field MRI can functionally image the cerebral cortex of human subjects at the submillimeter scale of cortical columns and laminae. Here, we investigate both in concert, by imaging ocular dominance columns (ODCs) in primary visual cortex (V1) across different cortical depths. We ensured that putative ODC patterns in V1 (a) are stable across runs, sessions, and scanners located in different continents, (b) have a width (~1.3 mm) expected from post-mortem and animal work and (c) are absent at the retinotopic location of the blind spot. We then dissociated the effects of bottom-up thalamo-cortical input and attentional feedback processes on activity in V1 across cortical depth. Importantly, the separation of bottom-up information flows into ODCs allowed us to validly compare attentional conditions while keeping the stimulus identical throughout the experiment. We find that, when correcting for draining vein effects and using both model-based and model-free approaches, the effect of monocular stimulation is largest at deep and middle cortical depths. Conversely, spatial attention influences BOLD activity exclusively near the pial surface. Our findings show that simultaneous interrogation of columnar and laminar dimensions of the cortical fold can dissociate thalamocortical inputs from top-down processing, and allow the investigation of their interactions without any stimulus manipulation.

Cross-Task Contributions of Frontobasal Ganglia Circuitry in Response Inhibition and Conflict-Induced Slowing

Why are we so slow in choosing the lesser of 2 evils? We considered whether such slowing relates to uncertainty about the value of these options, which arises from the tendency to avoid them during learning, and whether such slowing relates to frontosubthalamic inhibitory control mechanisms. In total, 49 participants performed a reinforcement-learning task and a stop-signal task while fMRI was recorded. A reinforcement-learning model was used to quantify learning strategies. Individual differences in lose-lose slowing related to information uncertainty due to sampling, and independently, to less efficient response inhibition in the stop-signal task. Neuroimaging analysis revealed an analogous dissociation: subthalamic nucleus (STN) BOLD activity related to variability in stopping latencies, whereas weaker frontosubthalamic connectivity related to slowing and information sampling. Across tasks, fast inhibitors increased STN activity for successfully canceled responses in the stop task, but decreased activity for lose-lose choices. These data support the notion that fronto-STN communication implements a rapid but transient brake on response execution, and that slowing due to decision uncertainty could result from an inefficient release of this "hold your horses" mechanism.

Learning in Visual Regions as Support for the Bias in Future Value-Driven Choice

Reinforcement learning can bias decision-making toward the option with the highest expected outcome. Cognitive learning theories associate this bias with the constant tracking of stimulus values and the evaluation of choice outcomes in the striatum and prefrontal cortex. Decisions however first require processing of sensory input, and to date, we know far less about the interplay between learning and perception. This functional magnetic resonance imaging study (N = 43) relates visual blood oxygen level-dependent (BOLD) responses to value beliefs during choice and signed prediction errors after outcomes. To understand these relationships, which co-occurred in the striatum, we sought relevance by evaluating the prediction of future value-based decisions in a separate transfer phase where learning was already established. We decoded choice outcomes with a 70% accuracy with a supervised machine learning algorithm that was given trial-by-trial BOLD from visual regions alongside more traditional motor, prefrontal, and striatal regions. Importantly, this decoding of future value-driven choice outcomes again highlighted an important role for visual activity. These results raise the intriguing possibility that the tracking of value in visual cortex is supportive for the striatal bias toward the more valued option in future choice.

Dopaminergic medication reduces striatal sensitivity to negative outcomes in Parkinson's disease

Reduced levels of dopamine in Parkinson's disease contribute to changes in learning, resulting from the loss of midbrain neurons that transmit a dopaminergic teaching signal to the striatum. Dopamine medication used by patients with Parkinson's disease has previously been linked to behavioural changes during learning as well as to adjustments in value-based decision-making after learning. To date, however, little is known about the specific relationship between dopaminergic medication-driven differences during learning and subsequent changes in approach/avoidance tendencies in individual patients. Twenty-four Parkinson's disease patients ON and OFF dopaminergic medication and 24 healthy controls subjects underwent functional MRI while performing a probabilistic reinforcement learning experiment. During learning, dopaminergic medication reduced an overemphasis on negative outcomes. Medication reduced negative (but not positive) outcome learning rates, while concurrent striatal blood oxygen level-dependent responses showed reduced prediction error sensitivity. Medication-induced shifts in negative learning rates were predictive of changes in approach/avoidance choice patterns after learning, and these changes were accompanied by systematic striatal blood oxygen level-dependent response alterations. These findings elucidate the role of dopamine-driven learning differences in Parkinson's disease, and show how these changes during learning impact subsequent value-based decision-making.

Visual Organization of the Default Network

The default network (DN) is a brain network with correlated activities spanning frontal, parietal, and temporal cortical lobes. The DN activates for high-level cognition tasks and deactivates when subjects are actively engaged in perceptual tasks. Despite numerous observations, the role of DN deactivation remains unclear. Using computational neuroimaging applied to a large dataset of the Human Connectome Project (HCP) and to two individual subjects scanned over many repeated runs, we demonstrate that the DN selectively deactivates as a function of the position of a visual stimulus. That is, we show that spatial vision is encoded within the DN by means of deactivation relative to baseline. Our results suggest that the DN functions as a set of high-level visual regions, opening up the possibility of using vision-science tools to understand its putative function in cognition and perception.

Implicit and explicit learning in reactive and voluntary saccade adaptation

Saccades can either be elicited automatically by salient peripheral stimuli or can additionally depend on explicit cognitive goals. Similarly, it is thought that motor adaptation is driven by the combination of a more automatic, implicit process and a more explicit, cognitive process. However, the degree to which such implicit and explicit learning contribute to the adaptation of more reactive and voluntary saccades remains elusive. To study this question, we employed a global saccadic adaptation paradigm with both increasing and decreasing saccade amplitudes. We assessed the resulting adaptation using a dual state model of motor adaptation. This model decomposes learning into a fast and slow process, which are thought to constitute explicit and implicit learning, respectively. Our results show that adaptation of reactive saccades is equally driven by fast and slow learning, while fast learning is nearly absent when adapting voluntary (i.e. scanning) saccades. This pattern of results was present both when saccade gain was increased or decreased. Our results suggest that the increased cognitive demands associated with voluntary compared to reactive saccade planning interfere specifically with explicit learning.

Spatial sampling in human visual cortex is modulated by both spatial and feature-based attention

Spatial attention changes the sampling of visual space. Behavioral studies suggest that feature-based attention modulates this resampling to optimize the attended feature's sampling. We investigate this hypothesis by estimating spatial sampling in visual cortex while independently varying both feature-based and spatial attention. Our results show that spatial and feature-based attention interacted: resampling of visual space depended on both the attended location and feature (color vs. temporal frequency). This interaction occurred similarly throughout visual cortex, regardless of an area's overall feature preference. However, the interaction did depend on spatial sampling properties of voxels that prefer the attended feature. These findings are parsimoniously explained by variations in the precision of an attentional gain field. Our results demonstrate that the deployment of spatial attention is tailored to the spatial sampling properties of units that are sensitive to the attended feature.

How pupil responses track value-based decision-making during and after reinforcement learning

Cognition can reveal itself in the pupil, as latent cognitive processes map onto specific pupil responses. For instance, the pupil dilates when we make decisions and these pupil size fluctuations reflect decision-making computations during and after a choice. Surprisingly little is known, however, about how pupil responses relate to decisions driven by the learned value of stimuli. This understanding is important, as most real-life decisions are guided by the outcomes of earlier choices. The goal of this study was to investigate which cognitive processes the pupil reflects during value-based decision-making. We used a reinforcement learning task to study pupil responses during value-based decisions and subsequent decision evaluations, employing computational modeling to quantitatively describe the underlying cognitive processes. We found that the pupil closely tracks reinforcement learning processes independently across participants and across trials. Prior to choice, the pupil dilated as a function of trial-by-trial fluctuations in value beliefs about the to-be chosen option and predicted an individual's tendency to exploit high value options. After feedback a biphasic pupil response was observed, the amplitude of which correlated with participants' learning rates. Furthermore, across trials, early feedback-related dilation scaled with value uncertainty, whereas later constriction scaled with signed reward prediction errors. These findings show that pupil size fluctuations can provide detailed information about the computations underlying value-based decisions and the subsequent updating of value beliefs. As these processes are affected in a host of psychiatric disorders, our results indicate that pupillometry can be used as an accessible tool to non-invasively study the processes underlying ongoing reinforcement learning in the clinic.

How Visual Cortical Organization Is Altered by Ophthalmologic and Neurologic Disorders

Receptive fields are a core property of cortical organization. Modern neuroimaging allows routine access to visual population receptive fields (pRFs), enabling investigations of clinical disorders. Yet how the underlying neural circuitry operates is controversial. The controversy surrounds observations that measurements of pRFs can change in healthy adults as well as in patients with a range of ophthalmological and neurological disorders. The debate relates to the balance between plasticity and stability of the underlying neural circuitry. We propose that to move the debate forward, the field needs to define the implied mechanism. First, we review the pRF changes in both healthy subjects and those with clinical disorders. Then, we propose a computational model that describes how pRFs can change in healthy humans. We assert that we can correctly interpret the pRF changes in clinical disorders only if we establish the capabilities and limitations of pRF dynamics in healthy humans with mechanistic models that provide quantitative predictions.

No Evidence That Frontal Eye Field tDCS Affects Latency or Accuracy of Prosaccades

Transcranial direct current stimulation (tDCS) may be used to directly affect neural activity from outside of the skull. However, its exact physiological mechanisms remain elusive, particularly when applied to new brain areas. The frontal eye field (FEF) has rarely been targeted with tDCS, even though it plays a crucial role in control of overt and covert spatial attention. Here, we investigate whether tDCS over the FEF can affect the latency and accuracy of saccadic eye movements. Twenty-six participants performed a prosaccade task in which they made eye movements to a sudden-onset eccentric visual target (lateral saccades). After each lateral saccade, they made an eye movement back to the center (center saccades). The task was administered before, during, and after anodal or cathodal tDCS over the FEF, in a randomized, double-blind, within-subject design. One previous study (Kanai et al., 2012) found that anodal tDCS over the FEF decreased the latency of saccades contralateral to the stimulated hemisphere. We did not find the same effect: neither anodal nor cathodal tDCS influenced the latency of lateral saccades. tDCS also did not affect accuracy of lateral saccades (saccade endpoint deviation and saccade endpoint variability). For center saccades, we found some differences between the anodal and cathodal sessions, but these were not consistent across analyses (latency, endpoint variability), or were already present before tDCS onset (endpoint deviation). We tried to improve on the design of Kanai et al. (2012) in several ways, including the tDCS duration and electrode montage, which could explain the discrepant results. Our findings add to a growing number of null results, which have sparked concerns that tDCS outcomes are highly variable. Future studies should aim to establish the boundary conditions for FEF-tDCS to be effective, in addition to increasing sample size and adding additional controls such as a sham condition. At present, we conclude that it is unclear whether eye movements or other aspects of spatial attention can be affected through tDCS of the frontal eye fields.

Porcupine: A visual pipeline tool for neuroimaging analysis

The field of neuroimaging is rapidly adopting a more reproducible approach to data acquisition and analysis. Data structures and formats are being standardised and data analyses are getting more automated. However, as data analysis becomes more complicated, researchers often have to write longer analysis scripts, spanning different tools across multiple programming languages. This makes it more difficult to share or recreate code, reducing the reproducibility of the analysis. We present a tool, Porcupine, that constructs one's analysis visually and automatically produces analysis code. The graphical representation improves understanding of the performed analysis, while retaining the flexibility of modifying the produced code manually to custom needs. Not only does Porcupine produce the analysis code, it also creates a shareable environment for running the code in the form of a Docker image. Together, this forms a reproducible way of constructing, visualising and sharing one's analysis. Currently, Porcupine links to Nipype functionalities, which in turn accesses most standard neuroimaging analysis tools. Our goal is to release researchers from the constraints of specific implementation details, thereby freeing them to think about novel and creative ways to solve a given problem. Porcupine improves the overview researchers have of their processing pipelines, and facilitates both the development and communication of their work. This will reduce the threshold at which less expert users can generate reusable pipelines. With Porcupine, we bridge the gap between a conceptual and an implementational level of analysis and make it easier for researchers to create reproducible and shareable science. We provide a wide range of examples and documentation, as well as installer files for all platforms on our website: https://timvanmourik.github.io/Porcupine. Porcupine is free, open source, and released under the GNU General Public License v3.0.

Individual differences in eye blink rate predict both transient and tonic pupil responses during reversal learning

The pupil response under constant illumination can be used as a marker of cognitive processes. In the past, pupillary responses have been studied in the context of arousal and decision-making. However, recent work involving Parkinson's patients suggested that pupillary responses are additionally affected by reward sensitivity. Here, we build on these findings by examining how pupil responses are modulated by reward and loss while participants (N = 30) performed a Pavlovian reversal learning task. In fast (transient) pupil responses, we observed arousal-based influences on pupil size both during the expectation of upcoming value and the evaluation of unexpected monetary outcomes. Importantly, after incorporating eye blink rate (EBR), a behavioral correlate of striatal dopamine levels, we observed that participants with lower EBR showed stronger pupil dilation during the expectation of upcoming reward. Subsequently, when reward expectations were violated, participants with lower EBR showed stronger pupil responses after experiencing unexpected loss. Across trials, the detection of a reward contingency reversal was reflected in a slow (tonic) dilatory pupil response observed already several trials prior to the behavioral report. Interestingly, EBR correlated positively with this tonic detection response, suggesting that variability in the arousal-based detection response may reflect individual differences in striatal dopaminergic tone. Our results provide evidence that a behavioral marker of baseline striatal dopamine level (EBR) can potentially be used to describe the differential effects of value-based learning in the arousal-based pupil response.

Multistable Perception and the Role of the Frontoparietal Cortex in Perceptual Inference

A given pattern of optical stimulation can arise from countless possible real-world sources, creating a dilemma for vision: What in the world actually gives rise to the current pattern? This dilemma was pointed out centuries ago by the astronomer and mathematician Ibn Al-Haytham and was forcefully restated 150 years ago when von Helmholtz characterized perception as unconscious inference. To buttress his contention, von Helmholtz cited multistable perception: recurring changes in perception despite unchanging sensory input. Recent neuroscientific studies have exploited multistable perception to identify brain areas uniquely activated in association with these perceptual changes, but the specific roles of those activations remain controversial. This article provides an overview of theoretical models of multistable perception, a review of recent neuroimaging and brain stimulation studies focused on mechanisms associated with these perceptual changes, and a synthesis of available evidence within the context of current notions about Bayesian inference that find their historical roots in von Helmholtz's work.

Dynamic modulation of decision biases by brainstem arousal systems

Decision-makers often arrive at different choices when faced with repeated presentations of the same evidence. Variability of behavior is commonly attributed to noise in the brain's decision-making machinery. We hypothesized that phasic responses of brainstem arousal systems are a significant source of this variability. We tracked pupil responses (a proxy of phasic arousal) during sensory-motor decisions in humans, across different sensory modalities and task protocols. Large pupil responses generally predicted a reduction in decision bias. Using fMRI, we showed that the pupil-linked bias reduction was (i) accompanied by a modulation of choice-encoding pattern signals in parietal and prefrontal cortex and (ii) predicted by phasic, pupil-linked responses of a number of neuromodulatory brainstem centers involved in the control of cortical arousal state, including the noradrenergic locus coeruleus. We conclude that phasic arousal suppresses decision bias on a trial-by-trial basis, thus accounting for a significant component of the variability of choice behavior.

Oculomotor Remapping of Visual Information to Foveal Retinotopic Cortex

Our eyes continually jump around the visual scene to bring the high-resolution, central part of our vision onto objects of interest. We are oblivious to these abrupt shifts, perceiving the visual world to appear reassuringly stable. A process called remapping has been proposed to mediate this perceptual stability for attended objects by shifting their retinotopic representation to compensate for the effects of the upcoming eye movement. In everyday vision, observers make goal-directed eye movements towards items of interest bringing them to the fovea and, for these items, the remapped activity should impinge on foveal regions of the retinotopic maps in visual cortex. Previous research has focused instead on remapping for targets that were not saccade goals, where activity is remapped to a new peripheral location rather than to the foveal representation. We used functional magnetic resonance imaging (fMRI) and a phase-encoding design to investigate remapping of spatial patterns of activity towards the fovea/parafovea for saccade targets that were removed prior to completion of the eye movement. We found strong evidence of foveal remapping in retinotopic visual areas, which failed to occur when observers merely attended to the same peripheral target without making eye movements towards it. Significantly, the spatial profile of the remapped response matched the orientation and size of the saccade target, and was appropriately scaled to reflect the retinal extent of the stimulus had it been foveated. We conclude that this remapping of spatially structured information to the fovea may serve as an important mechanism to support our world-centered sense of location across goal-directed eye movements under natural viewing conditions.

Cognitive and Ocular Factors Jointly Determine Pupil Responses under Equiluminance

Changes in pupil diameter can reflect high-level cognitive signals that depend on central neuromodulatory mechanisms. However, brain mechanisms that adjust pupil size are also exquisitely sensitive to changes in luminance and other events that would be considered a nuisance in cognitive experiments recording pupil size. We implemented a simple auditory experiment involving no changes in visual stimulation. Using finite impulse-response fitting we found pupil responses triggered by different types of events. Among these are pupil responses to auditory events and associated surprise: cognitive effects. However, these cognitive responses were overshadowed by pupil responses associated with blinks and eye movements, both inevitable nuisance factors that lead to changes in effective luminance. Of note, these latter pupil responses were not recording artifacts caused by blinks and eye movements, but endogenous pupil responses that occurred in the wake of these events. Furthermore, we identified slow (tonic) changes in pupil size that differentially influenced faster (phasic) pupil responses. Fitting all pupil responses using gamma functions, we provide accurate characterisations of cognitive and non-cognitive response shapes, and quantify each response's dependence on tonic pupil size. These results allow us to create a set of recommendations for pupil size analysis in cognitive neuroscience, which we have implemented in freely available software.

Negligible fronto-parietal BOLD activity accompanying unreportable switches in bistable perception

The human brain's executive systems play a vital role in deciding and selecting among actions. Selection among alternatives also occurs in the perceptual domain, for instance when perception switches between interpretations during perceptual bistability. Whether executive systems also underlie this functionality remains debated, with known fronto-parietal concomitants of perceptual switches being variously interpreted as reflecting the switches' cause, or as reflecting their consequences. We developed a paradigm where the two eyes receive different inputs and perception demonstrably switches between these inputs, yet where switches themselves are so inconspicuous as to become unreportable, minimizing their executive consequences. Fronto-parietal fMRI BOLD responses that accompany perceptual switches were similarly minimized in this paradigm, indicating that these reflect the switches' consequences rather than their cause. We conclude that perceptual switches do not always rely on executive brain areas, and that processes responsible for selection among alternatives may operate outside of the brain's executive systems.

GABA shapes the dynamics of bistable perception

Sometimes, perception fluctuates spontaneously between two distinct interpretations of a constant sensory input. These bistable perceptual phenomena provide a unique window into the neural mechanisms that create the contents of conscious perception. Models of bistable perception posit that mutual inhibition between stimulus-selective neural populations in visual cortex plays a key role in these spontaneous perceptual fluctuations. However, a direct link between neural inhibition and bistable perception has not yet been established experimentally. Here, we link perceptual dynamics in three distinct bistable visual illusions (binocular rivalry, motion-induced blindness, and structure from motion) to measurements of gamma-aminobutyric acid (GABA) concentrations in human visual cortex (as measured with magnetic resonance spectroscopy) and to pharmacological stimulation of the GABAA receptor by means of lorazepam. As predicted by a model of neural interactions underlying bistability, both higher GABA concentrations in visual cortex and lorazepam administration induced slower perceptual dynamics, as reflected in a reduced number of perceptual switches and a lengthening of percept durations. Thus, we show that GABA, the main inhibitory neurotransmitter, shapes the dynamics of bistable perception. These results pave the way for future studies into the competitive neural interactions across the visual cortical hierarchy that elicit conscious perception.

Opposite influence of perceptual memory on initial and prolonged perception of sensory ambiguity

Observers continually make unconscious inferences about the state of the world based on ambiguous sensory information. This process of perceptual decision-making may be optimized by learning from experience. We investigated the influence of previous perceptual experience on the interpretation of ambiguous visual information. Observers were pre-exposed to a perceptually stabilized sequence of an ambiguous structure-from-motion stimulus by means of intermittent presentation. At the subsequent re-appearance of the same ambiguous stimulus perception was initially biased toward the previously stabilized perceptual interpretation. However, prolonged viewing revealed a bias toward the alternative perceptual interpretation. The prevalence of the alternative percept during ongoing viewing was largely due to increased durations of this percept, as there was no reliable decrease in the durations of the pre-exposed percept. Moreover, the duration of the alternative percept was modulated by the specific characteristics of the pre-exposure, whereas the durations of the pre-exposed percept were not. The increase in duration of the alternative percept was larger when the pre-exposure had lasted longer and was larger after ambiguous pre-exposure than after unambiguous pre-exposure. Using a binocular rivalry stimulus we found analogous perceptual biases, while pre-exposure did not affect eye-bias. We conclude that previously perceived interpretations dominate at the onset of ambiguous sensory information, whereas alternative interpretations dominate prolonged viewing. Thus, at first instance ambiguous information seems to be judged using familiar percepts, while re-evaluation later on allows for alternative interpretations.