Inference and computation with population codes

Modelling neural probabilistic computation using vector symbolic architectures

Distributed vector representations are a key bridging point between connectionist and symbolic representations in cognition. It is unclear how uncertainty should be modelled in systems using such representations. In this paper we discuss how bundles of symbols in certain Vector Symbolic Architectures (VSAs) can be understood as defining an object that has a relationship to a probability distribution, and how statements in VSAs can be understood as being analogous to probabilistic statements. The aim of this paper is to show how (spiking) neural implementations of VSAs can be used to implement probabilistic operations that are useful in building cognitive models. We show how similarity operators between continuous values represented as Spatial Semantic Pointers (SSPs), an example of a technique known as fractional binding, induces a quasi-kernel function that can be used in density estimation. Further, we sketch novel designs for networks that compute entropy and mutual information of VSA-represented distributions and demonstrate their performance when implemented as networks of spiking neurons. We also discuss the relationship between our technique and quantum probability, another technique proposed for modelling uncertainty in cognition. While we restrict ourselves to operators proposed for Holographic Reduced Representations, and for representing real-valued data. We suggest that the methods presented in this paper should translate to any VSA where the dot product between fractionally bound symbols induces a valid kernel.

Electrical Signaling Beyond Neurons

Neural action potentials (APs) are difficult to interpret as signal encoders and/or computational primitives. Their relationships with stimuli and behaviors are obscured by the staggering complexity of nervous systems themselves. We can reduce this complexity by observing that "simpler" neuron-less organisms also transduce stimuli into transient electrical pulses that affect their behaviors. Without a complicated nervous system, APs are often easier to understand as signal/response mechanisms. We review examples of nonneural stimulus transductions in domains of life largely neglected by theoretical neuroscience: bacteria, protozoans, plants, fungi, and neuron-less animals. We report properties of those electrical signals-for example, amplitudes, durations, ionic bases, refractory periods, and particularly their ecological purposes. We compare those properties with those of neurons to infer the tasks and selection pressures that neurons satisfy. Throughout the tree of life, nonneural stimulus transductions time behavioral responses to environmental changes. Nonneural organisms represent the presence or absence of a stimulus with the presence or absence of an electrical signal. Their transductions usually exhibit high sensitivity and specificity to a stimulus, but are often slow compared to neurons. Neurons appear to be sacrificing the specificity of their stimulus transductions for sensitivity and speed. We interpret cellular stimulus transductions as a cell's assertion that it detected something important at that moment in time. In particular, we consider neural APs as fast but noisy detection assertions. We infer that a principal goal of nervous systems is to detect extremely weak signals from noisy sensory spikes under enormous time pressure. We discuss neural computation proposals that address this goal by casting neurons as devices that implement online, analog, probabilistic computations with their membrane potentials. Those proposals imply a measurable relationship between afferent neural spiking statistics and efferent neural membrane electrophysiology.

How Shape Perception Works, in Two Dimensions and Three Dimensions

The ventral visual pathway transforms retinal images into neural representations that support object understanding, including exquisite appreciation of precise 2D pattern shape and 3D volumetric shape. We articulate a framework for understanding the goals of this transformation and how they are achieved by neural coding at successive ventral pathway stages. The critical goals are (a) radical compression to make shape information communicable across axonal bundles and storable in memory, (b) explicit coding to make shape information easily readable by the rest of the brain and thus accessible for cognition and behavioral control, and (c) representational stability to maintain consistent perception across highly variable viewing conditions. We describe how each transformational step in ventral pathway vision serves one or more of these goals. This three-goal framework unifies discoveries about ventral shape processing into a neural explanation for our remarkable experience of shape as a vivid, richly detailed aspect of the natural world.

Neural mechanisms of resource allocation in working memory

To mitigate capacity limits of working memory, people allocate resources according to an item's relevance. However, the neural mechanisms supporting such a critical operation remain unknown. Here, we developed computational neuroimaging methods to decode and demix neural responses associated with multiple items in working memory with different priorities. In striate and extrastriate cortex, the gain of neural responses tracked the priority of memoranda. Higher-priority memoranda were decoded with smaller error and lower uncertainty. Moreover, these neural differences predicted behavioral differences in memory prioritization. Remarkably, trialwise variability in the magnitude of delay activity in frontal cortex predicted differences in decoded precision between low and high-priority items in visual cortex. These results suggest a model in which feedback signals broadcast from frontal cortex sculpt the gain of memory representations in visual cortex according to behavioral relevance, thus, identifying a neural mechanism for resource allocation.

Perceptual similarity judgments do not predict the distribution of errors in working memory

Population coding models provide a quantitative account of visual working memory (VWM) retrieval errors with a plausible link to the response characteristics of sensory neurons. Recent work has provided an important new perspective linking population coding to variables of signal detection, including d-prime, and put forward a new hypothesis: that the distribution of recall errors on, for example, a color wheel, is a consequence of the psychological similarity between points in that stimulus space, such that the exponential-like psychophysical distance scaling function can fulfil the role of population tuning and obviate the need to fit a tuning width parameter to recall data. Using four different visual feature spaces, we measured psychophysical similarity and memory errors in the same participants. Our results revealed strong evidence for a common source of variability affecting similarity judgments and recall estimates but did not support any consistent relationship between psychophysical similarity functions and VWM errors. At the group level, the responsiveness functions obtained from the psychophysical similarity task diverged strongly from those that provided the best fit to working memory errors. At the individual level, we found convincing evidence against an association between observed and best-fitting similarity functions. Finally, our results show that the newly proposed exponential-like responsiveness function has in general no advantage over the canonical von Mises (circular normal) function assumed by previous population coding models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Learning what matters: Synaptic plasticity with invariance to second-order input correlations

Cortical populations of neurons develop sparse representations adapted to the statistics of the environment. To learn efficient population codes, synaptic plasticity mechanisms must differentiate relevant latent features from spurious input correlations, which are omnipresent in cortical networks. Here, we develop a theory for sparse coding and synaptic plasticity that is invariant to second-order correlations in the input. Going beyond classical Hebbian learning, our learning objective explains the functional form of observed excitatory plasticity mechanisms, showing how Hebbian long-term depression (LTD) cancels the sensitivity to second-order correlations so that receptive fields become aligned with features hidden in higher-order statistics. Invariance to second-order correlations enhances the versatility of biologically realistic learning models, supporting optimal decoding from noisy inputs and sparse population coding from spatially correlated stimuli. In a spiking model with triplet spike-timing-dependent plasticity (STDP), we show that individual neurons can learn localized oriented receptive fields, circumventing the need for input preprocessing, such as whitening, or population-level lateral inhibition. The theory advances our understanding of local unsupervised learning in cortical circuits, offers new interpretations of the Bienenstock-Cooper-Munro and triplet STDP models, and assigns a specific functional role to synaptic LTD mechanisms in pyramidal neurons.

Neural encoding of multiple motion speeds in visual cortical area MT

Segmenting objects from each other and their background is critical for vision. The speed at which objects move provides a salient cue for segmentation. However, how the visual system represents and differentiates multiple speeds is largely unknown. Here we investigated the neural encoding of multiple speeds of overlapping stimuli in the primate visual cortex. We first characterized the perceptual capacity of human and monkey subjects to segment spatially overlapping stimuli moving at different speeds. We then determined how neurons in the motion-sensitive, middle-temporal (MT) cortex of macaque monkeys encode multiple speeds. We made a novel finding that the responses of MT neurons to two speeds of overlapping stimuli showed a robust bias toward the faster speed component when both speeds were slow (≤ 20°/s). The faster-speed bias occurred even when a neuron had a slow preferred speed and responded more strongly to the slower component than the faster component when presented alone. The faster-speed bias emerged very early in neuronal response and was robust over time and to manipulations of motion direction and attention. As the stimulus speed increased, the faster-speed bias changed to response averaging. Our finding can be explained by a modified divisive normalization model, in which the weights for the speed components are proportional to the responses of a population of neurons elicited by the individual speeds. Our results suggest that the neuron population, referred to as the weighting pool, includes neurons that have a broad range of speed preferences. As a result, the response weights for the speed components are determined by the stimulus speeds and invariant to the speed preferences of individual neurons. Our findings help to define the neural encoding rule of multiple stimuli and provide new insight into the underlying neural mechanisms. The faster-speed bias would benefit behavioral tasks such as figure-ground segregation if figural objects tend to move faster than the background in the natural environment.

Sampling-based Bayesian inference in recurrent circuits of stochastic spiking neurons

Two facts about cortex are widely accepted: neuronal responses show large spiking variability with near Poisson statistics and cortical circuits feature abundant recurrent connections between neurons. How these spiking and circuit properties combine to support sensory representation and information processing is not well understood. We build a theoretical framework showing that these two ubiquitous features of cortex combine to produce optimal sampling-based Bayesian inference. Recurrent connections store an internal model of the external world, and Poissonian variability of spike responses drives flexible sampling from the posterior stimulus distributions obtained by combining feedforward and recurrent neuronal inputs. We illustrate how this framework for sampling-based inference can be used by cortex to represent latent multivariate stimuli organized either hierarchically or in parallel. A neural signature of such network sampling are internally generated differential correlations whose amplitude is determined by the prior stored in the circuit, which provides an experimentally testable prediction for our framework.

Studying the neural representations of uncertainty

The study of the brain's representations of uncertainty is a central topic in neuroscience. Unlike most quantities of which the neural representation is studied, uncertainty is a property of an observer's beliefs about the world, which poses specific methodological challenges. We analyze how the literature on the neural representations of uncertainty addresses those challenges and distinguish between 'code-driven' and 'correlational' approaches. Code-driven approaches make assumptions about the neural code for representing world states and the associated uncertainty. By contrast, correlational approaches search for relationships between uncertainty and neural activity without constraints on the neural representation of the world state that this uncertainty accompanies. To compare these two approaches, we apply several criteria for neural representations: sensitivity, specificity, invariance and functionality. Our analysis reveals that the two approaches lead to different but complementary findings, shaping new research questions and guiding future experiments.

Neural tuning instantiates prior expectations in the human visual system

Perception is often modelled as a process of active inference, whereby prior expectations are combined with noisy sensory measurements to estimate the structure of the world. This mathematical framework has proven critical to understanding perception, cognition, motor control, and social interaction. While theoretical work has shown how priors can be computed from environmental statistics, their neural instantiation could be realised through multiple competing encoding schemes. Using a data-driven approach, here we extract the brain's representation of visual orientation and compare this with simulations from different sensory coding schemes. We found that the tuning of the human visual system is highly conditional on stimulus-specific variations in a way that is not predicted by previous proposals. We further show that the adopted encoding scheme effectively embeds an environmental prior for natural image statistics within the sensory measurement, providing the functional architecture necessary for optimal inference in the earliest stages of cortical processing.

Flexible integration of continuous sensory evidence in perceptual estimation tasks

Temporal accumulation of evidence is crucial for making accurate judgments based on noisy or ambiguous sensory input. The integration process leading to categorical decisions is thought to rely on competition between neural populations, each encoding a discrete categorical choice. How recurrent neural circuits integrate evidence for continuous perceptual judgments is unknown. Here, we show that a continuous bump attractor network can integrate a circular feature, such as stimulus direction, nearly optimally. As required by optimal integration, the population activity of the network unfolds on a two-dimensional manifold, in which the position of the network's activity bump tracks the stimulus average, and, simultaneously, the bump amplitude tracks stimulus uncertainty. Moreover, the temporal weighting of sensory evidence by the network depends on the relative strength of the stimulus compared to the internally generated bump dynamics, yielding either early (primacy), uniform, or late (recency) weighting. The model can flexibly switch between these regimes by changing a single control parameter, the global excitatory drive. We show that this mechanism can quantitatively explain individual temporal weighting profiles of human observers, and we validate the model prediction that temporal weighting impacts reaction times. Our findings point to continuous attractor dynamics as a plausible neural mechanism underlying stimulus integration in perceptual estimation tasks.

Bayesian mechanics for stationary processes

This paper develops a Bayesian mechanics for adaptive systems. Firstly, we model the interface between a system and its environment with a Markov blanket. This affords conditions under which states internal to the blanket encode information about external states. Second, we introduce dynamics and represent adaptive systems as Markov blankets at steady state. This allows us to identify a wide class of systems whose internal states appear to infer external states, consistent with variational inference in Bayesian statistics and theoretical neuroscience. Finally, we partition the blanket into sensory and active states. It follows that active states can be seen as performing active inference and well-known forms of stochastic control (such as PID control), which are prominent formulations of adaptive behaviour in theoretical biology and engineering.

Spatialization of Time in the Entorhinal-Hippocampal System

The functional role of the entorhinal-hippocampal system has been a long withstanding mystery. One key theory that has become most popular is that the entorhinal-hippocampal system represents space to facilitate navigation in one's surroundings. In this Perspective article, I introduce a novel idea that undermines the inherent uniqueness of spatial information in favor of time driving entorhinal-hippocampal activity. Specifically, by spatializing events that occur in succession (i.e., across time), the entorhinal-hippocampal system is critical for all types of cognitive representations. I back up this argument with empirical evidence that hints at a role for the entorhinal-hippocampal system in non-spatial representation, and computational models of the logarithmic compression of time in the brain.

A Double-Layer Multi-Resolution Classification Model for Decoding Spatiotemporal Patterns of Spikes With Small Sample Size

We build a double-layer, multiple temporal-resolution classification model for decoding single-trial spatiotemporal patterns of spikes. The model takes spiking activities as input signals and binary behavioral or cognitive variables as output signals and represents the input-output mapping with a double-layer ensemble classifier. In the first layer, to solve the underdetermined problem caused by the small sample size and the very high dimensionality of input signals, B-spline functional expansion and L1-regularized logistic classifiers are used to reduce dimensionality and yield sparse model estimations. A wide range of temporal resolutions of neural features is included by using a large number of classifiers with different numbers of B-spline knots. Each classifier serves as a base learner to classify spatiotemporal patterns into the probability of the output label with a single temporal resolution. A bootstrap aggregating strategy is used to reduce the estimation variances of these classifiers. In the second layer, another L1-regularized logistic classifier takes outputs of first-layer classifiers as inputs to generate the final output predictions. This classifier serves as a meta-learner that fuses multiple temporal resolutions to classify spatiotemporal patterns of spikes into binary output labels. We test this decoding model with both synthetic and experimental data recorded from rats and human subjects performing memory-dependent behavioral tasks. Results show that this method can effectively avoid overfitting and yield accurate prediction of output labels with small sample size. The double-layer, multi-resolution classifier consistently outperforms the best single-layer, single-resolution classifier by extracting and utilizing multi-resolution spatiotemporal features of spike patterns in the classification.

Neural implementations of Bayesian inference

Bayesian inference has emerged as a general framework that captures how organisms make decisions under uncertainty. Recent experimental findings reveal disparate mechanisms for how the brain generates behaviors predicted by normative Bayesian theories. Here, we identify two broad classes of neural implementations for Bayesian inference: a modular class, where each probabilistic component of Bayesian computation is independently encoded and a transform class, where uncertain measurements are converted to Bayesian estimates through latent processes. Many recent experimental neuroscience findings studying probabilistic inference broadly fall into these classes. We identify potential avenues for synthesis across these two classes and the disparities that, at present, cannot be reconciled. We conclude that to distinguish among implementation hypotheses for Bayesian inference, we require greater engagement among theoretical and experimental neuroscientists in an effort that spans different scales of analysis, circuits, tasks, and species.

Neuronal variability reflects probabilistic inference tuned to natural image statistics

Neuronal activity in sensory cortex fluctuates over time and across repetitions of the same input. This variability is often considered detrimental to neural coding. The theory of neural sampling proposes instead that variability encodes the uncertainty of perceptual inferences. In primary visual cortex (V1), modulation of variability by sensory and non-sensory factors supports this view. However, it is unknown whether V1 variability reflects the statistical structure of visual inputs, as would be required for inferences correctly tuned to the statistics of the natural environment. Here we combine analysis of image statistics and recordings in macaque V1 to show that probabilistic inference tuned to natural image statistics explains the widely observed dependence between spike count variance and mean, and the modulation of V1 activity and variability by spatial context in images. Our results show that the properties of a basic aspect of cortical responses-their variability-can be explained by a probabilistic representation tuned to naturalistic inputs.

Multisensory enhancement of overt behavior requires multisensory experience

The superior colliculus (SC) is richly endowed with neurons that integrate cues from different senses to enhance their physiological responses and the overt behaviors they mediate. However, in the absence of experience with cross-modal combinations (e.g., visual-auditory), they fail to develop this characteristic multisensory capability: Their multisensory responses are no greater than their most effective unisensory responses. Presumably, this impairment in neural development would be reflected as corresponding impairments in SC-mediated behavioral capabilities such as detection and localization performance. Here, we tested that assumption directly in cats raised to adulthood in darkness. They, along with a normally reared cohort, were trained to approach brief visual or auditory stimuli. The animals were then tested with these stimuli individually and in combination under ambient light conditions consistent with their rearing conditions and home environment as well as under the opposite lighting condition. As expected, normally reared animals detected and localized the cross-modal combinations significantly better than their individual component stimuli. However, dark-reared animals showed significant defects in multisensory detection and localization performance. The results indicate that a physiological impairment in single multisensory SC neurons is predictive of an impairment in overt multisensory behaviors.

Computational models link cellular mechanisms of neuromodulation to large-scale neural dynamics

Decades of neurobiological research have disclosed the diverse manners in which the response properties of neurons are dynamically modulated to support adaptive cognitive functions. This neuromodulation is achieved through alterations in the biophysical properties of the neuron. However, changes in cognitive function do not arise directly from the modulation of individual neurons, but are mediated by population dynamics in mesoscopic neural ensembles. Understanding this multiscale mapping is an important but nontrivial issue. Here, we bridge these different levels of description by showing how computational models parametrically map classic neuromodulatory processes onto systems-level models of neural activity. The ensuing critical balance of systems-level activity supports perception and action, although our knowledge of this mapping remains incomplete. In this way, quantitative models that link microscale neuronal neuromodulation to systems-level brain function highlight gaps in knowledge and suggest new directions for integrating theoretical and experimental work.

Correlated structure of neuronal firing in macaque visual cortex limits information for binocular depth discrimination

Variability in cortical neural activity potentially limits sensory discriminations. Theoretical work shows that information required to discriminate two similar stimuli is limited by the correlation structure of cortical variability. We investigated these information-limiting correlations by recording simultaneously from visual cortical areas primary visual cortex (V1) and extrastriate area V4 in macaque monkeys performing a binocular, stereo depth discrimination task. Within both areas, noise correlations on a rapid temporal scale (20–30 ms) were stronger for neuron pairs with similar selectivity for binocular depth, meaning that these correlations potentially limit information for making the discrimination. Between-area correlations (V1 to V4) were different, being weaker for neuron pairs with similar tuning and having a slower temporal scale (100+ ms). Fluctuations in these information-limiting correlations just prior to the detection event were associated with changes in behavioral accuracy. Although these correlations limit the recovery of information about sensory targets, their impact may be curtailed by integrative processing of signals across multiple brain areas.

NEW & NOTEWORTHY Correlated noise reduces the stimulus information in visual cortical neurons during experimental performance of binocular depth discriminations. The temporal scale of these correlations is important. Rapid (20–30 ms) correlations reduce information within and between areas V1 and V4, whereas slow (>100 ms) correlations between areas do not. Separate cortical areas appear to act together to maintain signal fidelity. Rapid correlations reduce the neuronal signal difference between stimuli and adversely affect perceptual discrimination.

Circular inference in bistable perception

When facing ambiguous images, the brain switches between mutually exclusive interpretations, a phenomenon known as bistable perception. Despite years of research, a consensus on whether bistability is driven primarily by bottom-up or top-down mechanisms has not been achieved. Here, we adopted a Bayesian approach to reconcile these two theories. Fifty-five healthy participants were exposed to an adaptation of the Necker cube paradigm, in which we manipulated sensory evidence and prior knowledge. Manipulations of both sensory evidence and priors significantly affected the way participants perceived the Necker cube. However, we observed an interaction between the effect of the cue and the effect of the instructions, a finding that is incompatible with Bayes-optimal integration. In contrast, the data were well predicted by a circular inference model. In this model, ambiguous sensory evidence is systematically biased in the direction of current expectations, ultimately resulting in a bistable percept.

Confidence in predicted position error explains saccadic decisions during pursuit

A fundamental problem in motor control is the coordination of complementary movement types to achieve a common goal. As a common example, humans view moving objects through coordinated pursuit and saccadic eye movements. Pursuit is initiated and continuously controlled by retinal image velocity. During pursuit, eye position may lag behind the target. This can be compensated by the discrete execution of a catch-up saccade. The decision to trigger a saccade is influenced by both position and velocity errors, and the timing of saccades can be highly variable. The observed distributions of saccade frequency and trigger time remain poorly understood, and this decision process remains imprecisely quantified. Here, we propose a predictive, probabilistic model explaining the decision to trigger saccades during pursuit to foveate moving targets. In this model, expected position error and its associated uncertainty are predicted through Bayesian inference across noisy, delayed sensory observations (Kalman filtering). This probabilistic prediction is used to estimate the confidence that a saccade is needed (quantified through log-probability ratio), triggering a saccade upon accumulating to a fixed threshold. The model qualitatively explains behavioral observations on the frequency and trigger time distributions of saccades during pursuit over a range of target motion trajectories. Furthermore, this model makes novel predictions that saccade decisions are highly sensitive to uncertainty for small predicted position errors, but this influence diminishes as the magnitude of predicted position error increases. We suggest that this predictive, confidence-based decision-making strategy represents a fundamental principle for the probabilistic neural control of coordinated movements.NEW & NOTEWORTHY This is the first stochastic dynamical systems model of pursuit-saccade coordination accounting for noise and delays in the sensorimotor system. The model uses Bayesian inference to predictively estimate visual motion, triggering saccades when confidence in predicted position error accumulates to a threshold. This model explains saccade frequency and trigger time distributions across target trajectories and makes novel predictions about the influence of sensory uncertainty in saccade decisions during pursuit.

Dual n-back training improves functional connectivity of the right inferior frontal gyrus at rest

Several studies have shown that the benefits of working memory (WM) training can be attributed to functional and structural neural changes in the underlying neural substrate. In the current study, we investigated whether the functional connectivity of the brain at rest in the default mode network (DMN) changes with WM training. We varied the complexity of the training intervention so, that half of the participants attended dual n-back training whereas the other half attended single n-back training. This way we could assess the effects of different training task parameters on possible connectivity changes. After 16 training sessions, the dual n-back training group showed improved performance accompanied by increased functional connectivity of the ventral DMN in the right inferior frontal gyrus, which correlated with improvements in WM. We also observed decreased functional connectivity in the left superior parietal cortex in this group. The single n-back training group did not show significant training-related changes. These results show that a demanding short-term WM training intervention can alter the default state of the brain.

The suboptimality of perceptual decision making with multiple alternatives

It is becoming widely appreciated that human perceptual decision making is suboptimal but the nature and origins of this suboptimality remain poorly understood. Most past research has employed tasks with two stimulus categories, but such designs cannot fully capture the limitations inherent in naturalistic perceptual decisions where choices are rarely between only two alternatives. We conduct four experiments with tasks involving multiple alternatives and use computational modeling to determine the decision-level representation on which the perceptual decisions are based. The results from all four experiments point to the existence of robust suboptimality such that most of the information in the sensory representation is lost during the transformation to a decision-level representation. These results reveal severe limits in the quality of decision-level representations for multiple alternatives and have strong implications about perceptual decision making in naturalistic settings.

Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex

Every decision we make is accompanied by a sense of confidence about its likely outcome. This sense informs subsequent behavior, such as investing more-whether time, effort, or money-when reward is more certain. A neural representation of confidence should originate from a statistical computation and predict confidence-guided behavior. An additional requirement for confidence representations to support metacognition is abstraction: they should emerge irrespective of the source of information and inform multiple confidence-guided behaviors. It is unknown whether neural confidence signals meet these criteria. Here, we show that single orbitofrontal cortex neurons in rats encode statistical decision confidence irrespective of the sensory modality, olfactory or auditory, used to make a choice. The activity of these neurons also predicts two confidence-guided behaviors: trial-by-trial time investment and cross-trial choice strategy updating. Orbitofrontal cortex thus represents decision confidence consistent with a metacognitive process that is useful for mediating confidence-guided economic decisions.

Improved fidelity of orientation perception: a learning effect dissociable from enhanced discriminability

Visual perception can be influenced by stimulus context, selective attention, and prior experience. Many previous studies have shown complex interactions among these influencing factors, but it remains unclear whether context-induced illusions could be reduced by perceptual training and whether such a change in perceptual fidelity is linked to improved perceptual discriminability. To address this question, we introduced a context-induced tilt illusion into an orientation discrimination training paradigm. This resulted in parallel and long-term improvements in the discriminability and fidelity of orientation perception. The improved discriminability was specific to the task-relevant target stimulus but nonspecific to the task-irrelevant context. By contrast, the improved perceptual fidelity was specific to the task-irrelevant contextual stimulus that induced the illusion, but not specific to the task-relevant target stimulus or task performed on one of its features. These results indicate two dissociable learning effects associated with the same training procedure. Such a dissociation was further supported by the observation that the sizes of the two learning effects were uncorrelated across the subjects. Our findings suggest two parallel learning processes: a task-dependent process giving rise to enhanced discriminability for the task-relevant stimulus attribute, and a context-dependent process leading to improved perceptual fidelity for the attended stimuli.

Bayesian Decision Models: A Primer

To understand decision-making behavior in simple, controlled environments, Bayesian models are often useful. First, optimal behavior is always Bayesian. Second, even when behavior deviates from optimality, the Bayesian approach offers candidate models to account for suboptimalities. Third, a realist interpretation of Bayesian models opens the door to studying the neural representation of uncertainty. In this tutorial, we review the principles of Bayesian models of decision making and then focus on five case studies with exercises. We conclude with reflections and future directions.

Theory of neural coding predicts an upper bound on estimates of memory variability

Observers reproducing elementary visual features from memory after a short delay produce errors consistent with the encoding-decoding properties of neural populations. While inspired by electrophysiological observations of sensory neurons in cortex, the population coding account of these errors is based on a mathematical idealization of neural response functions that abstracts away most of the heterogeneity and complexity of real neuronal populations. Here we examine a more physiologically grounded model based on the tuning of a large set of neurons recorded in macaque V1 and show that key predictions of the idealized model are preserved. Both models predict long-tailed distributions of error when memory resources are taxed, as observed empirically in behavioral experiments and commonly approximated with a mixture of normal and uniform error components. Specifically, for an idealized homogeneous neural population, the width of the fitted normal distribution cannot exceed the average tuning width of the component neurons, and this also holds to a good approximation for more biologically realistic populations. Examining eight published studies of orientation recall, we find a consistent pattern of results suggestive of a median tuning width of approximately 20°, which compares well with neurophysiological observations. The finding that estimates of variability obtained by the normal-plus-uniform mixture method are bounded from above leads us to reevaluate previous studies that interpreted a saturation in width of the normal component as evidence for fundamental limits on the precision of perception, working memory, and long-term memory.

Spatial resolution of local field potential signals in macaque V4

OBJECTIVE

An important challenge for the development of cortical visual prostheses is to generate spatially localized percepts of light, using artificial stimulation. Such percepts are called phosphenes, and the goal of prosthetic applications is to generate a pattern of phosphenes that matches the structure of the retinal image. A preliminary step in this process is to understand how the spatial positions of phosphene-like visual stimuli are encoded in the distributed activity of cortical neurons. The spatial resolution with which the distributed responses discriminate positions puts a limit on the capability of visual prosthesis devices to induce phosphenes at multiple positions. While most previous prosthetic devices have targeted the primary visual cortex, the extrastriate cortex has the advantage of covering a large part of the visual field with a smaller amount of cortical tissue, providing the possibility of a more compact implant. Here, we studied how well ensembles of Local Field Potentials (LFPs) and Multiunit activity (MUA) responses from extrastriate cortical visual area V4 of a behaving macaque monkey can discriminate between two-dimensional spatial positions.

APPROACH

We used support vector machines (SVM) to determine the capabilities of LFPs and MUA to discriminate responses to phosphene-like stimuli (probes) at different spatial separations. We proposed a selection strategy based on the combined responses of multiple electrodes and used the linear learning weights to find the minimum number of electrodes for fine and coarse discriminations. We also measured the contribution of correlated trial-to-trial variability in the responses to the discrimination performance for MUA and LFP.

MAIN RESULTS

We found that despite the large receptive field sizes in V4, the combined responses from multiple sites, whether MUA or LFP, are capable of fine and coarse discrimination of positions. Our electrode selection procedure significantly increased discrimination performance while reducing the required number of electrodes. Analysis of noise correlations in MUA and LFP responses showed that noise correlations in LFPs carry more information about spatial positions.

SIGNIFICANCE

This study determined the coding strategy for fine discrimination, suggesting that spatial positions could be well localized with patterned stimulation in extrastriate area V4. It also provides a novel approach to build a compact prosthesis with relatively few electrodes, which has the potential advantage of reducing tissue damage in real applications.

Coding with transient trajectories in recurrent neural networks

Following a stimulus, the neural response typically strongly varies in time and across neurons before settling to a steady-state. While classical population coding theory disregards the temporal dimension, recent works have argued that trajectories of transient activity can be particularly informative about stimulus identity and may form the basis of computations through dynamics. Yet the dynamical mechanisms needed to generate a population code based on transient trajectories have not been fully elucidated. Here we examine transient coding in a broad class of high-dimensional linear networks of recurrently connected units. We start by reviewing a well-known result that leads to a distinction between two classes of networks: networks in which all inputs lead to weak, decaying transients, and networks in which specific inputs elicit amplified transient responses and are mapped onto output states during the dynamics. Theses two classes are simply distinguished based on the spectrum of the symmetric part of the connectivity matrix. For the second class of networks, which is a sub-class of non-normal networks, we provide a procedure to identify transiently amplified inputs and the corresponding readouts. We first apply these results to standard randomly-connected and two-population networks. We then build minimal, low-rank networks that robustly implement trajectories mapping a specific input onto a specific orthogonal output state. Finally, we demonstrate that the capacity of the obtained networks increases proportionally with their size.

A neural basis of probabilistic computation in visual cortex

Bayesian models of behavior suggest that organisms represent uncertainty associated with sensory variables. However, the neural code of uncertainty remains elusive. A central hypothesis is that uncertainty is encoded in the population activity of cortical neurons in the form of likelihood functions. We tested this hypothesis by simultaneously recording population activity from primate visual cortex during a visual categorization task in which trial-to-trial uncertainty about stimulus orientation was relevant for the decision. We decoded the likelihood function from the trial-to-trial population activity and found that it predicted decisions better than a point estimate of orientation. This remained true when we conditioned on the true orientation, suggesting that internal fluctuations in neural activity drive behaviorally meaningful variations in the likelihood function. Our results establish the role of population-encoded likelihood functions in mediating behavior and provide a neural underpinning for Bayesian models of perception.

Extrinsic and intrinsic representations

We extend the discussion in the target article about distinctions between extrinsic coding (external references to known things, as required by information theory) and the alternative we and the target article both favor, intrinsic coding (internal relationships within sensory and motor signals). Central to our thinking about intrinsic coding is population coding and the concept of high-dimensional neural response spaces.

Plasticity of the neural coding metaphor: An unnoticed rhetoric in scientific discourse

The convincing argument that Brette makes for the neural coding metaphor as imposing one view of brain behavior can be further explained through discourse analysis. Instead of a unified view, we argue, the coding metaphor's plasticity, versatility, and robustness throughout time explain its success and conventionalization to the point that its rhetoric became overlooked.

The profile of attentional modulation to visual features

Although it is well established that feature-based attention (FBA) can enhance an attended feature, how it modulates unattended features remains less clear. Previous studies have generally supported either a graded profile as predicted by the feature-similarity gain model or a nonmonotonic profile predicted by the surround suppression model. To reconcile these different views, we systematically measured the attentional profile in three basic feature dimensions-orientation, motion direction, and spatial frequency. In three experiments, we instructed participants to detect a coherent feature signal against noise under attentional or neutral condition. Our results support a nonmonotonic hybrid model of attentional modulation consisting of feature-similarity gain and surround suppression for orientation and motion direction. For spatial frequency, we also found a similar nonmonotonic profile for higher frequencies than the attended frequency, but a lack of attentional modulation for lower frequencies than the attended frequency. The current findings can reconcile the discrepancies in the literature and suggest the hybrid model as a new framework for attentional modulation in feature space. In addition, a computational model incorporating known properties of spatial frequency channels and attentional modulations at the neural level reproduced the asymmetric attentional modulation, thus revealing a connection between surround suppression and the basic neural architecture of an early visual system.

Modeling stimulus-dependent variability improves decoding of population neural responses

OBJECTIVE

Neural responses to repeated presentations of an identical stimulus often show substantial trial-to-trial variability. How the mean firing rate varies in response to different stimuli or during different movements (tuning curves) has been extensively modeled in a wide variety of neural systems. However, the variability of neural responses can also have clear tuning independent of the tuning in the mean firing rate. This suggests that the variability could contain information regarding the stimulus/movement beyond what is encoded in the mean firing rate. Here we demonstrate how taking variability into account can improve neural decoding.

APPROACH

In a typical neural coding model spike counts are assumed to be Poisson with the mean response depending on an external variable, such as a stimulus or movement. Bayesian decoding methods then use the probabilities under these Poisson tuning models (the likelihood) to estimate the probability of each stimulus given the spikes on a given trial (the posterior). However, under the Poisson model, spike count variability is always exactly equal to the mean (Fano factor  =  1). Here we use two alternative models-the Conway-Maxwell-Poisson (CMP) model and negative binomial (NB) model-to more flexibly characterize how neural variability depends on external stimuli. These models both contain the Poisson distribution as a special case but have an additional parameter that allows the variance to be greater than the mean (Fano factor  >  1) or, for the CMP model, less than the mean (Fano factor  <  1).

MAIN RESULTS

We find that neural responses in primary motor (M1), visual (V1), and auditory (A1) cortices have diverse tuning in both their mean firing rates and response variability. Across cortical areas, we find that Bayesian decoders using the CMP or NB models improve stimulus/movement estimation accuracy by 4%-12% compared to the Poisson model.

SIGNIFICANCE

Moreover, the uncertainty of the non-Poisson decoders more accurately reflects the magnitude of estimation errors. In addition to tuning curves that reflect average neural responses, stimulus-dependent response variability may be an important aspect of the neural code. Modeling this structure could, potentially, lead to improvements in brain machine interfaces.

A common probabilistic framework for perceptual and statistical learning

System-level learning of sensory information is traditionally divided into two domains: perceptual learning that focuses on acquiring knowledge suitable for fine discrimination between similar sensory inputs, and statistical learning that explores the mechanisms that develop complex representations of unfamiliar sensory experiences. The two domains have been typically treated in complete separation both in terms of the underlying computational mechanisms and the brain areas and processes implementing those computations. However, a number of recent findings in both domains call in question this strict separation. We interpret classical and more recent results in the general framework of probabilistic computation, provide a unifying view of how various aspects of the two domains are interlinked, and suggest how the probabilistic approach can also alleviate the problem of dealing with widely different types of neural correlates of learning. Finally, we outline several directions along which our proposed approach fosters new types of experiments that can promote investigations of natural learning in humans and other species.

The resonant brain: How attentive conscious seeing regulates action sequences that interact with attentive cognitive learning, recognition, and prediction

This article describes mechanistic links that exist in advanced brains between processes that regulate conscious attention, seeing, and knowing, and those that regulate looking and reaching. These mechanistic links arise from basic properties of brain design principles such as complementary computing, hierarchical resolution of uncertainty, and adaptive resonance. These principles require conscious states to mark perceptual and cognitive representations that are complete, context sensitive, and stable enough to control effective actions. Surface-shroud resonances support conscious seeing and action, whereas feature-category resonances support learning, recognition, and prediction of invariant object categories. Feedback interactions between cortical areas such as peristriate visual cortical areas V2, V3A, and V4, and the lateral intraparietal area (LIP) and inferior parietal sulcus (IPS) of the posterior parietal cortex (PPC) control sequences of saccadic eye movements that foveate salient features of attended objects and thereby drive invariant object category learning. Learned categories can, in turn, prime the objects and features that are attended and searched. These interactions coordinate processes of spatial and object attention, figure-ground separation, predictive remapping, invariant object category learning, and visual search. They create a foundation for learning to control motor-equivalent arm movement sequences, and for storing these sequences in cognitive working memories that can trigger the learning of cognitive plans with which to read out skilled movement sequences. Cognitive-emotional interactions that are regulated by reinforcement learning can then help to select the plans that control actions most likely to acquire valued goal objects in different situations. Many interdisciplinary psychological and neurobiological data about conscious and unconscious behaviors in normal individuals and clinical patients have been explained in terms of these concepts and mechanisms.

Robust computation with rhythmic spike patterns

This work makes 2 contributions. First, we present a neural network model of associative memory that stores and retrieves sparse patterns of complex variables. This network can store analog information as fixed-point attractors in the complex domain; it is governed by an energy function and has increased memory capacity compared to early models. Second, we translate complex attractor networks into spiking networks, where the timing of the spike indicates the phase of a complex number. We show that complex fixed points correspond to stable periodic spike patterns. It is demonstrated that such networks can be constructed with resonate-and-fire or integrate-and-fire neurons with biologically plausible mechanisms and be used for robust computations, such as image retrieval.

Information coding by precise timing of spikes can be faster and more energy efficient than traditional rate coding. However, spike-timing codes are often brittle, which has limited their use in theoretical neuroscience and computing applications. Here, we propose a type of attractor neural network in complex state space and show how it can be leveraged to construct spiking neural networks with robust computational properties through a phase-to-timing mapping. Building on Hebbian neural associative memories, like Hopfield networks, we first propose threshold phasor associative memory (TPAM) networks. Complex phasor patterns whose components can assume continuous-valued phase angles and binary magnitudes can be stored and retrieved as stable fixed points in the network dynamics. TPAM achieves high memory capacity when storing sparse phasor patterns, and we derive the energy function that governs its fixed-point attractor dynamics. Second, we construct 2 spiking neural networks to approximate the complex algebraic computations in TPAM, a reductionist model with resonate-and-fire neurons and a biologically plausible network of integrate-and-fire neurons with synaptic delays and recurrently connected inhibitory interneurons. The fixed points of TPAM correspond to stable periodic states of precisely timed spiking activity that are robust to perturbation. The link established between rhythmic firing patterns and complex attractor dynamics has implications for the interpretation of spike patterns seen in neuroscience and can serve as a framework for computation in emerging neuromorphic devices.

Beyond the "Conceptual Nervous System": Can computational cognitive neuroscience transform learning theory?

In the last century, learning theory has been dominated by an approach assuming that associations between hypothetical representational nodes can support the acquisition of knowledge about the environment. The similarities between this approach and connectionism did not go unnoticed to learning theorists, with many of them explicitly adopting a neural network approach in the modeling of learning phenomena. Skinner famously criticized such use of hypothetical neural structures for the explanation of behavior (the "Conceptual Nervous System"), and one aspect of his criticism has proven to be correct: theory underdetermination is a pervasive problem in cognitive modeling in general, and in associationist and connectionist models in particular. That is, models implementing two very different cognitive processes often make the exact same behavioral predictions, meaning that important theoretical questions posed by contrasting the two models remain unanswered. We show through several examples that theory underdetermination is common in the learning theory literature, affecting the solvability of some of the most important theoretical problems that have been posed in the last decades. Computational cognitive neuroscience (CCN) offers a solution to this problem, by including neurobiological constraints in computational models of behavior and cognition. Rather than simply being inspired by neural computation, CCN models are built to reflect as much as possible about the actual neural structures thought to underlie a particular behavior. They go beyond the "Conceptual Nervous System" and offer a true integration of behavioral and neural levels of analysis.

Recall of facial expressions and simple orientations reveals competition for resources at multiple levels of the visual hierarchy

Many studies of visual working memory have tested humans' ability to reproduce primary visual features of simple objects, such as the orientation of a grating or the hue of a color patch, following a delay. A consistent finding of such studies is that precision of responses declines as the number of items in memory increases. Here we compared visual working memory for primary features and high-level objects. We presented participants with memory arrays consisting of oriented gratings, facial expressions, or a mixture of both. Precision of reproduction for all facial expressions declined steadily as the memory load was increased from one to five faces. For primary features, this decline and the specific distributions of error observed, have been parsimoniously explained in terms of neural population codes. We adapted the population coding model for circular variables to the non-circular and bounded parameter space used for expression estimation. Total population activity was held constant according to the principle of normalization and the intensity of expression was decoded by drawing samples from the Bayesian posterior distribution. The model fit the data well, showing that principles of population coding can be applied to model memory representations at multiple levels of the visual hierarchy. When both gratings and faces had to be remembered, an asymmetry was observed. Increasing the number of faces decreased precision of orientation recall, but increasing the number of gratings did not affect recall of expression, suggesting that memorizing faces involves the automatic encoding of low-level features, in addition to higher-level expression information.

Consciousness without report: insights from summary statistics and inattention 'blindness'

We contrast two theoretical positions on the relation between phenomenal and access consciousness. First, we discuss previous data supporting a mild Overflow position, according to which transient visual awareness can overflow report. These data are open to two interpretations: (i) observers transiently experience specific visual elements outside attentional focus without encoding them into working memory; (ii) no specific visual elements but only statistical summaries are experienced in such conditions. We present new data showing that under data-limited conditions observers cannot discriminate a simple relation (same versus different) without discriminating the elements themselves and, based on additional computational considerations, we argue that this supports the first interpretation: summary statistics (same/different) are grounded on the transient experience of elements. Second, we examine recent data from a variant of 'inattention blindness' and argue that contrary to widespread assumptions, it provides further support for Overflow by highlighting another factor, 'task relevance', which affects the ability to conceptualize and report (but not experience) visual elements.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Fitting predictive coding to the neurophysiological data

Recent neurophysiological data showing the effects of locomotion on neural activity in mouse primary visual cortex has been interpreted as providing strong support for the predictive coding account of cortical function. Specifically, this work has been interpreted as providing direct evidence that prediction-error, a distinguishing property of predictive coding, is encoded in cortex. This article evaluates these claims and highlights some of the discrepancies between the proposed predictive coding model and the neuro-biology. Furthermore, it is shown that the model can be modified so as to fit the empirical data more successfully.

Neuronal Adaptation Reveals a Suboptimal Decoding of Orientation Tuned Populations in the Mouse Visual Cortex

Sensory information is encoded by populations of cortical neurons. Yet, it is unknown how this information is used for even simple perceptual choices such as discriminating orientation. To determine the computation underlying this perceptual choice, we took advantage of the robust visual adaptation in mouse primary visual cortex (V1). We first designed a stimulus paradigm in which we could vary the degree of neuronal adaptation measured in V1 during an orientation discrimination task. We then determined how adaptation affects task performance for mice of both sexes and tested which neuronal computations are most consistent with the behavioral results given the adapted population responses in V1. Despite increasing the reliability of the population representation of orientation among neurons, and improving the ability of a variety of optimal decoders to discriminate target from distractor orientations, adaptation increases animals' behavioral thresholds. Decoding the animals' choice from neuronal activity revealed that this unexpected effect on behavior could be explained by an overreliance of the perceptual choice circuit on target preferring neurons and a failure to appropriately discount the activity of neurons that prefer the distractor. Consistent with this all-positive computation, we find that animals' task performance is susceptible to subtle perturbations of distractor orientation and optogenetic suppression of neuronal activity in V1. This suggests that to solve this task the circuit has adopted a suboptimal and task-specific computation that discards important task-related information.SIGNIFICANCE STATEMENT A major goal in systems neuroscience is to understand how sensory signals are used to guide behavior. This requires determining what information in sensory cortical areas is used, and how it is combined, by downstream perceptual choice circuits. Here we demonstrate that when performing a go/no-go orientation discrimination task, mice suboptimally integrate signals from orientation tuned visual cortical neurons. While they appropriately positively weight target-preferring neurons, they fail to negatively weight distractor-preferring neurons. We propose that this all-positive computation may be adopted because of its simple learning rules and faster processing, and may be a common approach to perceptual decision-making when task conditions allow.

Weighting neurons by selectivity produces near-optimal population codes

Perception is produced by "reading out" the representation of a sensory stimulus contained in the activity of a population of neurons. To examine experimentally how populations code information, a common approach is to decode a linearly weighted sum of the neurons' spike counts. This approach is popular because of the biological plausibility of weighted, nonlinear integration. For neurons recorded in vivo, weights are highly variable when derived through optimization methods, but it is unclear how the variability affects decoding performance in practice. To address this, we recorded from neurons in the middle temporal area (MT) of anesthetized marmosets (Callithrix jacchus) viewing stimuli comprising a sheet of dots that moved coherently in 1 of 12 different directions. We found that high peak response and direction selectivity both predicted that a neuron would be weighted more highly in an optimized decoding model. Although learned weights differed markedly from weights chosen according to a priori rules based on a neuron's tuning profile, decoding performance was only marginally better for the learned weights. In the models with a priori rules, selectivity is the best predictor of weighting, and defining weights according to a neuron's preferred direction and selectivity improves decoding performance to very near the maximum level possible, as defined by the learned weights. NEW & NOTEWORTHY We examined which aspects of a neuron's tuning account for its contribution to sensory coding. Strongly direction-selective neurons are weighted most highly by optimal decoders trained to discriminate motion direction. Models with predefined decoding weights demonstrate that this weighting scheme causally improved direction representation by a neuronal population. Optimizing decoders (using a generalized linear model or Fisher's linear discriminant) led to only marginally better performance than decoders based purely on a neuron's preferred direction and selectivity.

Attention to colors induces surround suppression at category boundaries

We investigated how attention to a visual feature modulates representations of other features. The feature-similarity gain model predicts a graded modulation, whereas an alternative model asserts an inhibitory surround in feature space. Although evidence for both types of modulations can be found, a consensus has not emerged in the literature. Here, we aimed to reconcile these different views by systematically measuring how attention modulates color perception. Based on previous literature, we also predicted that color categories would impact attentional modulation. Our results showed that both surround suppression and feature-similarity gain modulate perception of colors but they operate on different similarity scales. Furthermore, the region of the suppressive surround coincided with the color category boundary, suggesting a categorical sharpening effect. We implemented a neural population coding model to explain the observed behavioral effects, which revealed a hitherto unknown connection between neural tuning shift and surround suppression.

Explaining the Effect of Likelihood Manipulation and Prior Through a Neural Network of the Audiovisual Perception of Space

Results in the recent literature suggest that multisensory integration in the brain follows the rules of Bayesian inference. However, how neural circuits can realize such inference and how it can be learned from experience is still the subject of active research. The aim of this work is to use a recent neurocomputational model to investigate how the likelihood and prior can be encoded in synapses, and how they affect audio-visual perception, in a variety of conditions characterized by different experience, different cue reliabilities and temporal asynchrony. The model considers two unisensory networks (auditory and visual) with plastic receptive fields and plastic crossmodal synapses, trained during a learning period. During training visual and auditory stimuli are more frequent and more tuned close to the fovea. Model simulations after training have been performed in crossmodal conditions to assess the auditory and visual perception bias: visual stimuli were positioned at different azimuth (±10° from the fovea) coupled with an auditory stimulus at various audio-visual distances (±20°). The cue reliability has been altered by using visual stimuli with two different contrast levels. Model predictions are compared with behavioral data. Results show that model predictions agree with behavioral data, in a variety of conditions characterized by a different role of prior and likelihood. Finally, the effect of a different unimodal or crossmodal prior, re-learning, temporal correlation among input stimuli, and visual damage (hemianopia) are tested, to reveal the possible use of the model in the clarification of important multisensory problems.

The Redemption of Noise: Inference with Neural Populations

In 2006, Ma et al. presented an elegant theory for how populations of neurons might represent uncertainty to perform Bayesian inference. Critically, according to this theory, neural variability is no longer a nuisance, but rather a vital part of how the brain encodes probability distributions and performs computations with them.

Linking signal detection theory and encoding models to reveal independent neural representations from neuroimaging data

Many research questions in visual perception involve determining whether stimulus properties are represented and processed independently. In visual neuroscience, there is great interest in determining whether important object dimensions are represented independently in the brain. For example, theories of face recognition have proposed either completely or partially independent processing of identity and emotional expression. Unfortunately, most previous research has only vaguely defined what is meant by “independence,” which hinders its precise quantification and testing. This article develops a new quantitative framework that links signal detection theory from psychophysics and encoding models from computational neuroscience, focusing on a special form of independence defined in the psychophysics literature: perceptual separability. The new theory allowed us, for the first time, to precisely define separability of neural representations and to theoretically link behavioral and brain measures of separability. The framework formally specifies the relation between these different levels of perceptual and brain representation, providing the tools for a truly integrative research approach. In particular, the theory identifies exactly what valid inferences can be made about independent encoding of stimulus dimensions from the results of multivariate analyses of neuroimaging data and psychophysical studies. In addition, commonly used operational tests of independence are re-interpreted within this new theoretical framework, providing insights on their correct use and interpretation. Finally, we apply this new framework to the study of separability of brain representations of face identity and emotional expression (neutral/sad) in a human fMRI study with male and female participants.

A common question in vision research is whether certain stimulus properties, like face identity and expression, are represented and processed independently. We develop a theoretical framework that allowed us, for the first time, to link behavioral and brain measures of independence. Unlike previous approaches, our framework formally specifies the relation between these different levels of perceptual and brain representation, providing the tools for a truly integrative research approach in the study of independence. This allows to identify what kind of inferences can be made about brain representations from multivariate analyses of neuroimaging data or psychophysical studies. We apply this framework to the study of independent processing of face identity and expression.

Efficient Coding in Visual Working Memory Accounts for Stimulus-Specific Variations in Recall

Recall of visual features from working memory varies in both bias and precision depending on stimulus parameters. Whereas a number of models can approximate the average distribution of recall error across target stimuli, attempts to model how error varies with the choice of target have been ad hoc. Here we adapt a neural model of working memory to provide a principled account of these stimulus-specific effects, by allowing each neuron's tuning function to vary according to the principle of efficient coding, which states that neural responses should be optimized with respect to the frequency of stimuli in nature. For orientation, this means incorporating a prior that favors cardinal over oblique orientations. While continuing to capture the changes in error distribution with set size, the resulting model accurately described stimulus-specific variations as well, better than a slot-based competitor. Efficient coding produces a repulsive bias away from cardinal orientations, a bias that ought to be sensitive to changes in the environmental statistics. We subsequently tested whether shifts in the stimulus distribution influenced response bias to uniformly sampled target orientations in human subjects (of either sex). Across adaptation blocks, we manipulated the distribution of nontarget items by sampling from a bimodal congruent (incongruent) distribution with peaks centered on cardinal (oblique) orientations. Preadaptation responses were repulsed away from the cardinal axes. However, exposure to the incongruent distribution produced systematic decreases in repulsion that persisted after adaptation. This result confirms the role of prior expectation in generating stimulus-specific effects and validates the neural framework.

SIGNIFICANCE STATEMENT Theories of neural coding have been used successfully to explain how errors in recall from working memory depend on the number of items stored. However, recall of visual features also shows stimulus-specific variation in bias and precision. Here we unify two previously unconnected theories, the neural resource model of working memory and the efficient coding framework, to provide a principled account of these stimulus-specific effects. Given the importance of working memory limitations to multiple aspects of human and animal behavior, and the recent high-profile advances in theories of efficient coding, our modeling framework provides a richer, yet parsimonious, description of how orientation encoding influences visual working memory performance.

A biologically inspired neurocomputational model for audiovisual integration and causal inference

Recently, experimental and theoretical research has focused on the brain's abilities to extract information from a noisy sensory environment and how cross-modal inputs are processed to solve the causal inference problem to provide the best estimate of external events. Despite the empirical evidence suggesting that the nervous system uses a statistically optimal and probabilistic approach in addressing these problems, little is known about the brain's architecture needed to implement these computations. The aim of this work was to realize a mathematical model, based on physiologically plausible hypotheses, to analyze the neural mechanisms underlying multisensory perception and causal inference. The model consists of three layers topologically organized: two encode auditory and visual stimuli, separately, and are reciprocally connected via excitatory synapses and send excitatory connections to the third downstream layer. This synaptic organization realizes two mechanisms of cross-modal interactions: the first is responsible for the sensory representation of the external stimuli, while the second solves the causal inference problem. We tested the network by comparing its results to behavioral data reported in the literature. Among others, the network can account for the ventriloquism illusion, the pattern of sensory bias and the percept of unity as a function of the spatial auditory-visual distance, and the dependence of the auditory error on the causal inference. Finally, simulations results are consistent with probability matching as the perceptual strategy used in auditory-visual spatial localization tasks, agreeing with the behavioral data. The model makes untested predictions that can be investigated in future behavioral experiments.