Bayesian integration in sensorimotor learning

When we learn a new motor skill, such as playing an approaching tennis ball, both our sensors and the task possess variability. Our sensors provide imperfect information about the ball's velocity, so we can only estimate it. Combining information from multiple modalities can reduce the error in this estimate. On a longer time scale, not all velocities are a priori equally probable, and over the course of a match there will be a probability distribution of velocities. According to bayesian theory, an optimal estimate results from combining information about the distribution of velocities-the prior-with evidence from sensory feedback. As uncertainty increases, when playing in fog or at dusk, the system should increasingly rely on prior knowledge. To use a bayesian strategy, the brain would need to represent the prior distribution and the level of uncertainty in the sensory feedback. Here we control the statistical variations of a new sensorimotor task and manipulate the uncertainty of the sensory feedback. We show that subjects internally represent both the statistical distribution of the task and their sensory uncertainty, combining them in a manner consistent with a performance-optimizing bayesian process. The central nervous system therefore employs probabilistic models during sensorimotor learning.

Causal inference in multisensory perception

Perceptual events derive their significance to an animal from their meaning about the world, that is from the information they carry about their causes. The brain should thus be able to efficiently infer the causes underlying our sensory events. Here we use multisensory cue combination to study causal inference in perception. We formulate an ideal-observer model that infers whether two sensory cues originate from the same location and that also estimates their location(s). This model accurately predicts the nonlinear integration of cues by human subjects in two auditory-visual localization tasks. The results show that indeed humans can efficiently infer the causal structure as well as the location of causes. By combining insights from the study of causal inference with the ideal-observer approach to sensory cue combination, we show that the capacity to infer causal structure is not limited to conscious, high-level cognition; it is also performed continually and effortlessly in perception.

Bayesian decision theory in sensorimotor control

Action selection is a fundamental decision process for us, and depends on the state of both our body and the environment. Because signals in our sensory and motor systems are corrupted by variability or noise, the nervous system needs to estimate these states. To select an optimal action these state estimates need to be combined with knowledge of the potential costs or rewards of different action outcomes. We review recent studies that have investigated the mechanisms used by the nervous system to solve such estimation and decision problems, which show that human behaviour is close to that predicted by Bayesian Decision Theory. This theory defines optimal behaviour in a world characterized by uncertainty, and provides a coherent way of describing sensorimotor processes.

Inferring functional connections between neurons

A central question in neuroscience is how interactions between neurons give rise to behavior. In many electrophysiological experiments, the activity of a set of neurons is recorded while sensory stimuli or movement tasks are varied. Tools that aim to reveal underlying interactions between neurons from such data can be extremely useful. Traditionally, neuroscientists have studied these interactions using purely descriptive statistics (cross-correlograms or joint peri-stimulus time histograms). However, the interpretation of such data is often difficult, particularly as the number of recorded neurons grows. Recent research suggests that model-based, maximum likelihood methods can improve these analyses. In addition to estimating neural interactions, application of these techniques has improved decoding of external variables, created novel interpretations of existing electrophysiological data, and may provide new insight into how the brain represents information.

Integrating top-down and bottom-up sensory processing by somato-dendritic interactions

The classical view of cortical information processing is that of a bottom-up process in a feedforward hierarchy. However, psychophysical, anatomical, and physiological evidence suggests that top-down effects play a crucial role in the processing of input stimuli. Not much is known about the neural mechanisms underlying these effects. Here we investigate a physiologically inspired model of two reciprocally connected cortical areas. Each area receives bottom-up as well as top-down information. This information is integrated by a mechanism that exploits recent findings on somato-dendritic interactions. (1) This results in a burst signal that is robust in the context of noise in bottom-up signals. (2) Investigating the influence of additional top-down information, priming-like effects on the processing of bottom-up input can be demonstrated. (3) In accordance with recent physiological findings, interareal coupling in low-frequency ranges is characteristically enhanced by top-down mechanisms. The proposed scheme combines a qualitative influence of top-down directed signals on the temporal dynamics of neuronal activity with a limited effect on the mean firing rate of the targeted neurons. As it gives an account of the system properties on the cellular level, it is possible to derive several experimentally testable predictions.

Processing of complex stimuli and natural scenes in the visual cortex

A major part of vision research builds on the assumption that processing of visual stimuli can be understood on the basis of knowledge about the processing of simplified, artificial stimuli. Recent experimental advances, however, show that a combination of responses to simplified stimuli does not adequately describe responses to natural visual scenes. The systems performance exceeds the performance predicted from understanding its basic constituents. This highlights the fact that the visual system is specifically adapted to the properties of its everyday input and can only fully be understood when probed with naturalistic stimuli.

Bayesian Integration in Force Estimation

When we interact with objects in the world, the forces we exert are finely tuned to the dynamics of the situation. As our sensors do not provide perfect knowledge about the environment, a key problem is how to estimate the appropriate forces. Two sources of information can be used to generate such an estimate: sensory inputs about the object and knowledge about previously experienced objects, termed prior information. Bayesian integration defines the way in which these two sources of information should be combined to produce an optimal estimate. To investigate whether subjects use such a strategy in force estimation, we designed a novel sensorimotor estimation task. We controlled the distribution of forces experienced over the course of an experiment thereby defining the prior. We show that subjects integrate sensory information with their prior experience to generate an estimate. Moreover, subjects could learn different prior distributions. These results suggest that the CNS uses Bayesian models when estimating force requirements.

The world from a cat's perspective--statistics of natural videos

The mammalian visual system is one of the most intensively investigated sensory systems. However, our knowledge of the typical input it is operating on is surprisingly limited. To address this issue, we seek to learn about the natural visual environment and the world as seen by a cat. With a CCD camera attached to their head, cats explore several outdoor environments and videos of natural stimuli are recorded from the animals' perspective. The statistical analysis of these videos reveals several remarkable properties. First, we find an anisotropy of oriented contours with an enhanced occurrence of horizontal orientations, earlier described in the "oblique effect" as a predominance of the two cardinal orientations. Second, contrast is not elevated in the center of the images, suggesting different mechanisms of fixation point selection as compared to humans. Third, analyzing a sequence of images we find that the precise position of contours varies faster than their orientation. Finally, collinear contours prevail over parallel shifted contours, matching recent physiological and anatomical results. These findings demonstrate the rich structure of natural visual stimuli and its direct relation to extensively studied anatomical and physiological issues.

Statistics of natural movements are reflected in motor errors

Humans use their arms to engage in a wide variety of motor tasks during everyday life. However, little is known about the statistics of these natural arm movements. Studies of the sensory system have shown that the statistics of sensory inputs are key to determining sensory processing. We hypothesized that the statistics of natural everyday movements may, in a similar way, influence motor performance as measured in laboratory-based tasks. We developed a portable motion-tracking system that could be worn by subjects as they went about their daily routine outside of a laboratory setting. We found that the well-documented symmetry bias is reflected in the relative incidence of movements made during everyday tasks. Specifically, symmetric and antisymmetric movements are predominant at low frequencies, whereas only symmetric movements are predominant at high frequencies. Moreover, the statistics of natural movements, that is, their relative incidence, correlated with subjects' performance on a laboratory-based phase-tracking task. These results provide a link between natural movement statistics and motor performance and confirm that the symmetry bias documented in laboratory studies is a natural feature of human movement.

Bayesian inference of functional connectivity and network structure from spikes

Current multielectrode techniques enable the simultaneous recording of spikes from hundreds of neurons. To study neural plasticity and network structure it is desirable to infer the underlying functional connectivity between the recorded neurons. Functional connectivity is defined by a large number of parameters, which characterize how each neuron influences the other neurons. A Bayesian approach that combines information from the recorded spikes (likelihood) with prior beliefs about functional connectivity (prior) can improve inference of these parameters and reduce overfitting. Recent studies have used likelihood functions based on the statistics of point-processes and a prior that captures the sparseness of neural connections. Here we include a prior that captures the empirical finding that interactions tend to vary smoothly in time. We show that this method can successfully infer connectivity patterns in simulated data and apply the algorithm to spike data recorded from primary motor (M1) and premotor (PMd) cortices of a monkey. Finally, we present a new approach to studying structure in inferred connections based on a Bayesian clustering algorithm. Groups of neurons in M1 and PMd show common patterns of input and output that may correspond to functional assemblies.

Rewiring neural interactions by micro-stimulation

Plasticity is a crucial component of normal brain function and a critical mechanism for recovery from injury. In vitro, associative pairing of presynaptic spiking and stimulus-induced postsynaptic depolarization causes changes in the synaptic efficacy of the presynaptic neuron, when activated by extrinsic stimulation. In vivo, such paradigms can alter the responses of whole groups of neurons to stimulation. Here, we used in vivo spike-triggered stimulation to drive plastic changes in rat forelimb sensorimotor cortex, which we monitored using a statistical measure of functional connectivity inferred from the spiking statistics of the neurons during normal, spontaneous behavior. These induced plastic changes in inferred functional connectivity depended on the latency between trigger spike and stimulation, and appear to reflect a robust reorganization of the network. Such targeted connectivity changes might provide a tool for rerouting the flow of information through a network, with implications for both rehabilitation and brain–machine interface applications.

How are complex cell properties adapted to the statistics of natural stimuli?

Sensory areas should be adapted to the properties of their natural stimuli. What are the underlying rules that match the properties of complex cells in primary visual cortex to their natural stimuli? To address this issue, we sampled movies from a camera carried by a freely moving cat, capturing the dynamics of image motion as the animal explores an outdoor environment. We use these movie sequences as input to simulated neurons. Following the intuition that many meaningful high-level variables, e.g., identities of visible objects, do not change rapidly in natural visual stimuli, we adapt the neurons to exhibit firing rates that are stable over time. We find that simulated neurons, which have optimally stable activity, display many properties that are observed for cortical complex cells. Their response is invariant with respect to stimulus translation and reversal of contrast polarity. Furthermore, spatial frequency selectivity and the aspect ratio of the receptive field quantitatively match the experimentally observed characteristics of complex cells. Hence, the population of complex cells in the primary visual cortex can be described as forming an optimally stable representation of natural stimuli.

Pain: A Statistical Account

Perception is seen as a process that utilises partial and noisy information to construct a coherent understanding of the world. Here we argue that the experience of pain is no different; it is based on incomplete, multimodal information, which is used to estimate potential bodily threat. We outline a Bayesian inference model, incorporating the key components of cue combination, causal inference, and temporal integration, which highlights the statistical problems in everyday perception. It is from this platform that we are able to review the pain literature, providing evidence from experimental, acute, and persistent phenomena to demonstrate the advantages of adopting a statistical account in pain. Our probabilistic conceptualisation suggests a principles-based view of pain, explaining a broad range of experimental and clinical findings and making testable predictions.

A neuroeconomics approach to inferring utility functions in sensorimotor control

Making choices is a fundamental aspect of human life. For over a century experimental economists have characterized the decisions people make based on the concept of a utility function. This function increases with increasing desirability of the outcome, and people are assumed to make decisions so as to maximize utility. When utility depends on several variables, indifference curves arise that represent outcomes with identical utility that are therefore equally desirable. Whereas in economics utility is studied in terms of goods and services, the sensorimotor system may also have utility functions defining the desirability of various outcomes. Here, we investigate the indifference curves when subjects experience forces of varying magnitude and duration. Using a two-alternative forced-choice paradigm, in which subjects chose between different magnitude–duration profiles, we inferred the indifference curves and the utility function. Such a utility function defines, for example, whether subjects prefer to lift a 4-kg weight for 30 s or a 1-kg weight for a minute. The measured utility function depends nonlinearly on the force magnitude and duration and was remarkably conserved across subjects. This suggests that the utility function, a central concept in economics, may be applicable to the study of sensorimotor control.

Economists use the concept of a utility function, which increases with increasing desirability of the outcome, to characterize human decision making. This concept is shown here to apply to the control of movement

Quantifying Mesoscale Neuroanatomy Using X-Ray Microtomography

Methods for resolving the three-dimensional (3D) microstructure of the brain typically start by thinly slicing and staining the brain, followed by imaging numerous individual sections with visible light photons or electrons. In contrast, X-rays can be used to image thick samples, providing a rapid approach for producing large 3D brain maps without sectioning. Here we demonstrate the use of synchrotron X-ray microtomography (µCT) for producing mesoscale (∼1 µm 3 resolution) brain maps from millimeter-scale volumes of mouse brain. We introduce a pipeline for µCT-based brain mapping that develops and integrates methods for sample preparation, imaging, and automated segmentation of cells, blood vessels, and myelinated axons, in addition to statistical analyses of these brain structures. Our results demonstrate that X-ray tomography achieves rapid quantification of large brain volumes, complementing other brain mapping and connectomics efforts.

Supervised and unsupervised learning with two sites of synaptic integration

Many learning rules for neural networks derive from abstract objective functions. The weights in those networks are typically optimized utilizing gradient ascent on the objective function. In those networks each neuron needs to store two variables. One variable, called activity, contains the bottom-up sensory-fugal information involved in the core signal processing. The other variable typically describes the derivative of the objective function with respect to the cell's activity and is exclusively used for learning. This variable allows the objective function's derivative to be calculated with respect to each weight and thus the weight update. Although this approach is widely used, the mapping of such two variables onto physiology is unclear, and these learning algorithms are often considered biologically unrealistic. However, recent research on the properties of cortical pyramidal neurons shows that these cells have at least two sites of synaptic integration, the basal and the apical dendrite, and are thus appropriately described by at least two variables. Here we discuss whether these results could constitute a physiological basis for the described abstract learning rules. As examples we demonstrate an implementation of the backpropagation of error algorithm and a specific self-supervised learning algorithm using these principles. Thus, compared to standard, one-integration-site neurons, it is possible to incorporate interesting properties in neural networks that are inspired by physiology with a modest increase of complexity.

Learning the invariance properties of complex cells from their responses to natural stimuli

Neurons in primary visual cortex are typically classified as either simple or complex. Whereas simple cells respond strongly to grating and bar stimuli displayed at a certain phase and visual field location, complex cell responses are insensitive to small translations of the stimulus within the receptive field [Hubel & Wiesel (1962) J. Physiol. (Lond.), 160, 106-154; Kjaer et al. (1997) J. Neurophysiol., 78, 3187-3197]. This constancy in the response to variations of the stimuli is commonly called invariance. Hubel and Wiesel's classical model of the primary visual cortex proposes a connectivity scheme which successfully describes simple and complex cell response properties. However, the question as to how this connectivity arises during normal development is left open. Based on their work and inspired by recent physiological findings we suggest a network model capable of learning from natural stimuli and developing receptive field properties which match those of cortical simple and complex cells. Stimuli are drawn from videos obtained by a camera mounted to a cat's head, so they should approximate the natural input to the cat's visual system. The network uses a competitive scheme to learn simple and complex cell response properties. Employing delayed signals to learn connections between simple and complex cells enables the model to utilize temporal properties of the input. We show that the temporal structure of the input gives rise to the emergence and refinement of complex cell receptive fields, whereas removing temporal continuity prevents this processes. This model lends a physiologically based explanation of the development of complex cell invariance response properties.

Bayesian integration and non-linear feedback control in a full-body motor task

A large number of experiments have asked to what degree human reaching movements can be understood as being close to optimal in a statistical sense. However, little is known about whether these principles are relevant for other classes of movements. Here we analyzed movement in a task that is similar to surfing or snowboarding. Human subjects stand on a force plate that measures their center of pressure. This center of pressure affects the acceleration of a cursor that is displayed in a noisy fashion (as a cloud of dots) on a projection screen while the subject is incentivized to keep the cursor close to a fixed position. We find that salient aspects of observed behavior are well-described by optimal control models where a Bayesian estimation model (Kalman filter) is combined with an optimal controller (either a Linear-Quadratic-Regulator or Bang-bang controller). We find evidence that subjects integrate information over time taking into account uncertainty. However, behavior in this continuous steering task appears to be a highly non-linear function of the visual feedback. While the nervous system appears to implement Bayes-like mechanisms for a full-body, dynamic task, it may additionally take into account the specific costs and constraints of the task.

There is a growing body of work demonstrating that humans are close to statistically optimal in both their perception of the world and their actions on it. That is, we seem to combine information from our sensors with the constraints and costs of moving to minimize our errors and effort. Most of the evidence for this type of behavior comes from tasks such as reaching in a small workspace or standing on a force plate passively viewing a stimulus. Although humans appear to be near-optimal for these tasks, it is not clear whether the theory holds for other tasks. Here we introduce a full-body, goal-directed task similar to surfing or snowboarding where subjects steer a cursor with their center of pressure. We find that subjects respond to sensory uncertainty near-optimally in this task, but their behavior is highly non-linear. This suggests that the computations performed by the nervous system may take into account a more complicated set of costs and constraints than previously supposed.

Hierarchical Bayesian modeling and Markov chain Monte Carlo sampling for tuning-curve analysis

A central theme of systems neuroscience is to characterize the tuning of neural responses to sensory stimuli or the production of movement. Statistically, we often want to estimate the parameters of the tuning curve, such as preferred direction, as well as the associated degree of uncertainty, characterized by error bars. Here we present a new sampling-based, Bayesian method that allows the estimation of tuning-curve parameters, the estimation of error bars, and hypothesis testing. This method also provides a useful way of visualizing which tuning curves are compatible with the recorded data. We demonstrate the utility of this approach using recordings of orientation and direction tuning in primary visual cortex, direction of motion tuning in primary motor cortex, and simulated data.

A cryptography-based approach for movement decoding

Brain decoders use neural recordings to infer the activity or intent of a user. To train a decoder, one generally needs to infer the measured variables of interest (covariates) from simultaneously measured neural activity. However, there are cases for which obtaining supervised data is difficult or impossible. Here, we describe an approach for movement decoding that does not require access to simultaneously measured neural activity and motor outputs. We use the statistics of movement-much like cryptographers use the statistics of language-to find a mapping between neural activity and motor variables, and then align the distribution of decoder outputs with the typical distribution of motor outputs by minimizing their Kullback-Leibler divergence. By using datasets collected from the motor cortex of three non-human primates performing either a reaching task or an isometric force-production task, we show that the performance of such a distribution-alignment decoding algorithm is comparable to the performance of supervised approaches. Distribution-alignment decoding promises to broaden the set of potential applications of brain decoding.

Real-time evaluation of a noninvasive neuroprosthetic interface for control of reach

Injuries of the cervical spinal cord can interrupt the neural pathways controlling the muscles of the arm, resulting in complete or partial paralysis. For individuals unable to reach due to high-level injuries, neuroprostheses can restore some of the lost function. Natural, multidimensional control of neuroprosthetic devices for reaching remains a challenge. Electromyograms (EMGs) from muscles that remain under voluntary control can be used to communicate intended reach trajectories, but when the number of available muscles is limited control can be difficult and unintuitive. We combined shoulder EMGs with target estimates obtained from gaze. Natural gaze data were integrated with EMG during closed-loop robotic control of the arm, using a probabilistic mixture model. We tested the approach with two different sets of EMGs, as might be available to subjects with C4- and C5-level spinal cord injuries. Incorporating gaze greatly improved control of reaching, particularly when there were few EMG signals. We found that subjects naturally adapted their eye-movement precision as we varied the set of available EMGs, attaining accurate performance in both tested conditions. The system performs a near-optimal combination of both physiological signals, making control more intuitive and allowing a natural trajectory that reduces the burden on the user.

The uncertainty associated with visual flow fields and their influence on postural sway: Weber's law suffices to explain the nonlinearity of vection

When we stand upright, we integrate cues from multiple senses, such as vision and proprioception, to maintain and regulate our vertical posture. How these cues are combined has been the focus of a range of studies. These studies generally measured how subjects deviate from standing upright when confronted with a moving visual stimulus displayed in a virtual environment. Previous research had shown that uncertainty is central in such cue combination problems. Here we wanted to understand, quantitatively, how visual flow fields and uncertainty about them affect human posture. To do so, we combined experimental methods from perceptual psychophysics with methods from motor control studies. We used a two-alternative forced-choice paradigm to measure uncertainty as a function of the magnitude of a random-dot flow field and stimulus coherence. We subsequently measured movement amplitude as a function of visual stimulus parameters. In line with previous research, we find that sensorimotor behavior depends nonlinearly on the stimulus amplitude and, importantly, is affected by uncertainty. We find that this nonlinearity and uncertainty dependence is accurately predicted by standard Bayesian cue combination. Importantly, a Weber's law where visual uncertainty depends on stimulus amplitude is enough to explain the nonlinear behavior.

A learning rule for dynamic recruitment and decorrelation

The interest in neuronal networks originates for a good part in the option not to construct, but to train them. The mechanisms governing synaptic modifications during such training are assumed to depend on signals locally available at the synapses. In contrast, the performance of a network is suitably measured on a global scale. Here we propose a learning rule that addresses this conflict. It is inspired by recent physiological experiments and exploits the interaction of inhibitory input and backpropagating action potentials in pyramidal neurons. This mechanism makes information on the global scale available as a local signal. As a result, several desirable features can be combined: the learning rule allows fast synaptic modifications approaching one-shot learning. Nevertheless, it leads to stable representations during ongoing learning. Furthermore, the response properties of the neurons are not globally correlated, but cover the whole stimulus space.

Decoding with limited neural data: a mixture of time-warped trajectory models for directional reaches

Neuroprosthetic devices promise to allow paralyzed patients to perform the necessary functions of everyday life. However, to allow patients to use such tools it is necessary to decode their intent from neural signals such as electromyograms (EMGs). Because these signals are noisy, state of the art decoders integrate information over time. One systematic way of doing this is by taking into account the natural evolution of the state of the body--by using a so-called trajectory model. Here we use two insights about movements to enhance our trajectory model: (1) at any given time, there is a small set of likely movement targets, potentially identified by gaze; (2) reaches are produced at varying speeds. We decoded natural reaching movements using EMGs of muscles that might be available from an individual with spinal cord injury. Target estimates found from tracking eye movements were incorporated into the trajectory model, while a mixture model accounted for the inherent uncertainty in these estimates. Warping the trajectory model in time using a continuous estimate of the reach speed enabled accurate decoding of faster reaches. We found that the choice of richer trajectory models, such as those incorporating target or speed, improves decoding particularly when there is a small number of EMGs available.

Learning the Nonlinearity of Neurons from Natural Visual Stimuli

Learning in neural networks is usually applied to parameters related to linear kernels and keeps the nonlinearity of the model fixed. Thus, for successful models, properties and parameters of the nonlinearity have to be specified using a priori knowledge, which often is missing. Here, we investigate adapting the nonlinearity simultaneously with the linear kernel. We use natural visual stimuli for training a simple model of the visual system. Many of the neurons converge to an energy detector matching existing models of complex cells. The overall distribution of the parameter describing the nonlinearity well matches recent physiological results. Controls with randomly shuffled natural stimuli and pink noise demonstrate that the match of simulation and experimental results depends on the higher-order statistical properties of natural stimuli.