Learning from the past: A reverberation of past errors in the cerebellar climbing fiber signal

A cerebellar population coding model for sensorimotor learning

The cerebellum is crucial for sensorimotor adaptation, using error information to keep the sensorimotor system well-calibrated. Here we introduce a population-coding model to explain how cerebellar-dependent learning is modulated by contextual variation. The model consists of a two-layer network, designed to capture activity in both the cerebellar cortex and deep cerebellar nuclei. A core feature of the model is that within each layer, the processing units are tuned to both movement direction and the direction of movement error. The model captures a large range of contextual effects including interference from prior learning and the influence of error uncertainty and volatility. While these effects have traditionally been taken to indicate meta learning or context-dependent memory within the adaptation system, our results show that they are emergent properties that arise from the population dynamics within the cerebellum. Our results provide a novel framework to understand how the nervous system responds to variable environments.

Complex spikes perturb movements and reveal the sensorimotor map of Purkinje cells

Computations that are performed by the cerebellar cortex are transmitted via simple spikes of Purkinje cells (P-cells) to downstream structures, but because P-cells are many synapses away from muscles, we do not know the relationship between modulation of simple spikes and control of behavior. Here, we recorded the spiking activities of hundreds of P-cells in the oculomotor vermis of marmosets during saccadic eye movements and found that following the presentation of a visual stimulus, the olivary input to a P-cell coarsely described the direction and amplitude of the visual stimulus as well as the upcoming movement. Occasionally, the complex spike occurred just before saccade onset, suppressing the P-cell's simple spikes and disrupting its output during that saccade. Remarkably, this brief suppression of simple spikes altered the saccade's trajectory by pulling the eyes toward the part of the visual space that was preferentially encoded by the olivary input to that P-cell. Thus, there is an alignment between the sensory space encoded by the complex spikes and the behavior conveyed by the simple spikes: a reduction in simple spikes is a signal to bias the ongoing movement toward the part of the sensory space preferentially encoded by the olivary input to that P-cell.

Multidimensional cerebellar computations for flexible kinematic control of movements

Both the environment and our body keep changing dynamically. Hence, ensuring movement precision requires adaptation to multiple demands occurring simultaneously. Here we show that the cerebellum performs the necessary multi-dimensional computations for the flexible control of different movement parameters depending on the prevailing context. This conclusion is based on the identification of a manifold-like activity in both mossy fibers (MFs, network input) and Purkinje cells (PCs, output), recorded from monkeys performing a saccade task. Unlike MFs, the PC manifolds developed selective representations of individual movement parameters. Error feedback-driven climbing fiber input modulated the PC manifolds to predict specific, error type-dependent changes in subsequent actions. Furthermore, a feed-forward network model that simulated MF-to-PC transformations revealed that amplification and restructuring of the lesser variability in the MF activity is a pivotal circuit mechanism. Therefore, the flexible control of movements by the cerebellum crucially depends on its capacity for multi-dimensional computations.

Complex spikes perturb movements, revealing the sensorimotor map of Purkinje cells

The cerebellar cortex performs computations that are critical for control of our actions, and then transmits that information via simple spikes of Purkinje cells (P-cells) to downstream structures. However, because P-cells are many synapses away from muscles, we do not know how their output affects behavior. Furthermore, we do not know the level of abstraction, i.e., the coordinate system of the P-cell's output. Here, we recorded spiking activities of hundreds of P-cells in the oculomotor vermis of marmosets during saccadic eye movements and found that following the presentation of a visual stimulus, the olivary input to a P-cell encoded a probabilistic signal that coarsely described both the direction and the amplitude of that stimulus. When this input was present, the resulting complex spike briefly suppressed the P-cell's simple spikes, disrupting the P-cell's output during that saccade. Remarkably, this brief suppression altered the saccade's trajectory by pulling the eyes toward the part of the visual space that was preferentially encoded by the olivary input to that P-cell. Thus, analysis of behavior in the milliseconds following a complex spike unmasked how the P-cell's output influenced behavior: the preferred location in the coordinates of the visual system as conveyed probabilistically from the inferior olive to a P-cell defined the action in the coordinates of the motor system for which that P-cell's simple spikes directed behavior.

Cerebellar complex spikes multiplex complementary behavioral information

Purkinje cell (PC) discharge, the only output of cerebellar cortex, involves 2 types of action potentials, high-frequency simple spikes (SSs) and low-frequency complex spikes (CSs). While there is consensus that SSs convey information needed to optimize movement kinematics, the function of CSs, determined by the PC’s climbing fiber input, remains controversial. While initially thought to be specialized in reporting information on motor error for the subsequent amendment of behavior, CSs seem to contribute to other aspects of motor behavior as well. When faced with the bewildering diversity of findings and views unraveled by highly specific tasks, one may wonder if there is just one true function with all the other attributions wrong? Or is the diversity of findings a reflection of distinct pools of PCs, each processing specific streams of information conveyed by climbing fibers? With these questions in mind, we recorded CSs from the monkey oculomotor vermis deploying a repetitive saccade task that entailed sizable motor errors as well as small amplitude saccades, correcting them. We demonstrate that, in addition to carrying error-related information, CSs carry information on the metrics of both primary and small corrective saccades in a time-specific manner, with changes in CS firing probability coupled with changes in CS duration. Furthermore, we also found CS activity that seemed to predict the upcoming events. Hence PCs receive a multiplexed climbing fiber input that merges complementary streams of information on the behavior, separable by the recipient PC because they are staggered in time.

Purkinje cell (PC) discharge, the only output of cerebellar cortex, involves both high-frequency simple spikes and low-frequency complex spikes; the function of the latter, determined by a PC’s climbing fibre input, remains controversial. This study shows that PCs receive a multiplexed climbing fibre input that merges complementary streams of information relevant for behaviour.

Bidirectional learning in upbound and downbound microzones of the cerebellum

Over the past several decades, theories about cerebellar learning have evolved. A relatively simple view that highlighted the contribution of one major form of heterosynaptic plasticity to cerebellar motor learning has given way to a plethora of perspectives that suggest that many different forms of synaptic and non-synaptic plasticity, acting at various sites, can control multiple types of learning behaviour. However, there still seem to be contradictions between the various hypotheses with regard to the mechanisms underlying cerebellar learning. The challenge is therefore to reconcile these different views and unite them into a single overall concept. Here I review our current understanding of the changes in the activity of cerebellar Purkinje cells in different 'microzones' during various forms of learning. I describe an emerging model that indicates that the activity of each microzone is bound to either increase or decrease during the initial stages of learning, depending on the directional and temporal demands of its downstream circuitry and the behaviour involved.

Population coding in the cerebellum: a machine learning perspective

The cere resembles a feedforward, three-layer network of neurons in which the "hidden layer" consists of Purkinje cells (P-cells) and the output layer consists of deep cerebellar nucleus (DCN) neurons. In this analogy, the output of each DCN neuron is a prediction that is compared with the actual observation, resulting in an error signal that originates in the inferior olive. Efficient learning requires that the error signal reach the DCN neurons, as well as the P-cells that project onto them. However, this basic rule of learning is violated in the cerebellum: the olivary projections to the DCN are weak, particularly in adulthood. Instead, an extraordinarily strong signal is sent from the olive to the P-cells, producing complex spikes. Curiously, P-cells are grouped into small populations that converge onto single DCN neurons. Why are the P-cells organized in this way, and what is the membership criterion of each population? Here, I apply elementary mathematics from machine learning and consider the fact that P-cells that form a population exhibit a special property: they can synchronize their complex spikes, which in turn suppress activity of DCN neuron they project to. Thus complex spikes cannot only act as a teaching signal for a P-cell, but through complex spike synchrony, a P-cell population may act as a surrogate teacher for the DCN neuron that produced the erroneous output. It appears that grouping of P-cells into small populations that share a preference for error satisfies a critical requirement of efficient learning: providing error information to the output layer neuron (DCN) that was responsible for the error, as well as the hidden layer neurons (P-cells) that contributed to it. This population coding may account for several remarkable features of behavior during learning, including multiple timescales, protection from erasure, and spontaneous recovery of memory.

Using deep neural networks to detect complex spikes of cerebellar Purkinje cells

Purkinje cell “complex spikes,” fired at perplexingly low rates, play a crucial role in cerebellum-based motor learning. Careful interpretations of these spikes require manually detecting them, since conventional online or offline spike sorting algorithms are optimized for classifying much simpler waveform morphologies. We present a novel deep learning approach for identifying complex spikes, which also measures additional relevant neurophysiological features, with an accuracy level matching that of human experts yet with very little time expenditure.

Bidirectional short-term plasticity during single-trial learning of cerebellar-driven eyelid movements in mice

The brain is constantly monitoring its own performance, using error signals to trigger mechanisms of plasticity that help improve future behavior. Indeed, adaptive changes in behavior have been observed after a single error trial in many learning tasks, including cerebellum-dependent eyeblink conditioning. Here, we demonstrate that the plasticity underlying single-trial learning during eyeblink conditioning in mice is bidirectionally regulated by positive and negative prediction errors, has an ephemeral effect on behavior (decays in <1 min), and can be triggered in the absence of errors in performance. We suggest that these three properties of single-trial learning may be particularly useful for driving mechanisms of motor adaptation that can achieve optimal performance in the face of environmental disturbances with a fast timescale.

A generative learning model for saccade adaptation

Plasticity in the oculomotor system ensures that saccadic eye movements reliably meet their visual goals-to bring regions of interest into foveal, high-acuity vision. Here, we present a comprehensive description of sensorimotor learning in saccades. We induced continuous adaptation of saccade amplitudes using a double-step paradigm, in which participants saccade to a peripheral target stimulus, which then undergoes a surreptitious, intra-saccadic shift (ISS) as the eyes are in flight. In our experiments, the ISS followed a systematic variation, increasing or decreasing from one saccade to the next as a sinusoidal function of the trial number. Over a large range of frequencies, we confirm that adaptation gain shows (1) a periodic response, reflecting the frequency of the ISS with a delay of a number of trials, and (2) a simultaneous drift towards lower saccade gains. We then show that state-space-based linear time-invariant systems (LTIS) represent suitable generative models for this evolution of saccade gain over time. This state-equation algorithm computes the prediction of an internal (or hidden state-) variable by learning from recent feedback errors, and it can be compared to experimentally observed adaptation gain. The algorithm also includes a forgetting rate that quantifies per-trial leaks in the adaptation gain, as well as a systematic, non-error-based bias. Finally, we study how the parameters of the generative models depend on features of the ISS. Driven by a sinusoidal disturbance, the state-equation admits an exact analytical solution that expresses the parameters of the phenomenological description as functions of those of the generative model. Together with statistical model selection criteria, we use these correspondences to characterize and refine the structure of compatible state-equation models. We discuss the relation of these findings to established results and suggest that they may guide further design of experimental research across domains of sensorimotor adaptation.

Role of the Vermal Cerebellum in Visually Guided Eye Movements and Visual Motion Perception

The cerebellar cortex is a crystal-like structure consisting of an almost endless repetition of a canonical microcircuit that applies the same computational principle to different inputs. The output of this transformation is broadcasted to extracerebellar structures by way of the deep cerebellar nuclei. Visually guided eye movements are accommodated by different parts of the cerebellum. This review primarily discusses the role of the oculomotor part of the vermal cerebellum [the oculomotor vermis (OMV)] in the control of visually guided saccades and smooth-pursuit eye movements. Both types of eye movements require the mapping of retinal information onto motor vectors, a transformation that is optimized by the OMV, considering information on past performance. Unlike the role of the OMV in the guidance of eye movements, the contribution of the adjoining vermal cortex to visual motion perception is nonmotor and involves a cerebellar influence on information processing in the cerebral cortex.

Neurons of the inferior olive respond to broad classes of sensory input while subject to homeostatic control

KEY POINTS

Purkinje cells in the cerebellum integrate input from sensory organs with that from premotor centres. Purkinje cells use a variety of sensory inputs relaying information from the environment to modify motor control. Here we investigated to what extent the climbing fibre inputs to Purkinje cells signal mono- or multi-sensory information, and to what extent this signalling is subject to recent history of activity. We show that individual climbing fibres convey multiple types of sensory information, together providing a rich mosaic projection pattern of sensory signals across the cerebellar cortex. Moreover, firing probability of climbing fibres following sensory stimulation depends strongly on the recent history of activity, showing a tendency to homeostatic dampening.

ABSTRACT

Cerebellar Purkinje cells integrate sensory information with motor efference copies to adapt movements to behavioural and environmental requirements. They produce complex spikes that are triggered by the activity of climbing fibres originating in neurons of the inferior olive. These complex spikes can shape the onset, amplitude and direction of movements and the adaptation of such movements to sensory feedback. Clusters of nearby inferior olive neurons project to parasagittally aligned stripes of Purkinje cells, referred to as 'microzones'. It is currently unclear to what extent individual Purkinje cells within a single microzone integrate climbing fibre inputs from multiple sources of different sensory origins, and to what extent sensory-evoked climbing fibre responses depend on the strength and recent history of activation. Here we imaged complex spike responses in cerebellar lobule crus 1 to various types of sensory stimulation in awake mice. We find that different sensory modalities and receptive fields have a mild, but consistent, tendency to converge on individual Purkinje cells, with climbing fibres showing some degree of input-specificity. Purkinje cells encoding the same stimulus show increased events with coherent complex spike firing and tend to lie close together. Moreover, whereas complex spike firing is only mildly affected by variations in stimulus strength, it depends strongly on the recent history of climbing fibre activity. Our data point towards a mechanism in the olivo-cerebellar system that regulates complex spike firing during mono- or multi-sensory stimulation around a relatively low set-point, highlighting an integrative coding scheme of complex spike firing under homeostatic control.

Control of a Humanoid NAO Robot by an Adaptive Bioinspired Cerebellar Module in 3D Motion Tasks

A bioinspired adaptive model, developed by means of a spiking neural network made of thousands of artificial neurons, has been leveraged to control a humanoid NAO robot in real time. The learning properties of the system have been challenged in a classic cerebellum-driven paradigm, a perturbed upper limb reaching protocol. The neurophysiological principles used to develop the model succeeded in driving an adaptive motor control protocol with baseline, acquisition, and extinction phases. The spiking neural network model showed learning behaviours similar to the ones experimentally measured with human subjects in the same task in the acquisition phase, while resorted to other strategies in the extinction phase. The model processed in real-time external inputs, encoded as spikes, and the generated spiking activity of its output neurons was decoded, in order to provide the proper correction on the motor actuators. Three bidirectional long-term plasticity rules have been embedded for different connections and with different time scales. The plasticities shaped the firing activity of the output layer neurons of the network. In the perturbed upper limb reaching protocol, the neurorobot successfully learned how to compensate for the external perturbation generating an appropriate correction. Therefore, the spiking cerebellar model was able to reproduce in the robotic platform how biological systems deal with external sources of error, in both ideal and real (noisy) environments.

Learning from the past: A reverberation of past errors in the cerebellar climbing fiber signal

The cerebellum allows us to rapidly adjust motor behavior to the needs of the situation. It is commonly assumed that cerebellum-based motor learning is guided by the difference between the desired and the actual behavior, i.e., by error information. Not only immediate but also future behavior will benefit from an error because it induces lasting changes of parallel fiber synapses on Purkinje cells (PCs), whose output mediates the behavioral adjustments. Olivary climbing fibers, likewise connecting with PCs, are thought to transport information on instant errors needed for the synaptic modification yet not to contribute to error memory. Here, we report work on monkeys tested in a saccadic learning paradigm that challenges this concept. We demonstrate not only a clear complex spikes (CS) signature of the error at the time of its occurrence but also a reverberation of this signature much later, before a new manifestation of the behavior, suitable to improve it.