A model of Pavlovian eyelid conditioning based on the synaptic organization of the cerebellum

Discrimination training affects stimulus generalization in mice during Pavlovian eyeblink conditioning

The delicate balance between discrimination and generalization of responses is crucial for survival in our ever-changing environment. In particular, it is important to understand how stimulus discrimination affects the level of stimulus generalization. For example, when we use non-differential training for Pavlovian eyeblink conditioning to investigate generalization of cerebellar-related eyelid motor responses, we find generalization effects on amount, amplitude and timing of the conditioned responses. However, it is unknown what the generalization effects are following differential training. We trained mice to close their eyelids to a 10 kHz tone with an air-puff as the reinforcing stimulus (CS+), while alternatingly exposing them to a tone frequency of either 4 kHz, 9 kHz or 9.5 kHz without the air-puff (CS-) during the training blocks. We tested the generalization effects during the expression of the responses after the training period with tones ranging from 2 kHz to 20 kHz. Our results show that the level of generalization tended to positively correlate with the difference between the CS+ and the CS- training stimuli. These effects of generalization were found for the probability, amplitude but not for the timing of the conditioned eyelid responses. These data indicate the specificity of the generalization effects following differential versus non-differential training, highlighting the relevance of discrimination learning for stimulus generalization.

Climbing fibers provide essential instructive signals for associative learning

Supervised learning depends on instructive signals that shape the output of neural circuits to support learned changes in behavior. Climbing fiber (CF) inputs to the cerebellar cortex represent one of the strongest candidates in the vertebrate brain for conveying neural instructive signals. However, recent studies have shown that Purkinje cell stimulation can also drive cerebellar learning and the relative importance of these two neuron types in providing instructive signals for cerebellum-dependent behaviors remains unresolved. In the present study we used cell-type-specific perturbations of various cerebellar circuit elements to systematically evaluate their contributions to delay eyeblink conditioning in mice. Our findings reveal that, although optogenetic stimulation of either CFs or Purkinje cells can drive learning under some conditions, even subtle reductions in CF signaling completely block learning to natural stimuli. We conclude that CFs and corresponding Purkinje cell complex spike events provide essential instructive signals for associative cerebellar learning.

Local synaptic inhibition mediates cerebellar granule cell pattern separation and enables learned sensorimotor associations

The cerebellar cortex has a key role in generating predictive sensorimotor associations. To do so, the granule cell layer is thought to establish unique sensorimotor representations for learning. However, how this is achieved and how granule cell population responses contribute to behavior have remained unclear. To address these questions, we have used in vivo calcium imaging and granule cell-specific pharmacological manipulation of synaptic inhibition in awake, behaving mice. These experiments indicate that inhibition sparsens and thresholds sensory responses, limiting overlap between sensory ensembles and preventing spiking in many granule cells that receive excitatory input. Moreover, inhibition can be recruited in a stimulus-specific manner to powerfully decorrelate multisensory ensembles. Consistent with these results, granule cell inhibition is required for accurate cerebellum-dependent sensorimotor behavior. These data thus reveal key mechanisms for granule cell layer pattern separation beyond those envisioned by classical models.

Neural Sequences and the Encoding of Time

Converging experimental and computational evidence indicate that on the scale of seconds the brain encodes time through changing patterns of neural activity. Experimentally, two general forms of neural dynamic regimes that can encode time have been observed: neural population clocks and ramping activity. Neural population clocks provide a high-dimensional code to generate complex spatiotemporal output patterns, in which each neuron exhibits a nonlinear temporal profile. A prototypical example of neural population clocks are neural sequences, which have been observed across species, brain areas, and behavioral paradigms. Additionally, neural sequences emerge in artificial neural networks trained to solve time-dependent tasks. Here, we examine the role of neural sequences in the encoding of time, and how they may emerge in a biologically plausible manner. We conclude that neural sequences may represent a canonical computational regime to perform temporal computations.

Hypothermia increases cold-inducible protein expression and improves cerebellar-dependent learning after hypoxia ischemia in the neonatal rat

BACKGROUND

Hypoxic ischemic encephalopathy remains a significant cause of developmental disability.1,2 The standard of care for term infants is hypothermia, which has multifactorial effects.3-5 Therapeutic hypothermia upregulates the cold-inducible protein RNA binding motif 3 (RBM3) that is highly expressed in developing and proliferative regions of the brain.6,7 The neuroprotective effects of RBM3 in adults are mediated by its ability to promote the translation of mRNAs such as reticulon 3 (RTN3).8 METHODS: Hypoxia ischemia or control procedure was conducted in Sprague Dawley rat pups on postnatal day 10 (PND10). Pups were immediately assigned to normothermia or hypothermia at the end of the hypoxia. In adulthood, cerebellum-dependent learning was tested using the conditioned eyeblink reflex. The volume of the cerebellum and the magnitude of cerebral injury were measured. A second study quantified RBM3 and RTN3 protein levels in the cerebellum and hippocampus collected during hypothermia.

RESULTS

Hypothermia reduced cerebral tissue loss and protected cerebellar volume. Hypothermia also improved learning of the conditioned eyeblink response. RBM3 and RTN3 protein expression were increased in the cerebellum and hippocampus of rat pups subjected to hypothermia on PND10.

CONCLUSIONS

Hypothermia was neuroprotective in male and female pups and reversed subtle changes in the cerebellum after hypoxic ischemic.

IMPACT

Hypoxic ischemic produced tissue loss and a learning deficit in the cerebellum. Hypothermia reversed both the tissue loss and learning deficit. Hypothermia increased cold-responsive protein expression in the cerebellum and hippocampus. Our results confirm cerebellar volume loss contralateral to the carotid artery ligation and injured cerebral hemisphere, suggesting crossed-cerebellar diaschisis in this model. Understanding the endogenous response to hypothermia might improve adjuvant interventions and expand the clinical utility of this intervention.

Discrimination and learning of temporal input sequences in a cerebellar Purkinje cell model

Introduction

Temporal information processing is essential for sequential contraction of various muscles with the appropriate timing and amplitude for fast and smooth motor control. These functions depend on dynamics of neural circuits, which consist of simple neurons that accumulate incoming spikes and emit other spikes. However, recent studies indicate that individual neurons can perform complex information processing through the nonlinear dynamics of dendrites with complex shapes and ion channels. Although we have extensive evidence that cerebellar circuits play a vital role in motor control, studies investigating the computational ability of single Purkinje cells are few.

Methods

We found, through computer simulations, that a Purkinje cell can discriminate a series of pulses in two directions (from dendrite tip to soma, and from soma to dendrite), as cortical pyramidal cells do. Such direction sensitivity was observed in whatever compartment types of dendrites (spiny, smooth, and main), although they have dierent sets of ion channels.

Results

We found that the shortest and longest discriminable sequences lasted for 60 ms (6 pulses with 10 ms interval) and 4,000 ms (20 pulses with 200 ms interval), respectively. and that the ratio of discriminable sequences within the region of the interesting parameter space was, on average, 3.3% (spiny), 3.2% (smooth), and 1.0% (main). For the direction sensitivity, a T-type Ca²⁺ channel was necessary, in contrast with cortical pyramidal cells that have N-methyl-D-aspartate receptors (NMDARs). Furthermore, we tested whether the stimulus direction can be reversed by learning, specifically by simulated long-term depression, and obtained positive results.

Discussion

Our results show that individual Purkinje cells can perform more complex information processing than is conventionally assumed for a single neuron, and suggest that Purkinje cells act as sequence discriminators, a useful role in motor control and learning.

An emergent temporal basis set robustly supports cerebellar time-series learning

The cerebellum is considered a "learning machine" essential for time interval estimation underlying motor coordination and other behaviors. Theoretical work has proposed that the cerebellum's input recipient structure, the granule cell layer (GCL), performs pattern separation of inputs that facilitates learning in Purkinje cells (P-cells). However, the relationship between input reformatting and learning has remained debated, with roles emphasized for pattern separation features from sparsification to decorrelation. We took a novel approach by training a minimalist model of the cerebellar cortex to learn complex time-series data from time-varying inputs, typical during movements. The model robustly produced temporal basis sets from these inputs, and the resultant GCL output supported better learning of temporally complex target functions than mossy fibers alone. Learning was optimized at intermediate threshold levels, supporting relatively dense granule cell activity, yet the key statistical features in GCL population activity that drove learning differed from those seen previously for classification tasks. These findings advance testable hypotheses for mechanisms of temporal basis set formation and predict that moderately dense population activity optimizes learning.NEW & NOTEWORTHY During movement, mossy fiber inputs to the cerebellum relay time-varying information with strong intrinsic relationships to ongoing movement. Are such mossy fibers signals sufficient to support Purkinje signals and learning? In a model, we show how the GCL greatly improves Purkinje learning of complex, temporally dynamic signals relative to mossy fibers alone. Learning-optimized GCL population activity was moderately dense, which retained intrinsic input variance while also performing pattern separation.

Stimulus Generalization in Mice during Pavlovian Eyeblink Conditioning

Here, we investigate stimulus generalization in a cerebellar learning paradigm, called eyeblink conditioning. Mice were conditioned to close their eyes in response to a 10-kHz tone by repeatedly pairing this tone with an air puff to the eye 250 ms after tone onset. After 10 consecutive days of training, when mice showed reliable conditioned eyelid responses to the 10-kHz tone, we started to expose them to tones with other frequencies, ranging from 2 to 20 kHz. We found that mice had a strong generalization gradient, whereby the probability and amplitude of conditioned eyelid responses gradually decreases depending on the dissimilarity with the 10-kHz tone. Tones with frequencies closest to 10 kHz evoked the most and largest conditioned eyelid responses and each step away from the 10-kHz tone resulted in fewer and smaller conditioned responses (CRs). In addition, we found that tones with lower frequencies resulted in CRs that peaked earlier after tone onset compared with those to tones with higher frequencies. Together, our data show prominent generalization patterns in cerebellar learning. Since the known function of cerebellum is rapidly expanding from pure motor control to domains that include cognition, reward-learning, fear-learning, social function, and even addiction, our data imply generalization controlled by cerebellum in all these domains.

Feedback inhibition underlies new computational functions of cerebellar interneurons

The function of a feedback inhibitory circuit between cerebellar Purkinje cells and molecular layer interneurons (MLIs) was defined by combining optogenetics, neuronal activity recordings both in cerebellar slices and in vivo, and computational modeling. Purkinje cells inhibit a subset of MLIs in the inner third of the molecular layer. This inhibition is non-reciprocal, short-range (less than 200 μm) and is based on convergence of one to two Purkinje cells onto MLIs. During learning-related eyelid movements in vivo, the activity of a subset of MLIs progressively increases as Purkinje cell activity decreases, with Purkinje cells usually leading the MLIs. Computer simulations indicate that these relationships are best explained by the feedback circuit from Purkinje cells to MLIs and that this feedback circuit plays a central role in making cerebellar learning efficient.

Perception of Happening: How the Brain Deals with the No-History Problem

In physics, the temporal dimension has units of infinitesimally brief duration. Given this, how is it possible to perceive things, such as motion, music, and vibrotactile stimulation, that involve extension across many units of time? To address this problem, it is proposed that there is what is termed an "information construct of happening" (ICOH), a simultaneous representation of recent, temporally differentiated perceptual information on the millisecond time scale. The main features of the ICOH are (i) time marking, semantic labeling of all information in the ICOH with ordinal temporal information and distance from what is informationally identified as the present moment, (ii) vector informational features that specify kind, direction, and rate of change for every feature in a percept, and (iii) connectives, information relating vector informational features at adjacent temporal locations in the ICOH. The ICOH integrates products of perceptual processing with recent historical information in sensory memory on the subsecond time scale. Perceptual information about happening in informational sensory memory is encoded in semantic form that preserves connected semantic trails of vector and timing information. The basic properties of the ICOH must be supported by a general and widespread timing mechanism that generates ordinal and interval timing information and it is suggested that state-dependent networks may suffice for that purpose. Happening, therefore, is perceived at a moment and is constituted by an information structure of connected recent historical information.

Influence of various temporal recoding on pavlovian eyeblink conditioning in the cerebellum

We consider the Pavlovian eyeblink conditioning (EBC) via repeated presentation of paired conditioned stimulus (tone) and unconditioned stimulus (US; airpuff). In an effective cerebellar ring network, we change the connection probability p c from Golgi to granule (GR) cells, and make a dynamical classification of various firing patterns of the GR cells. Individual GR cells are thus found to show various well- and ill-matched firing patterns relative to the US timing signal. Then, these variously-recoded signals are fed into the Purkinje cells (PCs) through the parallel-fibers (PFs). Based on such unique dynamical classification of various firing patterns, we make intensive investigations on the influence of various temporal recoding (i.e., firing patterns) of the GR cells on the synaptic plasticity of the PF-PC synapses and the subsequent learning process for the EBC. We first note that the variously-recoded PF signals are effectively depressed by the (error-teaching) instructor climbing-fiber (CF) signals from the inferior olive neuron. In the case of well-matched PF signals, they are strongly depressed through strong long-term depression (LTD) by the instructor CF signals due to good association between the in-phase PF and the instructor CF signals. On the other hand, practically no LTD occurs for the ill-matched PF signals because most of them have no association with the instructor CF signals. This kind of "effective" depression at the PF-PC synapses coordinates firings of PCs effectively, which then makes effective inhibitory coordination on the cerebellar nucleus neuron [which elicits conditioned response (CR; eyeblink)]. When the learning trial passes a threshold, acquisition of CR begins. In this case, the timing degree T d of CR becomes good due to presence of the ill-matched firing group which plays a role of protection barrier for the timing. With further increase in the number of trials, strength S of CR (corresponding to the amplitude of eyelid closure) increases due to strong LTD in the well-matched firing group, while its timing degree T d decreases. In this way, the well- and the ill-matched firing groups play their own roles for the strength and the timing of CR, respectively. Thus, with increasing the number of learning trials, the (overall) learning efficiency degree L e (taking into consideration both timing and strength of CR) for the CR is increased, and eventually it becomes saturated. Finally, we also discuss dependence of the variety degree for firing patterns of the GR cells and the saturated learning efficiency degree L e of the CR on p c and their relations.

Human-scale Brain Simulation via Supercomputer: A Case Study on the Cerebellum

Performance of supercomputers has been steadily and exponentially increasing for the past 20 years, and is expected to increase further. This unprecedented computational power enables us to build and simulate large-scale neural network models composed of tens of billions of neurons and tens of trillions of synapses with detailed anatomical connections and realistic physiological parameters. Such "human-scale" brain simulation could be considered a milestone in computational neuroscience and even in general neuroscience. Towards this milestone, it is mandatory to introduce modern high-performance computing technology into neuroscience research. In this article, we provide an introductory landscape about large-scale brain simulation on supercomputers from the viewpoints of computational neuroscience and modern high-performance computing technology for specialists in experimental as well as computational neurosciences. This introduction to modeling and simulation methods is followed by a review of various representative large-scale simulation studies conducted to date. Then, we direct our attention to the cerebellum, with a review of more simulation studies specific to that region. Furthermore, we present recent simulation results of a human-scale cerebellar network model composed of 86 billion neurons on the Japanese flagship supercomputer K (now retired). Finally, we discuss the necessity and importance of human-scale brain simulation, and suggest future directions of such large-scale brain simulation research.

Effect of diverse recoding of granule cells on optokinetic response in a cerebellar ring network with synaptic plasticity

We consider a cerebellar ring network for the optokinetic response (OKR), and investigate the effect of diverse recoding of granule (GR) cells on OKR by varying the connection probability p_c from Golgi to GR cells. For an optimal value of p_c^∗(=0.06), individual GR cells exhibit diverse spiking patterns which are in-phase, anti-phase, or complex out-of-phase with respect to their population-averaged firing activity. Then, these diversely-recoded signals via parallel fibers (PFs) from GR cells are effectively depressed by the error-teaching signals via climbing fibers from the inferior olive which are also in-phase ones. Synaptic weights at in-phase PF-Purkinje cell (PC) synapses of active GR cells are strongly depressed via strong long-term depression (LTD), while those at anti-phase and complex out-of-phase PF-PC synapses are weakly depressed through weak LTD. This kind of "effective" depression (i.e., strong/weak LTD) at the PF-PC synapses causes a big modulation in firings of PCs, which then exert effective inhibitory coordination on the vestibular nucleus (VN) neuron (which evokes OKR). For the firing of the VN neuron, the learning gain degree L_g, corresponding to the modulation gain ratio, increases with increasing the learning cycle, and it saturates at about the 300th cycle. By varying p_c from p_c^∗, we find that a plot of saturated learning gain degree L_g^∗ versus p_c forms a bell-shaped curve with a peak at p_c^∗ (where the diversity degree in spiking patterns of GR cells is also maximum). Consequently, the more diverse in recoding of GR cells, the more effective in motor learning for the OKR adaptation.

Dynamic changes of timing precision in timed actions during a behavioural task in guinea pigs

Temporal precision is a determinant of performance in various motor activities. Although the accuracy and precision of timing in activities have been previously measured and quantified, temporal dynamics with flexible precision have not been considered. Here, we examined the temporal dynamics in timed motor activities (timed actions) using a guinea pig model in a behavioural task requiring an animal to control action timing to obtain a water reward. In well-trained animals, momentary variations in timing precision were extracted from the temporal distribution of the timed actions measured over daily 12-h sessions. The resampling of the observed time of action in each session demonstrated significant changes of timing precision within a session. Periods with higher timing precision appeared indiscriminately during the same session, and such periods lasted ~ 20 min on average. We conclude that the timing precision in trained actions is flexible and changes dynamically in guinea pigs. By elucidating the brain mechanisms involved in flexibility and dynamics with an animal model, future studies should establish more effective methods to actively enhance timing precision in our motor activities, such as sports.

A cerebello-olivary signal for negative prediction error is sufficient to cause extinction of associative motor learning

The brain generates negative prediction error (NPE) signals to trigger extinction, a type of inhibitory learning that is responsible for suppressing learned behaviors when they are no longer useful. Neurons encoding NPE have been reported in multiple brain regions. Here, we use an optogenetic approach to demonstrate that GABAergic cerebello-olivary neurons can generate a powerful NPE signal, capable of causing extinction of conditioned motor responses on its own.

Population coding in the cerebellum: a machine learning perspective

The cere resembles a feedforward, three-layer network of neurons in which the "hidden layer" consists of Purkinje cells (P-cells) and the output layer consists of deep cerebellar nucleus (DCN) neurons. In this analogy, the output of each DCN neuron is a prediction that is compared with the actual observation, resulting in an error signal that originates in the inferior olive. Efficient learning requires that the error signal reach the DCN neurons, as well as the P-cells that project onto them. However, this basic rule of learning is violated in the cerebellum: the olivary projections to the DCN are weak, particularly in adulthood. Instead, an extraordinarily strong signal is sent from the olive to the P-cells, producing complex spikes. Curiously, P-cells are grouped into small populations that converge onto single DCN neurons. Why are the P-cells organized in this way, and what is the membership criterion of each population? Here, I apply elementary mathematics from machine learning and consider the fact that P-cells that form a population exhibit a special property: they can synchronize their complex spikes, which in turn suppress activity of DCN neuron they project to. Thus complex spikes cannot only act as a teaching signal for a P-cell, but through complex spike synchrony, a P-cell population may act as a surrogate teacher for the DCN neuron that produced the erroneous output. It appears that grouping of P-cells into small populations that share a preference for error satisfies a critical requirement of efficient learning: providing error information to the output layer neuron (DCN) that was responsible for the error, as well as the hidden layer neurons (P-cells) that contributed to it. This population coding may account for several remarkable features of behavior during learning, including multiple timescales, protection from erasure, and spontaneous recovery of memory.

Impairment of cerebellar long-term depression and GABAergic transmission in prion protein deficient mice ectopically expressing PrPLP/Dpl

Prion protein (PrP^C) knockout mice, named as the “Ngsk” strain (Ngsk Prnp^0/0 mice), show late-onset cerebellar Purkinje cell (PC) degeneration because of ectopic overexpression of PrP^C-like protein (PrPLP/Dpl). Our previous study indicated that the mutant mice also exhibited alterations in cerebellum-dependent delay eyeblink conditioning, even at a young age (16 weeks of age) when neurological changes had not occurred. Thus, this electrophysiological study was designed to examine the synaptic function of the cerebellar cortex in juvenile Ngsk Prnp^0/0 mice. We showed that Ngsk Prnp^0/0 mice exhibited normal paired-pulse facilitation but impaired long-term depression of excitatory synaptic transmission at synapses between parallel fibres and PCs. GABA_A-mediated inhibitory postsynaptic currents recorded from PCs were also weakened in Ngsk Prnp^0/0 mice. Furthermore, we confirmed that Ngsk Prnp^0/0 mice (7–8-week-old) exhibited abnormalities in delay eyeblink conditioning. Our findings suggest that these alterations in both excitatory and inhibitory synaptic transmission to PCs caused deficits in delay eyeblink conditioning of Ngsk Prnp^0/0 mice. Therefore, the Ngsk Prnp^0/0 mouse model can contribute to study underlying mechanisms for impairments of synaptic transmission and neural plasticity, and cognitive deficits in the central nervous system.

The Rules of Cerebellar Learning: Around the Ito Hypothesis

As a tribute to Masao Ito, we propose a model of cerebellar learning that incorporates and extends his original model. We suggest four principles that align well with conclusions from multiple cerebellar learning systems. (1) Climbing fiber inputs to the cerebellum drive early, fast, poorly-retained learning in the parallel fiber to Purkinje cell synapse. (2) Learned Purkinje cell outputs drive late, slow, well-retained learning in non-Purkinje cell inputs to neurons in the cerebellar nucleus, transferring learning from the cortex to the nucleus. (3) Recurrent feedback from Purkinje cells to the inferior olive, through interneurons in the cerebellar nucleus, limits the magnitude of fast, early learning in the cerebellar cortex. (4) Functionally different inputs are subjected to plasticity in the cerebellar cortex versus the cerebellar nucleus. A computational neural circuit model that is based on these principles mimics a large amount of neural and behavioral data obtained from the smooth pursuit eye movements of monkeys.

Real-time cerebellar neuroprosthetic system based on a spiking neural network model of motor learning

OBJECTIVE

Damage to the brain, as a result of various medical conditions, impacts the everyday life of patients and there is still no complete cure to neurological disorders. Neuroprostheses that can functionally replace the damaged neural circuit have recently emerged as a possible solution to these problems. Here we describe the development of a real-time cerebellar neuroprosthetic system to substitute neural function in cerebellar circuitry for learning delay eyeblink conditioning (DEC).

APPROACH

The system was empowered by a biologically realistic spiking neural network (SNN) model of the cerebellar neural circuit, which considers the neuronal population and anatomical connectivity of the network. The model simulated synaptic plasticity critical for learning DEC. This SNN model was carefully implemented on a field programmable gate array (FPGA) platform for real-time simulation. This hardware system was interfaced in in vivo experiments with anesthetized rats and it used neural spikes recorded online from the animal to learn and trigger conditioned eyeblink in the animal during training.

MAIN RESULTS

This rat-FPGA hybrid system was able to process neuronal spikes in real-time with an embedded cerebellum model of ~10 000 neurons and reproduce learning of DEC with different inter-stimulus intervals. Our results validated that the system performance is physiologically relevant at both the neural (firing pattern) and behavioral (eyeblink pattern) levels.

SIGNIFICANCE

This integrated system provides the sufficient computation power for mimicking the cerebellar circuit in real-time. The system interacts with the biological system naturally at the spike level and can be generalized for including other neural components (neuron types and plasticity) and neural functions for potential neuroprosthetic applications.

Intracerebellar cannabinoid administration impairs delay but not trace eyeblink conditioning

Intracerebellar administration of cannabinoid agonists impairs cerebellum-dependent delay eyeblink conditioning (EBC) in rats. It is not known whether the cannabinoid-induced impairment in EBC is found with shorter interstimulus intervals (ISI), longer ISIs, or with trace EBC. Moreover, systemic administration of cannabinoid agonists does not impair trace EBC, suggesting that cannabinoid receptors within the cerebellum are not involved in trace EBC. To more precisely assess the effects of cannabinoids on cerebellar learning mechanisms the current study examined the effects of the cannabinoid agonist WIN55,212-2 (WIN) infusion into the area of the cerebellar cortex necessary for EBC (the eyeblink microzone) in rats during short delay (250 ms CS), long delay (750 ms CS), and trace (250 ms CS, 500 ms trace interval) EBC. WIN was infused into the eyeblink microzone 30 min before pretraining sessions and five EBC training sessions, followed by five EBC training sessions without infusions to assess recovery from drug effects and savings. WIN had no effect on spontaneous blinks or non-associative responses to the CS or US during the pretraining sessions. Short and long delay EBC were impaired by WIN but trace EBC was unaffected. The results indicate that trace EBC is mediated by mechanisms that are resistant to cannabinoid agonists.

A Biomimetic Control Method Increases the Adaptability of a Humanoid Robot Acting in a Dynamic Environment

One of the big challenges in robotics is to endow agents with autonomous and adaptive capabilities. With this purpose, we embedded a cerebellum-based control system into a humanoid robot that becomes capable of handling dynamical external and internal complexity. The cerebellum is the area of the brain that coordinates and predicts the body movements throughout the body-environment interactions. Different biologically plausible cerebellar models are available in literature and have been employed for motor learning and control of simplified objects. We built the canonical cerebellar microcircuit by combining machine learning and computational neuroscience techniques. The control system is composed of the adaptive cerebellar module and a classic control method; their combination allows a fast adaptive learning and robust control of the robotic movements when external disturbances appear. The control structure is built offline, but the dynamic parameters are learned during an online-phase training. The aforementioned adaptive control system has been tested in the Neuro-robotics Platform with the virtual humanoid robot iCub. In the experiment, the robot iCub has to balance with the hand a table with a ball running on it. In contrast with previous attempts of solving this task, the proposed neural controller resulted able to quickly adapt when the internal and external conditions change. Our bio-inspired and flexible control architecture can be applied to different robotic configurations without an excessive tuning of the parameters or customization. The cerebellum-based control system is indeed able to deal with changing dynamics and interactions with the environment. Important insights regarding the relationship between the bio-inspired control system functioning and the complexity of the task to be performed are obtained.

The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions

Timing is critical to most forms of learning, behavior, and sensory-motor processing. Converging evidence supports the notion that, precisely because of its importance across a wide range of brain functions, timing relies on intrinsic and general properties of neurons and neural circuits; that is, the brain uses its natural cellular and network dynamics to solve a diversity of temporal computations. Many circuits have been shown to encode elapsed time in dynamically changing patterns of neural activity-so-called population clocks. But temporal processing encompasses a wide range of different computations, and just as there are different circuits and mechanisms underlying computations about space, there are a multitude of circuits and mechanisms underlying the ability to tell time and generate temporal patterns.

Distinct Dopamine Receptor Pathways Underlie the Temporal Sensitivity of Associative Learning

Animals rely on the relative timing of events in their environment to form and update predictive associations, but the molecular and circuit mechanisms for this temporal sensitivity remain incompletely understood. Here, we show that olfactory associations in Drosophila can be written and reversed on a trial-by-trial basis depending on the temporal relationship between an odor cue and dopaminergic reinforcement. Through the synchronous recording of neural activity and behavior, we show that reversals in learned odor attraction correlate with bidirectional neural plasticity in the mushroom body, the associative olfactory center of the fly. Two dopamine receptors, DopR1 and DopR2, contribute to this temporal sensitivity by coupling to distinct second messengers and directing either synaptic depression or potentiation. Our results reveal how dopamine-receptor signaling pathways can detect the order of events to instruct opposing forms of synaptic and behavioral plasticity, allowing animals to flexibly update their associations in a dynamic environment.

Revisiting a theory of cerebellar cortex

Long-term depression at parallel fiber-Purkinje cell synapses plays a principal role in learning in the cerebellum, which acts as a supervised learning machine. Recent experiments demonstrate various forms of synaptic plasticity at different sites within the cerebellum. In this article, we take into consideration synaptic plasticity at parallel fiber-molecular layer interneuron synapses as well as at parallel fiber-Purkinje cell synapses, and propose that the cerebellar cortex performs reinforcement learning, another form of learning that is more capable than supervised learning. We posit that through the use of reinforcement learning, the need for explicit teacher signals for learning in the cerebellum is eliminated; instead, learning can occur via responses from evaluative feedback. We demonstrate the learning capacity of cerebellar reinforcement learning using simple computer simulations of delay eyeblink conditioning and the cart-pole balancing task.

No neuron is an island: Homeostatic plasticity and over-constraint in a neural circuit

To support computation the activity of neurons must vary within a useful range, which highlights one potential value of homeostatic plasticity. The interconnectedness of the brain, however, introduces the possibility that combinations of homeostatic mechanisms can produce over-constraint in which not all set points can be satisfied. We use a simulation of the cerebellum to investigate the potential for such conflict and its consequences. In this instance the conflict produces perpetual drift and eventual saturation of synaptic weights. We show that these problems can be resolved for this network by a particular combination of sites and rules for plasticity. We also show that simulations that implement these rules for homeostatic plasticity are more resistant to forgetting. These results illustrate the general principle that homeostatic plasticity within a system must not set up conflicts in which mutually exclusive set points exist and that one consequence can be perpetual induction of plasticity.

Medial Auditory Thalamus Is Necessary for Expression of Auditory Trace Eyelid Conditioning

Transforming a brief sensory event into a persistent neural response represents a mechanism for linking temporally disparate stimuli together to support learning. The cerebellum requires this type of persistent input during trace conditioning to engage associative plasticity and acquire adaptively timed conditioned responses (CRs). An initial step toward identifying the sites and mechanisms generating and transmitting persistent signals to the cerebellum is to identify the input pathway. The medial auditory thalamic nuclei (MATN) are the necessary and sufficient source of auditory input to the cerebellum for delay conditioning in rodents and a possible input to forebrain sites generating persistent signals. Using pharmacological and computational approaches, we test (1) whether the necessity of MATN during auditory eyelid conditioning is conserved across species, (2) whether the MATN are necessary for the expression of trace eyelid CRs, and if so, (3)whether this relates to the generation of persistent signals. We find that contralateral inactivation of MATN with muscimol largely abolished trace and delay CRs in male rabbits. Residual CRs were decreased in amplitude, but CR timing was unaffected. Results from large-scale cerebellar simulations are consistent with previous experimental demonstrations that silencing only CS-duration inputs does not abolish trace CRs, and instead affects their timing. Together, these results suggest that the MATN are a necessary component of both the direct auditory stimulus pathway to the cerebellum and the pathway generating task-essential persistent signals.SIGNIFICANCE STATEMENT Persistent activity is required for working memory-dependent tasks, such as trace conditioning, and represents a mechanism by which sensory information can be used over time for learning and cognition. This neuronal response entails the transformation of a discrete sensory-evoked response into a signal that extends beyond the stimulus event. Understanding the generation and transmission of this stimulus transformation requires identifying the input sources necessary for task-essential persistent signals. We report that the medial auditory thalamic nuclei are required for the expression of auditory trace conditioning and suggest that these nuclei are a component of the pathway-generating persistent signals. Our study provides a foundation for testing circuit-level mechanisms underlying persistent activity in a cerebellar learning model with identified inputs and well defined behavioral outputs.

Cannabinoid agonist administration within the cerebellar cortex impairs motor learning

Systemic administration of cannabinoid agonists impairs cerebellum-dependent motor learning. The cannabinoid-induced impairment of motor learning has been hypothesized to be due to disruption of Purkinje cell plasticity within the cerebellar cortex. In the current study, we tested this hypothesis in rats with localized microinfusions of cannabinoid agonists and antagonists into the cerebellar cortex during eyeblink conditioning, a type of cerebellum-dependent motor learning. Infusions of the cannabinoid agonists WIN55,212-2 or ACEA directly into the eyeblink conditioning microzone of the cerebellar cortex severely impaired acquisition of eyeblink conditioning, whereas the CB1R antagonist SR141716A did not produce a significant impairment. Infusions of WIN55,212-2 outside of the eyeblink conditioning microzone did not impair motor learning, establishing anatomical specificity for the agonist effects. The motor learning impairment caused by WIN55,212-2 and ACEA was rescued by SR141716A, indicating that the learning deficit was produced through CB1Rs. The current findings demonstrate that the effects of cannabinoid receptor agonists on motor learning are localized to CB1Rs within a discrete microzone of the cerebellar cortex.

Encoding of error and learning to correct that error by the Purkinje cells of the cerebellum

The primary output cells of the cerebellar cortex, Purkinje cells, make kinematic predictions about ongoing movements via high-frequency simple spikes, but receive sensory error information about that movement via low-frequency complex spikes (CS). How is the vector space of sensory errors encoded by this low-frequency signal? Here we measured Purkinje cell activity in the oculomotor vermis of animals during saccades, then followed the chain of events from experience of visual error, generation of CS, modulation of simple spikes, and ultimately change in motor output. We found that while error direction affected the probability of CS, error magnitude altered its temporal distribution. Production of CS changed the simple spikes on the next trial, but regardless of the actual visual error, this change biased the movement only along a vector that was parallel to the Purkinje cell's preferred error. From these results, we inferred the anatomy of a sensory-to-motor adaptive controller that transformed visual error vectors into motor-corrections.

Encoding Time in Feedforward Trajectories of a Recurrent Neural Network Model

Brain activity evolves through time, creating trajectories of activity that underlie sensorimotor processing, behavior, and learning and memory. Therefore, understanding the temporal nature of neural dynamics is essential to understanding brain function and behavior. In vivo studies have demonstrated that sequential transient activation of neurons can encode time. However, it remains unclear whether these patterns emerge from feedforward network architectures or from recurrent networks and, furthermore, what role network structure plays in timing. We address these issues using a recurrent neural network (RNN) model with distinct populations of excitatory and inhibitory units. Consistent with experimental data, a single RNN could autonomously produce multiple functionally feedforward trajectories, thus potentially encoding multiple timed motor patterns lasting up to several seconds. Importantly, the model accounted for Weber's law, a hallmark of timing behavior. Analysis of network connectivity revealed that efficiency-a measure of network interconnectedness-decreased as the number of stored trajectories increased. Additionally, the balance of excitation (E) and inhibition (I) shifted toward excitation during each unit's activation time, generating the prediction that observed sequential activity relies on dynamic control of the E/I balance. Our results establish for the first time that the same RNN can generate multiple functionally feedforward patterns of activity as a result of dynamic shifts in the E/I balance imposed by the connectome of the RNN. We conclude that recurrent network architectures account for sequential neural activity, as well as for a fundamental signature of timing behavior: Weber's law.

Morphological Constraints on Cerebellar Granule Cell Combinatorial Diversity

Combinatorial expansion by the cerebellar granule cell layer (GCL) is fundamental to theories of cerebellar contributions to motor control and learning. Granule cells (GrCs) sample approximately four mossy fiber inputs and are thought to form a combinatorial code useful for pattern separation and learning. We constructed a spatially realistic model of the cerebellar GCL and examined how GCL architecture contributes to GrC combinatorial diversity. We found that GrC combinatorial diversity saturates quickly as mossy fiber input diversity increases, and that this saturation is in part a consequence of short dendrites, which limit access to diverse inputs and favor dense sampling of local inputs. This local sampling also produced GrCs that were combinatorially redundant, even when input diversity was extremely high. In addition, we found that mossy fiber clustering, which is a common anatomical pattern, also led to increased redundancy of GrC input combinations. We related this redundancy to hypothesized roles of temporal expansion of GrC information encoding in service of learned timing, and we show that GCL architecture produces GrC populations that support both temporal and combinatorial expansion. Finally, we used novel anatomical measurements from mice of either sex to inform modeling of sparse and filopodia-bearing mossy fibers, finding that these circuit features uniquely contribute to enhancing GrC diversification and redundancy. Our results complement information theoretic studies of granule layer structure and provide insight into the contributions of granule layer anatomical features to afferent mixing.SIGNIFICANCE STATEMENT Cerebellar granule cells are among the simplest neurons, with tiny somata and, on average, just four dendrites. These characteristics, along with their dense organization, inspired influential theoretical work on the granule cell layer as a combinatorial expander, where each granule cell represents a unique combination of inputs. Despite the centrality of these theories to cerebellar physiology, the degree of expansion supported by anatomically realistic patterns of inputs is unknown. Using modeling and anatomy, we show that realistic input patterns constrain combinatorial diversity by producing redundant combinations, which nevertheless could support temporal diversification of like combinations, suitable for learned timing. Our study suggests a neural substrate for producing high levels of both combinatorial and temporal diversity in the granule cell layer.

Differential Encoding of Time by Prefrontal and Striatal Network Dynamics

Telling time is fundamental to many forms of learning and behavior, including the anticipation of rewarding events. Although the neural mechanisms underlying timing remain unknown, computational models have proposed that the brain represents time in the dynamics of neural networks. Consistent with this hypothesis, changing patterns of neural activity dynamically in a number of brain areas-including the striatum and cortex-has been shown to encode elapsed time. To date, however, no studies have explicitly quantified and contrasted how well different areas encode time by recording large numbers of units simultaneously from more than one area. Here, we performed large-scale extracellular recordings in the striatum and orbitofrontal cortex of mice that learned the temporal relationship between a stimulus and a reward and reported their response with anticipatory licking. We used a machine-learning algorithm to quantify how well populations of neurons encoded elapsed time from stimulus onset. Both the striatal and cortical networks encoded time, but the striatal network outperformed the orbitofrontal cortex, a finding replicated both in simultaneously and nonsimultaneously recorded corticostriatal datasets. The striatal network was also more reliable in predicting when the animals would lick up to ∼1 s before the actual lick occurred. Our results are consistent with the hypothesis that temporal information is encoded in a widely distributed manner throughout multiple brain areas, but that the striatum may have a privileged role in timing because it has a more accurate "clock" as it integrates information across multiple cortical areas.

SIGNIFICANCE STATEMENT

The neural representation of time is thought to be distributed across multiple functionally specialized brain structures, including the striatum and cortex. However, until now, the neural code for time has not been compared quantitatively between these areas. Here, we performed large-scale recordings in the striatum and orbitofrontal cortex of mice trained on a stimulus-reward association task involving a delay period and used a machine-learning algorithm to quantify how well populations of simultaneously recorded neurons encoded elapsed time from stimulus onset. We found that, although both areas encoded time, the striatum consistently outperformed the orbitofrontal cortex. These results suggest that the striatum may refine the code for time by integrating information from multiple inputs.

Heterogeneity of Purkinje cell simple spike-complex spike interactions: zebrin- and non-zebrin-related variations

KEY POINTS

Cerebellar Purkinje cells (PCs) generate two types of action potentials, simple and complex spikes. Although they are generated by distinct mechanisms, interactions between the two spike types exist. Zebrin staining produces alternating positive and negative stripes of PCs across most of the cerebellar cortex. Thus, here we compared simple spike-complex spike interactions both within and across zebrin populations. Simple spike activity undergoes a complex modulation preceding and following a complex spike. The amplitudes of the pre- and post-complex spike modulation phases were correlated across PCs. On average, the modulation was larger for PCs in zebrin positive regions. Correlations between aspects of the complex spike waveform and simple spike activity were found, some of which varied between zebrin positive and negative PCs. The implications of the results are discussed with regard to hypotheses that complex spikes are triggered by rises in simple spike activity for either motor learning or homeostatic functions.

ABSTRACT

Purkinje cells (PCs) generate two types of action potentials, called simple and complex spikes (SSs and CSs). We first investigated the CS-associated modulation of SS activity and its relationship to the zebrin status of the PC. The modulation pattern consisted of a pre-CS rise in SS activity, and then, following the CS, a pause, a rebound, and finally a late inhibition of SS activity for both zebrin positive (Z+) and negative (Z-) cells, though the amplitudes of the phases were larger in Z+ cells. Moreover, the amplitudes of the pre-CS rise with the late inhibitory phase of the modulation were correlated across PCs. In contrast, correlations between modulation phases across CSs of individual PCs were generally weak. Next, the relationship between CS spikelets and SS activity was investigated. The number of spikelets/CS correlated with the average SS firing rate only for Z+ cells. In contrast, correlations across CSs between spikelet numbers and the amplitudes of the SS modulation phases were generally weak. Division of spikelets into likely axonally propagated and non-propagated groups (based on their interspikelet interval) showed that the correlation of spikelet number with SS firing rate primarily reflected a relationship with non-propagated spikelets. In sum, the results show both zebrin-related and non-zebrin-related physiological heterogeneity in SS-CS interactions among PCs, which suggests that the cerebellar cortex is more functionally diverse than is assumed by standard theories of cerebellar function.

Aberrant connections between climbing fibres and Purkinje cells induce alterations in the timing of an instrumental response in the rat

Cerebellar participation in timing and sensory-motor sequences has been supported by several experimental and clinical studies. A relevant role of the cerebellum in timing of conditioned responses in the range of milliseconds has been demonstrated, but less is known regarding the role of the cerebellum in supra-second timing of operant responses. A dissociated role of the cerebellum and striatum in timing in the millisecond and second range had been reported, respectively. The climbing fibre-Purkinje cell synapse is crucial in timing models; thus, the aberrant connection between these cellular elements is a suitable model for evaluating the contribution of the cerebellum in timing in the supra-second range. The aberrant connection between climbing fibres and Purkinje cells was induced by administration of the antagonist of NMDA receptors MK-801 to Sprague-Dawley rats at postnatal days 7-14. The timing of an operant response with two fixed intervals (5 and 8 s) and egocentric sequential learning was evaluated in 60-day-old adult rats. The aberrant connections caused a reduced accuracy in the timing of the instrumental response that was more evident in the 8-s interval and a reduced number of successive correct responses (responses emitted in the correct second without any other response between them) in the 8-s interval. In addition, an inability to incorporate new information in a sequence previously learned in egocentric-based sequence learning was apparent in rats with aberrant CF-PC synapses. These results support a relevant role for the cerebellum in the fine-tuning of the timing of operant responses in the supra-second range.

Machine Learning Capabilities of a Simulated Cerebellum

This paper describes the learning and control capabilities of a biologically constrained bottom-up model of the mammalian cerebellum. Results are presented from six tasks: 1) eyelid conditioning; 2) pendulum balancing; 3) proportional-integral-derivative control; 4) robot balancing; 5) pattern recognition; and 6) MNIST handwritten digit recognition. These tasks span several paradigms of machine learning, including supervised learning, reinforcement learning, control, and pattern recognition. Results over these six domains indicate that the cerebellar simulation is capable of robustly identifying static input patterns even when randomized across the sensory apparatus. This capability allows the simulated cerebellum to perform several different supervised learning and control tasks. On the other hand, both reinforcement learning and temporal pattern recognition prove problematic due to the delayed nature of error signals and the simulator's inability to solve the credit assignment problem. These results are consistent with previous findings which hypothesize that in the human brain, the basal ganglia is responsible for reinforcement learning, while the cerebellum handles supervised learning.

Links Between Single-Trial Changes and Learning Rate in Eyelid Conditioning

The discovery of single-trial learning effects, where the presence or absence (or the number) of climbing fiber inputs produces measureable changes in Purkinje cell response and in behavior, represents a major breakthrough in cerebellar learning. Among other things, these observations provide strong links between climbing fiber-mediated plasticity and cerebellar learning. They also demonstrate that cerebellar learning is stochastic, with each instantiation of a movement producing a small increment or decrement in gain. The sum of the small changes give rise to the macroscopic properties of cerebellar learning. We used a relatively large data set from another example of cerebellar-dependent learning, classical conditioning of eyelid responses, to attempt a behavioral replication and extension of single-trial learning effects. As a normal part of training, stimulus-alone trials provide instances where the climbing fiber response would be omitted, similar to non-climbing-fiber trials (gain down) during smooth pursuit training. The consequences of the stimulus-alone trial on the amplitude and timing of the conditioned response on the following paired trials were examined. We find that the amplitude of the conditioned response during the trial after a stimulus-alone trial (no climbing fiber input) was measurably smaller than the amplitude on the previous trials, and this single-trial effect on amplitude is larger for longer interstimulus intervals. The magnitude of the single-trial effect parallels the rate of extinction at different interstimulus intervals supporting the previously observed link between single-trial effects and learning.

Cannabinoid modulation of memory consolidation within the cerebellum

Cannabinoid receptors contribute to learning and synaptic plasticity mechanisms. The cerebellum contains a high density of cannabinoid receptors and manipulations of cannabinoid receptors affect synaptic plasticity within the cerebellar cortex. In vivo studies have found that cannabinoid agonists impair learning of cerebellum-dependent eyeblink conditioning in rodents and humans. However, the role of cannabinoid receptors or endocannabinoids in memory consolidation within the cerebellum has not been examined. In the current study, we examined the role of cannabinoid receptors and endocannabinoids during learning and consolidation of eyeblink conditioning in rats. Administration of the cannabinoid receptor agonist WIN55,212-2 or drugs that increase/decrease endocannabinoid levels directly into the cerebellar cortex before each training session resulted in marked learning impairments. When administered 1 h after each training session, during memory consolidation, the cannabinoid inverse agonist SR141716A or the endocannabinoid suppressor THL impaired memory. In contrast, increasing endocannabinoid levels with JZL-184 or infusion of WIN55,212-2 within the cerebellar cortex facilitated memory consolidation 1h post-training. Intracerebellar manipulations of cannabinoid receptors or endocannabinoid levels had no effect on memory consolidation when administered 3 or 6h after each training session. The results demonstrate that cannabinoids impair cerebellar learning, but facilitate memory consolidation mechanisms within the cerebellar cortex 1-3h after training.

Real-Time Simulation of Passage-of-Time Encoding in Cerebellum Using a Scalable FPGA-Based System

The cerebellum plays a critical role for sensorimotor control and learning. However, dysmetria or delays in movements' onsets consequent to damages in cerebellum cannot be cured completely at the moment. Neuroprosthesis is an emerging technology that can potentially substitute such motor control module in the brain. A pre-requisite for this to become practical is the capability to simulate the cerebellum model in real-time, with low timing distortion for proper interfacing with the biological system. In this paper, we present a frame-based network-on-chip (NoC) hardware architecture for implementing a bio-realistic cerebellum model with  ∼ 100 000 neurons, which has been used for studying timing control or passage-of-time (POT) encoding mediated by the cerebellum. The simulation results verify that our implementation reproduces the POT representation by the cerebellum properly. Furthermore, our field-programmable gate array (FPGA)-based system demonstrates excellent computational speed that it can complete 1sec real world activities within 25.6 ms. It is also highly scalable such that it can maintain approximately the same computational speed even if the neuron number increases by one order of magnitude. Our design is shown to outperform three alternative approaches previously used for implementing spiking neural network model. Finally, we show a hardware electronic setup and illustrate how the silicon cerebellum can be adapted as a potential neuroprosthetic platform for future biological or clinical application.

Systematic variation of acquisition rate in delay eyelid conditioning

Averaging artifacts inherent in group acquisition curves can mask behavioral phenomena that are potentially revealing in terms of underlying neural mechanisms. To address this, we implemented a behavioral analysis of 106 rabbits trained over 4 sessions using delay eyelid conditioning. Group results showed the typical monotonic increase in conditioned responses (CRs). For most subjects CRs first appeared (as indexed by the criterion of 8 CRs in 9 trials) during the first 18 trials of the second training session. Subdividing subjects according to the training block at which they met criterion revealed systematic differences in the subsequent rate that CR amplitudes increased, but not in asymptotic CR amplitudes. Subjects meeting criterion early in sessions showed more rapid increases in CR amplitude than those meeting criterion later in sessions. This effect was solely dependent on how early within a session criterion was met, as subjects meeting criterion at the beginning of the third and fourth sessions showed more rapid increases in CR amplitude than those meeting criterion after the first 18 trials of the second session. The exceptions were the 7% of the subjects that met criterion late in the first session. Their CR amplitudes increased at a rate similar to subjects meeting criterion early in sessions. These results suggest an interplay between consolidation processes and a previously reported short-term plasticity process that makes CR acquisition a nonmonotonic and complex function of the point during training sessions at which CRs first appear. (PsycINFO Database Record

Neurocomputational Models of Interval and Pattern Timing

Most of the computations and tasks performed by the brain require the ability to tell time, and process and generate temporal patterns. Thus, there is a diverse set of neural mechanisms in place to allow the brain to tell time across a wide range of scales: from interaural delays on the order of microseconds to circadian rhythms and beyond. Temporal processing is most sophisticated on the scale of tens of milliseconds to a few seconds, because it is within this range that the brain must recognize and produce complex temporal patterns-such as those that characterize speech and music. Most models of timing, however, have focused primarily on simple intervals and durations, thus it is not clear whether they will generalize to complex pattern-based temporal tasks. Here, we review neurobiologically based models of timing in the subsecond range, focusing on whether they generalize to tasks that require placing consecutive intervals in the context of an overall pattern, that is, pattern timing.

Climbing fibers encode a temporal-difference prediction error during cerebellar learning in mice

Climbing fiber inputs to Purkinje cells are thought to play a teaching role by generating the instructive signals that drive cerebellar learning. To investigate how these instructive signals are encoded, we recorded the activity of individual climbing fibers during cerebellar-dependent eyeblink conditioning in mice. Our findings show that climbing fibers signal both the unexpected delivery and the unexpected omission of the periocular airpuff that serves as the instructive signal for eyeblink conditioning. In addition, we report the surprising discovery that climbing fibers activated by periocular airpuffs also respond to stimuli from other sensory modalities, if those stimuli are novel or if they predict that the periocular airpuff is about to be presented. This pattern of climbing fiber activity is strikingly similar to the responses of dopamine neurons during reinforcement learning, which have been shown to encode a particular type of instructive signal known as a temporal difference prediction error.

At the Edge of Chaos: How Cerebellar Granular Layer Network Dynamics Can Provide the Basis for Temporal Filters

Models of the cerebellar microcircuit often assume that input signals from the mossy-fibers are expanded and recoded to provide a foundation from which the Purkinje cells can synthesize output filters to implement specific input-signal transformations. Details of this process are however unclear. While previous work has shown that recurrent granule cell inhibition could in principle generate a wide variety of random outputs suitable for coding signal onsets, the more general application for temporally varying signals has yet to be demonstrated. Here we show for the first time that using a mechanism very similar to reservoir computing enables random neuronal networks in the granule cell layer to provide the necessary signal separation and extension from which Purkinje cells could construct basis filters of various time-constants. The main requirement for this is that the network operates in a state of criticality close to the edge of random chaotic behavior. We further show that the lack of recurrent excitation in the granular layer as commonly required in traditional reservoir networks can be circumvented by considering other inherent granular layer features such as inverted input signals or mGluR2 inhibition of Golgi cells. Other properties that facilitate filter construction are direct mossy fiber excitation of Golgi cells, variability of synaptic weights or input signals and output-feedback via the nucleocortical pathway. Our findings are well supported by previous experimental and theoretical work and will help to bridge the gap between system-level models and detailed models of the granular layer network.

Species-specific differences in the medial prefrontal projections to the pons between rat and rabbit

The medial prefrontal cortex (mPFC) of both rats and rabbits has been shown to support trace eyeblink conditioning, presumably by providing an input to the cerebellum via the pons that bridges the temporal gap between conditioning stimuli. The pons of rats and rabbits, however, shows divergence in gross anatomical organization, leaving open the question of whether the topography of prefrontal inputs to the pons is similar in rats and rabbits. To investigate this question, we injected anterograde tracer into the mPFC of rats and rabbits to visualize and map in 3D the distribution of labeled terminals in the pons. Effective mPFC injections showed labeled axons in the ipsilateral descending pyramidal tract in both species. In rats, discrete clusters of densely labeled terminals were observed primarily in the rostromedial pons. Clusters of labeled terminals were also observed contralateral to mPFC injection sites in rats, appearing as a less dense "mirror-image" of ipsilateral labeling. In rabbits, mPFC labeled corticopontine terminals were absent in the rostral pons, and instead were restricted to the intermediate pons. The densest terminal fields were typically observed in association with the ipsilateral pyramidal tract as it descended ventromedially through the rabbit pons. No contralateral terminal labeling was observed for any injections made in the rabbit mPFC. The results suggest the possibility that mPFC inputs to the pons may be integrated with different sources of cortical inputs between rats and rabbits. The resulting implications for mPFC or pons manipulations for studies of trace eyeblink in each species are discussed.

Relating cerebellar purkinje cell activity to the timing and amplitude of conditioned eyelid responses

How Purkinje cell (PC) activity may be altered by learning is central to theories of the cerebellum. Pavlovian eyelid conditioning, because of how directly it engages the cerebellum, has helped reveal many aspects of cerebellar learning and the underlying mechanisms. Theories of cerebellar learning assert that climbing fiber inputs control plasticity at synapses onto PCs, and thus PCs control the expression of learned responses. We tested this assertion by recording 184 eyelid PCs and 240 non-eyelid PCs during the expression of conditioned eyelid responses (CRs) in well trained rabbits. By contrasting the responses of eyelid and non-eyelid PCs and by contrasting the responses of eyelid PCs under conditions that produce differently timed CRs, we test the hypothesis that learning-related changes in eyelid PCs contribute to the learning and adaptive timing of the CRs. We used a variety of analyses to test the quantitative relationships between eyelid PC responses and the kinematic properties of the eyelid CRs. We find that the timing of eyelid PC responses varies systematically with the timing of the behavioral CRs and that there are differences in the magnitude of eyelid PC responses between larger-CR, smaller-CR, and non-CR trials. However, eyelid PC activity does not encode any single kinematic property of the behavioral CRs at a fixed time lag, nor does it linearly encode CR amplitude. Even so, the results are consistent with the hypothesis that learning-dependent changes in PC activity contribute to the adaptively timed expression of conditioned eyelid responses.

Neural coding: time contraction and dilation in the striatum

How the brain encodes time is poorly understood. New research on rats provides evidence that striatal neurons encode time, and that the code can dilate or contract to time different intervals.

Integrated plasticity at inhibitory and excitatory synapses in the cerebellar circuit

The way long-term potentiation (LTP) and depression (LTD) are integrated within the different synapses of brain neuronal circuits is poorly understood. In order to progress beyond the identification of specific molecular mechanisms, a system in which multiple forms of plasticity can be correlated with large-scale neural processing is required. In this paper we take as an example the cerebellar network, in which extensive investigations have revealed LTP and LTD at several excitatory and inhibitory synapses. Cerebellar LTP and LTD occur in all three main cerebellar subcircuits (granular layer, molecular layer, deep cerebellar nuclei) and correspondingly regulate the function of their three main neurons: granule cells (GrCs), Purkinje cells (PCs) and deep cerebellar nuclear (DCN) cells. All these neurons, in addition to be excited, are reached by feed-forward and feed-back inhibitory connections, in which LTP and LTD may either operate synergistically or homeostatically in order to control information flow through the circuit. Although the investigation of individual synaptic plasticities in vitro is essential to prove their existence and mechanisms, it is insufficient to generate a coherent view of their impact on network functioning in vivo. Recent computational models and cell-specific genetic mutations in mice are shedding light on how plasticity at multiple excitatory and inhibitory synapses might regulate neuronal activities in the cerebellar circuit and contribute to learning and memory and behavioral control.

Circuit mechanisms underlying motor memory formation in the cerebellum

The cerebellum stores associative motor memories essential for properly timed movement; however, the mechanisms by which these memories form and are acted upon remain unclear. To determine how cerebellar activity relates to movement and motor learning, we used optogenetics to manipulate spontaneously firing Purkinje neurons (PNs) in mouse simplex lobe. Using high-speed videography and motion tracking, we found that altering PN activity produced rapid forelimb movement. PN inhibition drove movements time-locked to stimulus onset, whereas PN excitation drove delayed movements time-locked to stimulus offset. Pairing either PN inhibition or excitation with sensory stimuli triggered the formation of robust, associative motor memories; however, PN excitation led to learned movements whose timing more closely matched training intervals. These findings implicate inhibition of PNs as a teaching signal, consistent with a model whereby learning leads first to reductions in PN firing that subsequently instruct circuit changes in the cerebellar nucleus.

Cerebellar cortex and cerebellar nuclei are concomitantly activated during eyeblink conditioning: a 7T fMRI study in humans

There are controversies whether learning of conditioned eyeblink responses primarily takes place within the cerebellar cortex, the interposed nuclei, or both. It has also been suggested that the cerebellar cortex may be important during early stages of learning, and that there is a shift to the cerebellar nuclei during later stages. As yet, human studies have provided little to resolve this question. In the present study, we established a setup that allows ultra-high-field 7T functional magnetic resonance imaging (fMRI) of the cerebellar cortex and interposed cerebellar nuclei simultaneously during delay eyeblink conditioning in humans. Event-related fMRI signals increased concomitantly in the cerebellar cortex and nuclei during early acquisition of conditioned eyeblink responses in 20 healthy human subjects. ANOVAs with repeated-measures showed significant effects of time across five blocks of 20 conditioning trials in the cortex and nuclei (p < 0.05, permutation corrected). Activations were most pronounced in, but not limited to, lobules VI and interposed nuclei. Increased activations were most prominent at the first time the maximum number of conditioned responses was achieved. Our data are consistent with a simultaneous and synergistic two-site model of learning during acquisition of classically conditioned eyeblinks. Because increased MRI signal reflects synaptic activity, concomitantly increased signals in the cerebellar nuclei and cortex are consistent with findings of learning related potentiation at the mossy fiber to nuclear cell synapse and mossy fiber to granule cell synapse. Activity related to the expression of conditioned responses, however, cannot be excluded.

Extinction, reacquisition, and rapid forgetting of eyeblink conditioning in developing rats

Eyeblink conditioning is a well-established model for studying the developmental neurobiology of associative learning and memory. However, age differences in extinction and subsequent reacquisition have yet to be studied using this model. The present study examined extinction and reacquisition of eyeblink conditioning in developing rats. In Experiment 1, post-natal day (P) 17 and 24 rats were trained to a criterion of 80% conditioned responses (CRs) using stimulation of the middle cerebellar peduncle (MCP) as a conditioned stimulus (CS). Stimulation CS-alone extinction training commenced 24 h later, followed by reacquisition training after the fourth extinction session. Contrary to expected results, rats trained starting on P17 showed significantly fewer CRs to stimulation CS-alone presentations relative to P24s, including fewer CRs as early as the first block of extinction session 1. Furthermore, the P17 group was slower to reacquire following extinction. Experiment 2 was run to determine the extent to which the low CR percentage observed in P17s early in extinction reflected rapid forgetting versus rapid extinction. Twenty-four hours after reaching criterion, subjects were trained in a session split into 50 stimulation CS-unconditioned stimulus paired trials followed immediately by 50 stimulation CS-alone trials. With this "immediate" extinction protocol, CR percentages during the first block of stimulation CS-alone presentations were equivalent to terminal acquisition levels at both ages but extinction was more rapid in the P17 group. These findings indicate that forgetting is observed in P17 relative to P24 rats 24 h following acquisition. The forgetting in P17 rats has important implications for the neurobiological mechanisms of memory in the developing cerebellum.