Cerebellar supervised learning revisited: biophysical modeling and degrees-of-freedom control

Theoretical limits on the speed of learning inverse models explain the rate of adaptation in arm reaching tasks

An essential aspect of human motor learning is the formation of inverse models, which map desired actions to motor commands. Inverse models can be learned by adjusting parameters in neural circuits to minimize errors in the performance of motor tasks through gradient descent. However, the theory of gradient descent establishes limits on the learning speed. Specifically, the eigenvalues of the Hessian of the error surface around a minimum determine the maximum speed of learning in a task. Here, we use this theoretical framework to analyze the speed of learning in different inverse model learning architectures in a set of isometric arm-reaching tasks. We show theoretically that, in these tasks, the error surface and, thus the speed of learning, are determined by the shapes of the force manipulability ellipsoid of the arm and the distribution of targets in the task. In particular, rounder manipulability ellipsoids generate a rounder error surface, allowing for faster learning of the inverse model. Rounder target distributions have a similar effect. We tested these predictions experimentally in a quasi-isometric reaching task with a visuomotor transformation. The experimental results were consistent with our theoretical predictions. Furthermore, our analysis accounts for the speed of learning in previous experiments with incompatible and compatible virtual surgery tasks, and with visuomotor rotation tasks with different numbers of targets. By identifying aspects of a task that influence the speed of learning, our results provide theoretical principles for the design of motor tasks that allow for faster learning.

Emergence of syntax and word prediction in an artificial neural circuit of the cerebellum

The cerebellum, interconnected with the cerebral neocortex, plays a vital role in human-characteristic cognition such as language processing, however, knowledge about the underlying circuit computation of the cerebellum remains very limited. To gain a better understanding of the computation underlying cerebellar language processing, we developed a biologically constrained cerebellar artificial neural network (cANN) model, which implements the recently identified cerebello-cerebellar recurrent pathway. We found that while cANN acquires prediction of future words, another function of syntactic recognition emerges in the middle layer of the prediction circuit. The recurrent pathway of the cANN was essential for the two language functions, whereas cANN variants with further biological constraints preserved these functions. Considering the uniform structure of cerebellar circuitry across all functional domains, the single-circuit computation, which is the common basis of the two language functions, can be generalized to fundamental cerebellar functions of prediction and grammar-like rule extraction from sequences, that underpin a wide range of cerebellar motor and cognitive functions. This is a pioneering study to understand the circuit computation of human-characteristic cognition using biologically-constrained ANNs.

Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional components reduces dimensions for reinforcement learning

Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing 'Go/No-go' reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.

Excitatory nucleo-olivary pathway shapes cerebellar outputs for motor control

The brain generates predictive motor commands to control the spatiotemporal precision of high-velocity movements. Yet, how the brain organizes automated internal feedback to coordinate the kinematics of such fast movements is unclear. Here we unveil a unique nucleo-olivary loop in the cerebellum and its involvement in coordinating high-velocity movements. Activating the excitatory nucleo-olivary pathway induces well-timed internal feedback complex spike signals in Purkinje cells to shape cerebellar outputs. Anatomical tracing reveals extensive axonal collaterals from the excitatory nucleo-olivary neurons to downstream motor regions, supporting integration of motor output and internal feedback signals within the cerebellum. This pathway directly drives saccades and head movements with a converging direction, while curtailing their amplitude and velocity via the powerful internal feedback mechanism. Our finding challenges the long-standing dogma that the cerebellum inhibits the inferior olivary pathway and provides a new circuit mechanism for the cerebellar control of high-velocity movements.

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Although it may appear infeasible and impractical, building artificial intelligence (AI) using a bottom-up approach based on the understanding of neuroscience is straightforward. The lack of a generalized governing principle for biological neural networks (BNNs) forces us to address this problem by converting piecemeal information on the diverse features of neurons, synapses, and neural circuits into AI. In this review, we described recent attempts to build a biologically plausible neural network by following neuroscientifically similar strategies of neural network optimization or by implanting the outcome of the optimization, such as the properties of single computational units and the characteristics of the network architecture. In addition, we proposed a formalism of the relationship between the set of objectives that neural networks attempt to achieve, and neural network classes categorized by how closely their architectural features resemble those of BNN. This formalism is expected to define the potential roles of top-down and bottom-up approaches for building a biologically plausible neural network and offer a map helping the navigation of the gap between neuroscience and AI engineering.

A cerebellum inspired spiking neural network as a multi-model for pattern classification and robotic trajectory prediction

Spiking neural networks were introduced to understand spatiotemporal information processing in neurons and have found their application in pattern encoding, data discrimination, and classification. Bioinspired network architectures are considered for event-driven tasks, and scientists have looked at different theories based on the architecture and functioning. Motor tasks, for example, have networks inspired by cerebellar architecture where the granular layer recodes sparse representations of the mossy fiber (MF) inputs and has more roles in motor learning. Using abstractions from cerebellar connections and learning rules of deep learning network (DLN), patterns were discriminated within datasets, and the same algorithm was used for trajectory optimization. In the current work, a cerebellum-inspired spiking neural network with dynamics of cerebellar neurons and learning mechanisms attributed to the granular layer, Purkinje cell (PC) layer, and cerebellar nuclei interconnected by excitatory and inhibitory synapses was implemented. The model’s pattern discrimination capability was tested for two tasks on standard machine learning (ML) datasets and on following a trajectory of a low-cost sensor-free robotic articulator. Tuned for supervised learning, the pattern classification capability of the cerebellum-inspired network algorithm has produced more generalized models than data-specific precision models on smaller training datasets. The model showed an accuracy of 72%, which was comparable to standard ML algorithms, such as MLP (78%), Dl4jMlpClassifier (64%), RBFNetwork (71.4%), and libSVM-linear (85.7%). The cerebellar model increased the network’s capability and decreased storage, augmenting faster computations. Additionally, the network model could also implicitly reconstruct the trajectory of a 6-degree of freedom (DOF) robotic arm with a low error rate by reconstructing the kinematic parameters. The variability between the actual and predicted trajectory points was noted to be ± 3 cm (while moving to a position in a cuboid space of 25 × 30 × 40 cm). Although a few known learning rules were implemented among known types of plasticity in the cerebellum, the network model showed a generalized processing capability for a range of signals, modulating the data through the interconnected neural populations. In addition to potential use on sensor-free or feed-forward based controllers for robotic arms and as a generalized pattern classification algorithm, this model adds implications to motor learning theory.

Localization of long-term synaptic plasticity defects in cerebellar circuits using optokinetic reflex learning profile

Objective.Functional maps of the central nervous system attribute the coordination and control of many body movements directly or indirectly to the cerebellum. Despite this general picture, there is little information on the function of cerebellar neural components at the circuit level. The presence of multiple synaptic junctions and the synergistic action of different types of plasticity make it virtually difficult to determine the distinct contribution of cerebellar neural processes to behavioral manifestations. In this study, investigating the effect of long-term synaptic changes on cerebellar motor learning, we intend to provide quantitative criteria for localizing defects in the major forms of synaptic plasticity in the cerebellum.Approach.To this end, we develop a firing rate model of the cerebellar circuits to simulate learning of optokinetic reflex (OKR), one of the most well-known cerebellar-dependent motor tasks. In the following, by comparing the simulated OKR learning profile for normal and pathosynaptic conditions, we extract the learning features affected by long-term plasticity disorders. Next, conducting simulation with different massed (continuous with no rest) and spaced (interleaved with rest periods) learning paradigms, we estimate the detrimental impact of plasticity defects at corticonuclear synapses on short- and long-term motor memory.Main results.Our computational approach predicts a correlation between location and grade of the defect with some learning factors such as the rate of formation and retention of motor memory, baseline performance, and even cerebellar motor reserve capacity. Further, spacing analysis reveal the dependence of learning paradigm efficiency on the spatiotemporal characteristic of defect in the network. Indeed, defects in cortical memory formation and nuclear memory consolidation mainly harm massed and spaced learning, respectively. This result is used to design a differential assay for identifying the faulty phases of cerebellar learning.Significance.The proposed computational framework can help develop neural-screening systems and prepare meso-scale functional maps of the cerebellar circuits.

Chaos may enhance expressivity in cerebellar granular layer

Recent evidence suggests that Golgi cells in the cerebellar granular layer are densely connected to each other with massive gap junctions. Here, we propose that the massive gap junctions between the Golgi cells contribute to the representational complexity of the granular layer of the cerebellum by inducing chaotic dynamics. We construct a model of cerebellar granular layer with diffusion coupling through gap junctions between the Golgi cells, and evaluate the representational capability of the network with the reservoir computing framework. First, we show that the chaotic dynamics induced by diffusion coupling results in complex output patterns containing a wide range of frequency components. Second, the long non-recursive time series of the reservoir represents the passage of time from an external input. These properties of the reservoir enable mapping different spatial inputs into different temporal patterns.

Unconscious reinforcement learning of hidden brain states supported by confidence

Can humans be trained to make strategic use of latent representations in their own brains? We investigate how human subjects can derive reward-maximizing choices from intrinsic high-dimensional information represented stochastically in neural activity. Reward contingencies are defined in real-time by fMRI multivoxel patterns; optimal action policies thereby depend on multidimensional brain activity taking place below the threshold of consciousness, by design. We find that subjects can solve the task within two hundred trials and errors, as their reinforcement learning processes interact with metacognitive functions (quantified as the meaningfulness of their decision confidence). Computational modelling and multivariate analyses identify a frontostriatal neural mechanism by which the brain may untangle the 'curse of dimensionality': synchronization of confidence representations in prefrontal cortex with reward prediction errors in basal ganglia support exploration of latent task representations. These results may provide an alternative starting point for future investigations into unconscious learning and functions of metacognition.

Electrical coupling controls dimensionality and chaotic firing of inferior olive neurons

We previously proposed, on theoretical grounds, that the cerebellum must regulate the dimensionality of its neuronal activity during motor learning and control to cope with the low firing frequency of inferior olive neurons, which form one of two major inputs to the cerebellar cortex. Such dimensionality regulation is possible via modulation of electrical coupling through the gap junctions between inferior olive neurons by inhibitory GABAergic synapses. In addition, we previously showed in simulations that intermediate coupling strengths induce chaotic firing of inferior olive neurons and increase their information carrying capacity. However, there is no in vivo experimental data supporting these two theoretical predictions. Here, we computed the levels of synchrony, dimensionality, and chaos of the inferior olive code by analyzing in vivo recordings of Purkinje cell complex spike activity in three different coupling conditions: carbenoxolone (gap junctions blocker), control, and picrotoxin (GABA-A receptor antagonist). To examine the effect of electrical coupling on dimensionality and chaotic dynamics, we first determined the physiological range of effective coupling strengths between inferior olive neurons in the three conditions using a combination of a biophysical network model of the inferior olive and a novel Bayesian model averaging approach. We found that effective coupling co-varied with synchrony and was inversely related to the dimensionality of inferior olive firing dynamics, as measured via a principal component analysis of the spike trains in each condition. Furthermore, for both the model and the data, we found an inverted U-shaped relationship between coupling strengths and complexity entropy, a measure of chaos for spiking neural data. These results are consistent with our hypothesis according to which electrical coupling regulates the dimensionality and the complexity in the inferior olive neurons in order to optimize both motor learning and control of high dimensional motor systems by the cerebellum.

Computational theory suggests that the cerebellum must decrease the dimensionality of its neuronal activity to learn and control high dimensional motor systems effectively, while being constrained by the low firing frequency of inferior olive neurons, one of the two major source of input signals to the cerebellum. We previously proposed that the cerebellum adaptively controls the dimensionality of inferior olive firing by adjusting the level of synchrony and that such control is made possible by modulating the electrical coupling strength between inferior olive neurons. Here, we developed a novel method that uses a biophysical model of the inferior olive to accurately estimate the effective coupling strengths between inferior olive neurons from in vivo recordings of spike activity in three different coupling conditions. We found that high coupling strengths induce synchronous firing and decrease the dimensionality of inferior olive firing dynamics. In contrast, intermediate coupling strengths lead to chaotic firing and increase the dimensionality of the firing dynamics. Thus, electrical coupling is a feasible mechanism to control dimensionality and chaotic firing of inferior olive neurons. In sum, our results provide insights into possible mechanisms underlying cerebellar function and, in general, a biologically plausible framework to control the dimensionality of neural coding.

50 Years Since the Marr, Ito, and Albus Models of the Cerebellum

Fifty years have passed since David Marr, Masao Ito, and James Albus proposed seminal models of cerebellar functions. These models share the essential concept that parallel-fiber-Purkinje-cell synapses undergo plastic changes, guided by climbing-fiber activities during sensorimotor learning. However, they differ in several important respects, including holistic versus complementary roles of the cerebellum, pattern recognition versus control as computational objectives, potentiation versus depression of synaptic plasticity, teaching signals versus error signals transmitted by climbing-fibers, sparse expansion coding by granule cells, and cerebellar internal models. In this review, we evaluate different features of the three models based on recent computational and experimental studies. While acknowledging that the three models have greatly advanced our understanding of cerebellar control mechanisms in eye movements and classical conditioning, we propose a new direction for computational frameworks of the cerebellum, that is, hierarchical reinforcement learning with multiple internal models.

A Metric for Evaluating Neural Input Representation in Supervised Learning Networks

Supervised learning has long been attributed to several feed-forward neural circuits within the brain, with particular attention being paid to the cerebellar granular layer. The focus of this study is to evaluate the input activity representation of these feed-forward neural networks. The activity of cerebellar granule cells is conveyed by parallel fibers and translated into Purkinje cell activity, which constitutes the sole output of the cerebellar cortex. The learning process at this parallel-fiber-to-Purkinje-cell connection makes each Purkinje cell sensitive to a set of specific cerebellar states, which are roughly determined by the granule-cell activity during a certain time window. A Purkinje cell becomes sensitive to each neural input state and, consequently, the network operates as a function able to generate a desired output for each provided input by means of supervised learning. However, not all sets of Purkinje cell responses can be assigned to any set of input states due to the network's own limitations (inherent to the network neurobiological substrate), that is, not all input-output mapping can be learned. A key limiting factor is the representation of the input states through granule-cell activity. The quality of this representation (e.g., in terms of heterogeneity) will determine the capacity of the network to learn a varied set of outputs. Assessing the quality of this representation is interesting when developing and studying models of these networks to identify those neuron or network characteristics that enhance this representation. In this study we present an algorithm for evaluating quantitatively the level of compatibility/interference amongst a set of given cerebellar states according to their representation (granule-cell activation patterns) without the need for actually conducting simulations and network training. The algorithm input consists of a real-number matrix that codifies the activity level of every considered granule-cell in each state. The capability of this representation to generate a varied set of outputs is evaluated geometrically, thus resulting in a real number that assesses the goodness of the representation.

A Multiple-Plasticity Spiking Neural Network Embedded in a Closed-Loop Control System to Model Cerebellar Pathologies

The cerebellum plays a crucial role in sensorimotor control and cerebellar disorders compromise adaptation and learning of motor responses. However, the link between alterations at network level and cerebellar dysfunction is still unclear. In principle, this understanding would benefit of the development of an artificial system embedding the salient neuronal and plastic properties of the cerebellum and operating in closed-loop. To this aim, we have exploited a realistic spiking computational model of the cerebellum to analyze the network correlates of cerebellar impairment. The model was modified to reproduce three different damages of the cerebellar cortex: (i) a loss of the main output neurons (Purkinje Cells), (ii) a lesion to the main cerebellar afferents (Mossy Fibers), and (iii) a damage to a major mechanism of synaptic plasticity (Long Term Depression). The modified network models were challenged with an Eye-Blink Classical Conditioning test, a standard learning paradigm used to evaluate cerebellar impairment, in which the outcome was compared to reference results obtained in human or animal experiments. In all cases, the model reproduced the partial and delayed conditioning typical of the pathologies, indicating that an intact cerebellar cortex functionality is required to accelerate learning by transferring acquired information to the cerebellar nuclei. Interestingly, depending on the type of lesion, the redistribution of synaptic plasticity and response timing varied greatly generating specific adaptation patterns. Thus, not only the present work extends the generalization capabilities of the cerebellar spiking model to pathological cases, but also predicts how changes at the neuronal level are distributed across the network, making it usable to infer cerebellar circuit alterations occurring in cerebellar pathologies.

The architecture challenge: Future artificial-intelligence systems will require sophisticated architectures, and knowledge of the brain might guide their construction

In this commentary, we highlight a crucial challenge posed by the proposal of Lake et al. to introduce key elements of human cognition into deep neural networks and future artificial-intelligence systems: the need to design effective sophisticated architectures. We propose that looking at the brain is an important means of facing this great challenge.

Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex

Despite increasing evidence suggesting the cerebellum works in concert with the cortex and basal ganglia, the nature of the reciprocal interactions between these three brain regions remains unclear. This consensus paper gathers diverse recent views on a variety of important roles played by the cerebellum within the cerebello-basal ganglia-thalamo-cortical system across a range of motor and cognitive functions. The paper includes theoretical and empirical contributions, which cover the following topics: recent evidence supporting the dynamical interplay between cerebellum, basal ganglia, and cortical areas in humans and other animals; theoretical neuroscience perspectives and empirical evidence on the reciprocal influences between cerebellum, basal ganglia, and cortex in learning and control processes; and data suggesting possible roles of the cerebellum in basal ganglia movement disorders. Although starting from different backgrounds and dealing with different topics, all the contributors agree that viewing the cerebellum, basal ganglia, and cortex as an integrated system enables us to understand the function of these areas in radically different ways. In addition, there is unanimous consensus between the authors that future experimental and computational work is needed to understand the function of cerebellar-basal ganglia circuitry in both motor and non-motor functions. The paper reports the most advanced perspectives on the role of the cerebellum within the cerebello-basal ganglia-thalamo-cortical system and illustrates other elements of consensus as well as disagreements and open questions in the field.

The Roles of the Olivocerebellar Pathway in Motor Learning and Motor Control. A Consensus Paper

For many decades, the predominant view in the cerebellar field has been that the olivocerebellar system's primary function is to induce plasticity in the cerebellar cortex, specifically, at the parallel fiber-Purkinje cell synapse. However, it has also long been proposed that the olivocerebellar system participates directly in motor control by helping to shape ongoing motor commands being issued by the cerebellum. Evidence consistent with both hypotheses exists; however, they are often investigated as mutually exclusive alternatives. In contrast, here, we take the perspective that the olivocerebellar system can contribute to both the motor learning and motor control functions of the cerebellum and might also play a role in development. We then consider the potential problems and benefits of it having multiple functions. Moreover, we discuss how its distinctive characteristics (e.g., low firing rates, synchronization, and variable complex spike waveforms) make it more or less suitable for one or the other of these functions, and why having multiple functions makes sense from an evolutionary perspective. We did not attempt to reach a consensus on the specific role(s) the olivocerebellar system plays in different types of movements, as that will ultimately be determined experimentally; however, collectively, the various contributions highlight the flexibility of the olivocerebellar system, and thereby suggest that it has the potential to act in both the motor learning and motor control functions of the cerebellum.

New insights into olivo-cerebellar circuits for learning from a small training sample

Artificial intelligence such as deep neural networks exhibited remarkable performance in simulated video games and 'Go'. In contrast, most humanoid robots in the DARPA Robotics Challenge fell down to ground. The dramatic contrast in performance is mainly due to differences in the amount of training data, which is huge and small, respectively. Animals are not allowed with millions of the failed trials, which lead to injury and death. Humans fall only several thousand times before they balance and walk. We hypothesize that a unique closed-loop neural circuit formed by the Purkinje cells, the cerebellar deep nucleus and the inferior olive in and around the cerebellum and the highest density of gap junctions, which regulate synchronous activities of the inferior olive nucleus, are computational machinery for learning from a small sample. We discuss recent experimental and computational advances associated with this hypothesis.

Small-Volume Effect Enables Robust, Sensitive, and Efficient Information Transfer in the Spine

Why is the spine of a neuron so small that it can contain only small numbers of molecules and reactions inevitably become stochastic? We previously showed that, despite such noisy conditions, the spine exhibits robust, sensitive, and efficient features of information transfer using the probability of Ca²⁺ increase; however, the mechanisms are unknown. In this study, we show that the small volume effect enables robust, sensitive, and efficient information transfer in the spine volume, but not in the cell volume. In the spine volume, the intrinsic noise in reactions becomes larger than the extrinsic noise of input, resulting in robust information transfer despite input fluctuation. In the spine volume, stochasticity makes the Ca²⁺ increase occur with a lower intensity of input, causing higher sensitivity to lower intensity of input. The volume-dependency of information transfer increases its efficiency in the spine volume. Thus, we propose that the small-volume effect is the functional reason why the spine has to be so small.

Dysfunctions of the basal ganglia-cerebellar-thalamo-cortical system produce motor tics in Tourette syndrome

Motor tics are a cardinal feature of Tourette syndrome and are traditionally associated with an excess of striatal dopamine in the basal ganglia. Recent evidence increasingly supports a more articulated view where cerebellum and cortex, working closely in concert with basal ganglia, are also involved in tic production. Building on such evidence, this article proposes a computational model of the basal ganglia-cerebellar-thalamo-cortical system to study how motor tics are generated in Tourette syndrome. In particular, the model: (i) reproduces the main results of recent experiments about the involvement of the basal ganglia-cerebellar-thalamo-cortical system in tic generation; (ii) suggests an explanation of the system-level mechanisms underlying motor tic production: in this respect, the model predicts that the interplay between dopaminergic signal and cortical activity contributes to triggering the tic event and that the recently discovered basal ganglia-cerebellar anatomical pathway may support the involvement of the cerebellum in tic production; (iii) furnishes predictions on the amount of tics generated when striatal dopamine increases and when the cortex is externally stimulated. These predictions could be important in identifying new brain target areas for future therapies. Finally, the model represents the first computational attempt to study the role of the recently discovered basal ganglia-cerebellar anatomical links. Studying this non-cortex-mediated basal ganglia-cerebellar interaction could radically change our perspective about how these areas interact with each other and with the cortex. Overall, the model also shows the utility of casting Tourette syndrome within a system-level perspective rather than viewing it as related to the dysfunction of a single brain area.

Tourette syndrome is a neuropsychiatric disorder characterized by vocal and motor tics. Tics represent a cardinal symptom traditionally associated with a dysfunction of the basal ganglia leading to an excess of the dopamine neurotransmitter. This view gives a restricted clinical picture and limits therapeutic approaches because it ignores the influence of altered interactions between the basal ganglia and other brain areas. In this respect, recent evidence supports a more articulated framework where cerebellum and cortex are also involved in tic production. Building on these data, we propose a computational model of the basal ganglia-cerebellar-thalamo-cortical network to investigate the specific mechanisms underlying motor tic production. The model reproduces the results of recent experiments and suggests an explanation of the system-level processes underlying tic production. Moreover, it furnishes predictions related to the amount of tics generated when there are dysfunctions in the basal ganglia-cerebellar-thalamo-cortical circuits. These predictions could be important in identifying new brain target areas for future therapies based on a system-level view of Tourette syndrome.

Chaotic Resonance in Coupled Inferior Olive Neurons with the Llinás Approach Neuron Model

It is well known that cerebellar motor control is fine-tuned by the learning process adjusted according to rich error signals from inferior olive (IO) neurons. Schweighofer and colleagues proposed that these signals can be produced by chaotic irregular firing in the IO neuron assembly; such chaotic resonance (CR) was replicated in their computer demonstration of a Hodgkin-Huxley (HH)-type compartment model. In this study, we examined the response of CR to a periodic signal in the IO neuron assembly comprising the Llinás approach IO neuron model. This system involves empirically observed dynamics of the IO membrane potential and is simpler than the HH-type compartment model. We then clarified its dependence on electrical coupling strength, input signal strength, and frequency. Furthermore, we compared the physiological validity for IO neurons such as low firing rate and sustaining subthreshold oscillation between CR and conventional stochastic resonance (SR) and examined the consistency with asynchronous firings indicated by the previous model-based studies in the cerebellar learning process. In addition, the signal response of CR and SR was investigated in a large neuron assembly. As the result, we confirmed that CR was consistent with the above IO neuron's characteristics, but it was not as easy for SR.

Computational neurorehabilitation: modeling plasticity and learning to predict recovery

Despite progress in using computational approaches to inform medicine and neuroscience in the last 30 years, there have been few attempts to model the mechanisms underlying sensorimotor rehabilitation. We argue that a fundamental understanding of neurologic recovery, and as a result accurate predictions at the individual level, will be facilitated by developing computational models of the salient neural processes, including plasticity and learning systems of the brain, and integrating them into a context specific to rehabilitation. Here, we therefore discuss Computational Neurorehabilitation, a newly emerging field aimed at modeling plasticity and motor learning to understand and improve movement recovery of individuals with neurologic impairment. We first explain how the emergence of robotics and wearable sensors for rehabilitation is providing data that make development and testing of such models increasingly feasible. We then review key aspects of plasticity and motor learning that such models will incorporate. We proceed by discussing how computational neurorehabilitation models relate to the current benchmark in rehabilitation modeling - regression-based, prognostic modeling. We then critically discuss the first computational neurorehabilitation models, which have primarily focused on modeling rehabilitation of the upper extremity after stroke, and show how even simple models have produced novel ideas for future investigation. Finally, we conclude with key directions for future research, anticipating that soon we will see the emergence of mechanistic models of motor recovery that are informed by clinical imaging results and driven by the actual movement content of rehabilitation therapy as well as wearable sensor-based records of daily activity.

Clocks within Clocks: Timing by Coincidence Detection

The many existent models of timing rely on vastly different mechanisms to track temporal information. Here we examine these differences, and identify coincidence detection in its most general form as a common mechanism that many apparently different timing models share, as well as a common mechanism of biological circadian, millisecond and interval timing. This view predicts that timing by coincidence detection is a ubiquitous phenomenon at many biological levels, explains the reports of biological timing in many brain areas, explains the role of neural noise at different time scales at both biological and theoretical levels, and provides cohesion within the timing field.

Redefining cerebellar ataxia in degenerative ataxias: lessons from recent research on cerebellar systems

Recent advances in our understanding of neurophysiological functions in the cerebellar system have revealed that each region involved in degenerative ataxias contributes differently. To regulate voluntary movements, the cerebellum forms internal models within its neural circuits that mimic the behaviour of the sensorimotor system and objects in the external environment. The cerebellum forms two different internal models: forward and inverse. The forward model is formed by efference copy signals conveyed by the corticopontocerebellar system, and it derives the estimated consequences for action. The inverse model describes sequences of motor commands to accomplish an aim. During motor learning, we improve internal models by comparing the estimated consequence of an action from the forward model with the actual consequence of the action produced by the inverse model. The functions of the cerebellum encompass the formation, storage and selection of internal models. Considering the neurophysiological properties of the cerebellar system, we have classified degenerative ataxias into four types depending on which system is involved: Purkinje cells, the corticopontocerebellar system, the spinocerebellar system and the cerebellar deep nuclei. With regard to their respective contributions to the internal models, we speculate that loss of Purkinje cells leads to malformation of the internal models, whereas disturbance of the afferent system, corticopontocerebellar system or spinocerebellar system leads to mis-selection of the proper internal model. An understanding of the pathophysiological properties of ataxias in each degenerative ataxia enables the development of new methods to evaluate ataxias.

The neural basis of the imitation drive

Spontaneous imitation is assumed to underlie the acquisition of important skills by infants, including language and social interaction. In this study, functional magnetic resonance imaging (fMRI) was used to examine the neural basis of ‘spontaneously’ driven imitation, which has not yet been fully investigated. Healthy participants were presented with movie clips of meaningless bimanual actions and instructed to observe and imitate them during an fMRI scan. The participants were subsequently shown the movie clips again and asked to evaluate the strength of their ‘urge to imitate’ (Urge) for each action. We searched for cortical areas where the degree of activation positively correlated with Urge scores; significant positive correlations were observed in the right supplementary motor area (SMA) and bilateral midcingulate cortex (MCC) under the imitation condition. These areas were not explained by explicit reasons for imitation or the kinematic characteristics of the actions. Previous studies performed in monkeys and humans have implicated the SMA and MCC/caudal cingulate zone in voluntary actions. This study also confirmed the functional connectivity between Urge and imitation performance using a psychophysiological interaction analysis. Thus, our findings reveal the critical neural components that underlie spontaneous imitation and provide possible reasons why infants imitate spontaneously.

Duration of Purkinje cell complex spikes increases with their firing frequency

Climbing fiber (CF) triggered complex spikes (CS) are massive depolarization bursts in the cerebellar Purkinje cell (PC), showing several high frequency spikelet components (±600 Hz). Since its early observations, the CS is known to vary in shape. In this study we describe CS waveforms, extracellularly recorded in awake primates (Macaca mulatta) performing saccades. Every PC analyzed showed a range of CS shapes with profoundly different duration and number of spikelets. The initial part of the CS was rather constant but the later part differed greatly, with a pronounced jitter of the last spikelets causing a large variation in total CS duration. Waveforms did not effect the following pause duration in the simple spike (SS) train, nor were SS firing rates predictive of the waveform shapes or vice versa. The waveforms did not differ between experimental conditions nor was there a preferred sequential order of CS shapes throughout the recordings. Instead, part of their variability, the timing jitter of the CS’s last spikelets, strongly correlated with interval length to the preceding CS: shorter CS intervals resulted in later appearance of the last spikelets in the CS burst, and vice versa. A similar phenomenon was observed in rat PCs recorded in vitro upon repeated extracellular stimulation of CFs at different frequencies in slice experiments. All together these results strongly suggest that the variability in the timing of the last spikelet is due to CS frequency dependent changes in PC excitability.

Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control

Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.

Stochasticity in Ca2+ increase in spines enables robust and sensitive information coding

A dendritic spine is a very small structure (∼0.1 µm³) of a neuron that processes input timing information. Why are spines so small? Here, we provide functional reasons; the size of spines is optimal for information coding. Spines code input timing information by the probability of Ca²⁺ increases, which makes robust and sensitive information coding possible. We created a stochastic simulation model of input timing-dependent Ca²⁺ increases in a cerebellar Purkinje cell's spine. Spines used probability coding of Ca²⁺ increases rather than amplitude coding for input timing detection via stochastic facilitation by utilizing the small number of molecules in a spine volume, where information per volume appeared optimal. Probability coding of Ca²⁺ increases in a spine volume was more robust against input fluctuation and more sensitive to input numbers than amplitude coding of Ca²⁺ increases in a cell volume. Thus, stochasticity is a strategy by which neurons robustly and sensitively code information.

Delegation to automaticity: the driving force for cognitive evolution?

The ability to delegate control over repetitive tasks from higher to lower neural centers may be a fundamental innovation in human cognition. Plausibly, the massive neurocomputational challenges associated with the mastery of balance during the evolution of bipedality in proto-humans provided a strong selective advantage to individuals with brains capable of efficiently transferring tasks in this way. Thus, the shift from quadrupedal to bipedal locomotion may have driven the rapid evolution of distinctive features of human neuronal functioning. We review recent studies of functional neuroanatomy that bear upon this hypothesis, and identify ways to test our ideas.

A spiking network model of decision making employing rewarded STDP

Reward-modulated spike timing dependent plasticity (STDP) combines unsupervised STDP with a reinforcement signal that modulates synaptic changes. It was proposed as a learning rule capable of solving the distal reward problem in reinforcement learning. Nonetheless, performance and limitations of this learning mechanism have yet to be tested for its ability to solve biological problems. In our work, rewarded STDP was implemented to model foraging behavior in a simulated environment. Over the course of training the network of spiking neurons developed the capability of producing highly successful decision-making. The network performance remained stable even after significant perturbations of synaptic structure. Rewarded STDP alone was insufficient to learn effective decision making due to the difficulty maintaining homeostatic equilibrium of synaptic weights and the development of local performance maxima. Our study predicts that successful learning requires stabilizing mechanisms that allow neurons to balance their input and output synapses as well as synaptic noise.

Role of the olivo-cerebellar complex in motor learning and control

How is the cerebellum capable of efficient motor learning and control despite very low firing of the inferior olive (IO) inputs, which are postulated to carry errors needed for learning and contribute to on-line motor control? IO neurons form the largest electrically coupled network in the adult human brain. Here, we discuss how intermediate coupling strengths can lead to chaotic resonance and increase information transmission of the error signal despite the very low IO firing rate. This increased information transmission can then lead to more efficient learning than with weak or strong coupling. In addition, we argue that a dynamic modulation of IO electrical coupling via the Purkinje cell-deep cerebellar neurons – IO triangle could speed up learning and improve on-line control. Initially strong coupling would allow transmission of large errors to multiple functionally related Purkinje cells, resulting in fast but coarse learning as well as significant effects on deep cerebellar nucleus and on-line motor control. In the late phase of learning decreased coupling would allow desynchronized IO firing, allowing high-fidelity transmission of error, resulting in slower but fine learning, and little on-line motor control effects.

Spike timing regulation on the millisecond scale by distributed synaptic plasticity at the cerebellum input stage: a simulation study

The way long-term synaptic plasticity regulates neuronal spike patterns is not completely understood. This issue is especially relevant for the cerebellum, which is endowed with several forms of long-term synaptic plasticity and has been predicted to operate as a timing and a learning machine. Here we have used a computational model to simulate the impact of multiple distributed synaptic weights in the cerebellar granular-layer network. In response to mossy fiber (MF) bursts, synaptic weights at multiple connections played a crucial role to regulate spike number and positioning in granule cells. The weight at MF to granule cell synapses regulated the delay of the first spike and the weight at MF and parallel fiber to Golgi cell synapses regulated the duration of the time-window during which the first-spike could be emitted. Moreover, the weights of synapses controlling Golgi cell activation regulated the intensity of granule cell inhibition and therefore the number of spikes that could be emitted. First-spike timing was regulated with millisecond precision and the number of spikes ranged from zero to three. Interestingly, different combinations of synaptic weights optimized either first-spike timing precision or spike number, efficiently controlling transmission and filtering properties. These results predict that distributed synaptic plasticity regulates the emission of quasi-digital spike patterns on the millisecond time-scale and allows the cerebellar granular layer to flexibly control burst transmission along the MF pathway.

A common framework of signal processing in the induction of cerebellar LTD and cortical STDP

Cerebellar long-term depression (LTD) and cortical spike-timing-dependent synaptic plasticity (STDP) are two well-known and well-characterized types of synaptic plasticity. Induction of both types of synaptic plasticity depends on the spike timing, pairing frequency, and pairing numbers of two different sources of spiking. This implies that the induction of synaptic plasticity may share common frameworks in terms of signal processing regardless of the different signaling pathways involved in the two types of synaptic plasticity. Here we propose that both types share common frameworks of signal processing for spike-timing, pairing-frequency, and pairing-numbers detection. We developed system models of both types of synaptic plasticity and analyzed signal processing in the induction of synaptic plasticity. We found that both systems have upstream subsystems for spike-timing detection and downstream subsystems for pairing-frequency and pairing-numbers detection. The upstream systems used multiplication of signals from the feedback filters and nonlinear functions for spike-timing detection. The downstream subsystems used temporal filters with longer time constants for pairing-frequency detection and nonlinear switch-like functions for pairing-numbers detection, indicating that the downstream subsystems serve as a leaky integrate-and-fire system. Thus, our findings suggest that a common conceptual framework for the induction of synaptic plasticity exists despite the differences in molecular species and pathways.

Solution to the inverse problem of estimating gap-junctional and inhibitory conductance in inferior olive neurons from spike trains by network model simulation

The inferior olive (IO) possesses synaptic glomeruli, which contain dendritic spines from neighboring neurons and presynaptic terminals, many of which are inhibitory and GABAergic. Gap junctions between the spines electrically couple neighboring neurons whereas the GABAergic synaptic terminals are thought to act to decrease the effectiveness of this coupling. Thus, the glomeruli are thought to be important for determining the oscillatory and synchronized activity displayed by IO neurons. Indeed, the tendency to display such activity patterns is enhanced or reduced by the local administration of the GABA-A receptor blocker picrotoxin (PIX) or the gap junction blocker carbenoxolone (CBX), respectively. We studied the functional roles of the glomeruli by solving the inverse problem of estimating the inhibitory (gi) and gap-junctional conductance (gc) using an IO network model. This model was built upon a prior IO network model, in which the individual neurons consisted of soma and dendritic compartments, by adding a glomerular compartment comprising electrically coupled spines that received inhibitory synapses. The model was used in the forward mode to simulate spike data under PIX and CBX conditions for comparison with experimental data consisting of multi-electrode recordings of complex spikes from arrays of Purkinje cells (complex spikes are generated in a one-to-one manner by IO spikes and thus can substitute for directly measuring IO spike activity). The spatiotemporal firing dynamics of the experimental and simulation spike data were evaluated as feature vectors, including firing rates, local variation, auto-correlogram, cross-correlogram, and minimal distance, and were contracted onto two-dimensional principal component analysis (PCA) space. gc and gi were determined as the solution to the inverse problem such that the simulation and experimental spike data were closely matched in the PCA space. The goodness of the match was confirmed by an analysis of variance (ANOVA) of the PCA scores between the experimental and simulation spike data. In the PIX condition, gi was found to decrease to approximately half its control value. CBX caused an approximately 30% decrease in gc from control levels. These results support the hypothesis that the glomeruli are control points for determining the spatiotemporal characteristics of olivocerebellar activity and thus may shape its ability to convey signals to the cerebellum that may be used for motor learning or motor control purposes.

Adaptive coupling of inferior olive neurons in cerebellar learning

In the cerebellar learning hypothesis, inferior olive neurons are presumed to transmit high fidelity error signals, despite their low firing rates. The idea of chaotic resonance has been proposed to realize efficient error transmission by desynchronized spiking activities induced by moderate electrical coupling between inferior olive neurons. A recent study suggests that the coupling strength between inferior olive neurons can be adaptive and may decrease during the learning process. We show that such a decrease in coupling strength can be beneficial for motor learning, since efficient coupling strength depends upon the magnitude of the error signals. We introduce a scheme of adaptive coupling that enhances the learning of a neural controller for fast arm movements. Our numerical study supports the view that the controlling strategy of the coupling strength provides an additional degree of freedom to optimize the actual learning in the cerebellum.

Mechanisms producing time course of cerebellar long-term depression

Cerebellar long-term depression (LTD) is induced by short-lasting synaptic activities, progressively expressed, and then maintained for hours or longer. Short-lasting events, such as calcium transients, are activated and required for the induction of LTD. Further, a positive-feedback kinase loop was shown to follow the transient events and to aid the transition between LTD induction and prolonged synaptic depression. Yet, it is not entirely clear as to how LTD is maintained and how the maintenance mechanisms are activated, mainly because of a lack of experimental studies regarding this topic, while an idea has been theoretically proposed. A new analysis of the experimental results suggests that early maintenance mechanisms display a threshold behavior and that they may be of stochastic nature. This suggestion is conceptually consistent with an idea from a computational study, which postulates that other bistable switch systems are required for LTD maintenance. We thus propose that cellular mechanisms showing a threshold behavior and a stochastic nature maintain LTD, and that future experimental studies in search of such mechanisms would be an important step toward fully understanding the time course of LTD.

Cerebellar LTD vs. motor learning-lessons learned from studying GluD2

Synaptic plasticity, such as long-term potentiation and long-term depression (LTD), is believed to underlie learning and memory processes in vivo. The cerebellum is an ideal brain region to obtain definitive proof for this hypothesis. The current belief is that the acquisition of motor learning is stored by LTD at the parallel fiber (PF)-Purkinje cell synapse in the cerebellar cortex. Recently, however, several lines of mutant mice that display normal motor learning in the absence of cerebellar LTD have been reported. A similar dichotomy between synaptic plasticity at the circuitry level and learning at the behavioral level has also been reported in the hippocampus. One possible explanation for this dichotomy is that compensatory pathways at the molecular and circuitry levels play an important role in mice that have been genetically modified for their entire lives. Mice that are genetically modified to be deficient in or to express mutant versions of the δ2 glutamate receptor (GluD2) serve as an interesting model due to the predominant expression of GluD2 at PF-Purkinje cell synapses. Furthermore, two major functions of GluD2-PF synapse formation and LTD induction-can be mechanistically dissociated so that the role of LTD in motor learning can be investigated in the absence of morphological abnormalities caused by altered synapse formation. Therefore, genetic manipulations of GluD2 will help to clarify the relationship between LTD and motor learning in the cerebellum.