Adaptive coupling of inferior olive neurons in cerebellar learning

Chaotic recurrent neural networks for brain modelling: A review

Even in the absence of external stimuli, the brain is spontaneously active. Indeed, most cortical activity is internally generated by recurrence. Both theoretical and experimental studies suggest that chaotic dynamics characterize this spontaneous activity. While the precise function of brain chaotic activity is still puzzling, we know that chaos confers many advantages. From a computational perspective, chaos enhances the complexity of network dynamics. From a behavioural point of view, chaotic activity could generate the variability required for exploration. Furthermore, information storage and transfer are maximized at the critical border between order and chaos. Despite these benefits, many computational brain models avoid incorporating spontaneous chaotic activity due to the challenges it poses for learning algorithms. In recent years, however, multiple approaches have been proposed to overcome this limitation. As a result, many different algorithms have been developed, initially within the reservoir computing paradigm. Over time, the field has evolved to increase the biological plausibility and performance of the algorithms, sometimes going beyond the reservoir computing framework. In this review article, we examine the computational benefits of chaos and the unique properties of chaotic recurrent neural networks, with a particular focus on those typically utilized in reservoir computing. We also provide a detailed analysis of the algorithms designed to train chaotic RNNs, tracing their historical evolution and highlighting key milestones in their development. Finally, we explore the applications and limitations of chaotic RNNs for brain modelling, consider their potential broader impacts beyond neuroscience, and outline promising directions for future research.

Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional components reduces dimensions for reinforcement learning

Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing 'Go/No-go' reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.

Chaos may enhance expressivity in cerebellar granular layer

Recent evidence suggests that Golgi cells in the cerebellar granular layer are densely connected to each other with massive gap junctions. Here, we propose that the massive gap junctions between the Golgi cells contribute to the representational complexity of the granular layer of the cerebellum by inducing chaotic dynamics. We construct a model of cerebellar granular layer with diffusion coupling through gap junctions between the Golgi cells, and evaluate the representational capability of the network with the reservoir computing framework. First, we show that the chaotic dynamics induced by diffusion coupling results in complex output patterns containing a wide range of frequency components. Second, the long non-recursive time series of the reservoir represents the passage of time from an external input. These properties of the reservoir enable mapping different spatial inputs into different temporal patterns.

Electrical coupling controls dimensionality and chaotic firing of inferior olive neurons

We previously proposed, on theoretical grounds, that the cerebellum must regulate the dimensionality of its neuronal activity during motor learning and control to cope with the low firing frequency of inferior olive neurons, which form one of two major inputs to the cerebellar cortex. Such dimensionality regulation is possible via modulation of electrical coupling through the gap junctions between inferior olive neurons by inhibitory GABAergic synapses. In addition, we previously showed in simulations that intermediate coupling strengths induce chaotic firing of inferior olive neurons and increase their information carrying capacity. However, there is no in vivo experimental data supporting these two theoretical predictions. Here, we computed the levels of synchrony, dimensionality, and chaos of the inferior olive code by analyzing in vivo recordings of Purkinje cell complex spike activity in three different coupling conditions: carbenoxolone (gap junctions blocker), control, and picrotoxin (GABA-A receptor antagonist). To examine the effect of electrical coupling on dimensionality and chaotic dynamics, we first determined the physiological range of effective coupling strengths between inferior olive neurons in the three conditions using a combination of a biophysical network model of the inferior olive and a novel Bayesian model averaging approach. We found that effective coupling co-varied with synchrony and was inversely related to the dimensionality of inferior olive firing dynamics, as measured via a principal component analysis of the spike trains in each condition. Furthermore, for both the model and the data, we found an inverted U-shaped relationship between coupling strengths and complexity entropy, a measure of chaos for spiking neural data. These results are consistent with our hypothesis according to which electrical coupling regulates the dimensionality and the complexity in the inferior olive neurons in order to optimize both motor learning and control of high dimensional motor systems by the cerebellum.

Computational theory suggests that the cerebellum must decrease the dimensionality of its neuronal activity to learn and control high dimensional motor systems effectively, while being constrained by the low firing frequency of inferior olive neurons, one of the two major source of input signals to the cerebellum. We previously proposed that the cerebellum adaptively controls the dimensionality of inferior olive firing by adjusting the level of synchrony and that such control is made possible by modulating the electrical coupling strength between inferior olive neurons. Here, we developed a novel method that uses a biophysical model of the inferior olive to accurately estimate the effective coupling strengths between inferior olive neurons from in vivo recordings of spike activity in three different coupling conditions. We found that high coupling strengths induce synchronous firing and decrease the dimensionality of inferior olive firing dynamics. In contrast, intermediate coupling strengths lead to chaotic firing and increase the dimensionality of the firing dynamics. Thus, electrical coupling is a feasible mechanism to control dimensionality and chaotic firing of inferior olive neurons. In sum, our results provide insights into possible mechanisms underlying cerebellar function and, in general, a biologically plausible framework to control the dimensionality of neural coding.

50 Years Since the Marr, Ito, and Albus Models of the Cerebellum

Fifty years have passed since David Marr, Masao Ito, and James Albus proposed seminal models of cerebellar functions. These models share the essential concept that parallel-fiber-Purkinje-cell synapses undergo plastic changes, guided by climbing-fiber activities during sensorimotor learning. However, they differ in several important respects, including holistic versus complementary roles of the cerebellum, pattern recognition versus control as computational objectives, potentiation versus depression of synaptic plasticity, teaching signals versus error signals transmitted by climbing-fibers, sparse expansion coding by granule cells, and cerebellar internal models. In this review, we evaluate different features of the three models based on recent computational and experimental studies. While acknowledging that the three models have greatly advanced our understanding of cerebellar control mechanisms in eye movements and classical conditioning, we propose a new direction for computational frameworks of the cerebellum, that is, hierarchical reinforcement learning with multiple internal models.

Resonance phenomena controlled by external feedback signals and additive noise in neural systems

Chaotic resonance is a phenomenon that can replace the fluctuation source in stochastic resonance from additive noise to chaos. We previously developed a method to control the chaotic state for suitably generating chaotic resonance by external feedback even when the external adjustment of chaos is difficult, establishing a method named reduced region of orbit (RRO) feedback. However, a feedback signal was utilized only for dividing the merged attractor. In addition, the signal sensitivity in chaotic resonance induced by feedback signals and that of stochastic resonance by additive noise have not been compared. To merge the separated attractor, we propose a negative strength of the RRO feedback signal in a discrete neural system which is composed of excitatory and inhibitory neurons. We evaluate the features of chaotic resonance and compare it to stochastic resonance. The RRO feedback signal with negative strength can merge the separated attractor and induce chaotic resonance. We also confirm that additive noise induces stochastic resonance through attractor merging. The comparison of these resonance modalities verifies that chaotic resonance provides more applicability than stochastic resonance given its capability to handle attractor separation and merging.

Computational Principles of Supervised Learning in the Cerebellum

Supervised learning plays a key role in the operation of many biological and artificial neural networks. Analysis of the computations underlying supervised learning is facilitated by the relatively simple and uniform architecture of the cerebellum, a brain area that supports numerous motor, sensory, and cognitive functions. We highlight recent discoveries indicating that the cerebellum implements supervised learning using the following organizational principles: ( a) extensive preprocessing of input representations (i.e., feature engineering), ( b) massively recurrent circuit architecture, ( c) linear input-output computations, ( d) sophisticated instructive signals that can be regulated and are predictive, ( e) adaptive mechanisms of plasticity with multiple timescales, and ( f) task-specific hardware specializations. The principles emerging from studies of the cerebellum have striking parallels with those in other brain areas and in artificial neural networks, as well as some notable differences, which can inform future research on supervised learning and inspire next-generation machine-based algorithms.

Variability and directionality of inferior olive neuron dendrites revealed by detailed 3D characterization of an extensive morphological library

The inferior olive (IO) is an evolutionarily conserved brain stem structure and its output activity plays a major role in the cerebellar computation necessary for controlling the temporal accuracy of motor behavior. The precise timing and synchronization of IO network activity has been attributed to the dendro-dendritic gap junctions mediating electrical coupling within the IO nucleus. Thus, the dendritic morphology and spatial arrangement of IO neurons governs how synchronized activity emerges in this nucleus. To date, IO neuron structural properties have been characterized in few studies and with small numbers of neurons; these investigations have described IO neurons as belonging to two morphologically distinct types, “curly” and “straight”. In this work we collect a large number of individual IO neuron morphologies visualized using different labeling techniques and present a thorough examination of their morphological properties and spatial arrangement within the olivary neuropil. Our results show that the extensive heterogeneity in IO neuron dendritic morphologies occupies a continuous range between the classically described “curly” and “straight” types, and that this continuum is well represented by a relatively simple measure of “straightness”. Furthermore, we find that IO neuron dendritic trees are often directionally oriented. Combined with an examination of cell body density distributions and dendritic orientation of adjacent IO neurons, our results suggest that the IO network may be organized into groups of densely coupled neurons interspersed with areas of weaker coupling.

Individual variability in the structural properties of neurons in the human inferior olive

The inferior olive (IO) is the sole source of the climbing fibers innervating the cerebellar cortex. We have previously shown both individual differences in the size and folding pattern of the principal nucleus (IOpr) in humans as well as in the expression of different proteins in IOpr neurons. This high degree of variability was not present in chimpanzee samples. The neurochemical differences might reflect static differences among individuals, but might also reflect age-related processes resulting in alterations of protein synthesis. Several observations support the latter idea. First, accumulation of lipofuscin, the "age pigment" is well documented in IOpr neurons. Second, there are silver- and abnormal tau-immunostained intraneuronal granules in IOpr neurons (Ikeda et al. Neurosci Lett 258:113-116, 1998). Finally, Olszewski and Baxter (Cytoarchitecture of the human brain stem, Second edn. Karger, Basel, 1954) observed an apparent loss of IOpr neurons in older individuals. We have further investigated the possibility of age-related changes in IOpr neurons using silver- and immunostained sections. We found silver-labeled intraneuronal granules in neurons of the IOpr in all human cases studied (n = 17, ages 25-71). We did not, however, confirm immunostaining with antibodies to abnormal tau. There was individual variability in the density of neurons as well as in the expression of the calcium-binding protein calretinin. In the chimpanzee, there were neither silver-stained intraneuronal granules nor irregularities in immunostaining. Overall, the data support the hypothesis that in some, but not all, humans there are functional changes in IOpr neurons and ultimately cell death. Neurochemical changes of IOpr neurons may contribute to age-related changes in motor and cognitive skills mediated by the cerebellum.

The Roles of the Olivocerebellar Pathway in Motor Learning and Motor Control. A Consensus Paper

For many decades, the predominant view in the cerebellar field has been that the olivocerebellar system's primary function is to induce plasticity in the cerebellar cortex, specifically, at the parallel fiber-Purkinje cell synapse. However, it has also long been proposed that the olivocerebellar system participates directly in motor control by helping to shape ongoing motor commands being issued by the cerebellum. Evidence consistent with both hypotheses exists; however, they are often investigated as mutually exclusive alternatives. In contrast, here, we take the perspective that the olivocerebellar system can contribute to both the motor learning and motor control functions of the cerebellum and might also play a role in development. We then consider the potential problems and benefits of it having multiple functions. Moreover, we discuss how its distinctive characteristics (e.g., low firing rates, synchronization, and variable complex spike waveforms) make it more or less suitable for one or the other of these functions, and why having multiple functions makes sense from an evolutionary perspective. We did not attempt to reach a consensus on the specific role(s) the olivocerebellar system plays in different types of movements, as that will ultimately be determined experimentally; however, collectively, the various contributions highlight the flexibility of the olivocerebellar system, and thereby suggest that it has the potential to act in both the motor learning and motor control functions of the cerebellum.

New insights into olivo-cerebellar circuits for learning from a small training sample

Artificial intelligence such as deep neural networks exhibited remarkable performance in simulated video games and 'Go'. In contrast, most humanoid robots in the DARPA Robotics Challenge fell down to ground. The dramatic contrast in performance is mainly due to differences in the amount of training data, which is huge and small, respectively. Animals are not allowed with millions of the failed trials, which lead to injury and death. Humans fall only several thousand times before they balance and walk. We hypothesize that a unique closed-loop neural circuit formed by the Purkinje cells, the cerebellar deep nucleus and the inferior olive in and around the cerebellum and the highest density of gap junctions, which regulate synchronous activities of the inferior olive nucleus, are computational machinery for learning from a small sample. We discuss recent experimental and computational advances associated with this hypothesis.

Chaotic Resonance in Coupled Inferior Olive Neurons with the Llinás Approach Neuron Model

It is well known that cerebellar motor control is fine-tuned by the learning process adjusted according to rich error signals from inferior olive (IO) neurons. Schweighofer and colleagues proposed that these signals can be produced by chaotic irregular firing in the IO neuron assembly; such chaotic resonance (CR) was replicated in their computer demonstration of a Hodgkin-Huxley (HH)-type compartment model. In this study, we examined the response of CR to a periodic signal in the IO neuron assembly comprising the Llinás approach IO neuron model. This system involves empirically observed dynamics of the IO membrane potential and is simpler than the HH-type compartment model. We then clarified its dependence on electrical coupling strength, input signal strength, and frequency. Furthermore, we compared the physiological validity for IO neurons such as low firing rate and sustaining subthreshold oscillation between CR and conventional stochastic resonance (SR) and examined the consistency with asynchronous firings indicated by the previous model-based studies in the cerebellar learning process. In addition, the signal response of CR and SR was investigated in a large neuron assembly. As the result, we confirmed that CR was consistent with the above IO neuron's characteristics, but it was not as easy for SR.

Activity-dependent plasticity of electrical synapses: increasing evidence for its presence and functional roles in the mammalian brain

Gap junctions mediate electrical synaptic transmission between neurons. While the actions of neurotransmitter modulators on the conductance of gap junctions have been extensively documented, increasing evidence indicates they can also be influenced by the ongoing activity of neural networks, in most cases via local interactions with nearby glutamatergic synapses. We review here early evidence for the existence of activity-dependent regulatory mechanisms as well recent examples reported in mammalian brain. The ubiquitous distribution of both neuronal connexins and the molecules involved suggest this phenomenon is widespread and represents a property of electrical transmission in general.

Neurología de la anticipación y sus implicaciones en el deporte

<p>El movimiento es una acción que involucra interconexiones complejas, por lo cual se requiere profundizar en los procesos de adaptación, predicción y anticipación que permiten entender la importancia de estos aspectos desde sus bases filogenéticas y ontogenéticas hasta su implicación en movimientos complejos. Parte de la optimización de los procesos descritos se haya en la calidad de información aferente, la cual permite la relación con el entorno —especialmente la entrada visual— que reconoce un flujo de imágenes y una proyección al contexto en el que se está inmerso. Las estructuras e interconexiones implicadas en la anticipación y predicción de movimientos son descritas de modo que se evidencia la congruencia y continuidad del flujo de información que caracteriza esta especialidad neuromecánica de movimiento. Por otro lado, se aborda la integración de centros puntuales del sistema nervioso central y redes neuronales que permiten el entramado de procesos de aprendizaje por observación, además de proveer equilibrio y eficiencia al sistema en la recepción de estímulos y su relación con la generación de eferencias motoras que cumplan con objetivos específicos. En el ámbito deportivo estos procesos favorecen la eficiencia del gesto optimizando el movimiento.</p>

Redefining cerebellar ataxia in degenerative ataxias: lessons from recent research on cerebellar systems

Recent advances in our understanding of neurophysiological functions in the cerebellar system have revealed that each region involved in degenerative ataxias contributes differently. To regulate voluntary movements, the cerebellum forms internal models within its neural circuits that mimic the behaviour of the sensorimotor system and objects in the external environment. The cerebellum forms two different internal models: forward and inverse. The forward model is formed by efference copy signals conveyed by the corticopontocerebellar system, and it derives the estimated consequences for action. The inverse model describes sequences of motor commands to accomplish an aim. During motor learning, we improve internal models by comparing the estimated consequence of an action from the forward model with the actual consequence of the action produced by the inverse model. The functions of the cerebellum encompass the formation, storage and selection of internal models. Considering the neurophysiological properties of the cerebellar system, we have classified degenerative ataxias into four types depending on which system is involved: Purkinje cells, the corticopontocerebellar system, the spinocerebellar system and the cerebellar deep nuclei. With regard to their respective contributions to the internal models, we speculate that loss of Purkinje cells leads to malformation of the internal models, whereas disturbance of the afferent system, corticopontocerebellar system or spinocerebellar system leads to mis-selection of the proper internal model. An understanding of the pathophysiological properties of ataxias in each degenerative ataxia enables the development of new methods to evaluate ataxias.

Coding of stimulus strength via analog calcium signals in Purkinje cell dendrites of awake mice

The climbing fiber input to Purkinje cells acts as a teaching signal by triggering a massive influx of dendritic calcium that marks the occurrence of instructive stimuli during cerebellar learning. Here, we challenge the view that these calcium spikes are all-or-none and only signal whether the instructive stimulus has occurred, without providing parametric information about its features. We imaged ensembles of Purkinje cell dendrites in awake mice and measured their calcium responses to periocular airpuffs that serve as instructive stimuli during cerebellar-dependent eyeblink conditioning. Information about airpuff duration and pressure was encoded probabilistically across repeated trials, and in two additional signals in single trials: the synchrony of calcium spikes in the Purkinje cell population, and the amplitude of the calcium spikes, which was modulated by a non-climbing fiber pathway. These results indicate that calcium-based teaching signals in Purkinje cells contain analog information that encodes the strength of instructive stimuli trial-by-trial.

DOI:http://dx.doi.org/10.7554/eLife.03663.001

A region of the brain known as the cerebellum plays a key role in learning how to anticipate an event. For example, if you know that a puff of air is going to be directed at your eye, it's a good idea to close it in advance. However, how much you need to close it depends on how strong that puff of air is. A very strong puff might require closing the eye completely to protect it. In contrast, it is probably better to only partially close the eye if you know a lighter puff of air is coming, so that you can still see.

Extensive research has focused on how neurons in and around the cerebellum work together to achieve this goal. When an event—such as a puff of air—occurs, signals are sent to large neurons in the cerebellum, called Purkinje cells, by ‘climbing fibers’. However, climbing fibers were thought to be able to respond in only two ways: either they fire in a single burst to signal that an event has occurred, or they don't fire. It was therefore unclear how the finer details of the event (for example, the strength of the puff of air) are transmitted to the cerebellum.

Najafi et al. imaged the level of calcium in the cerebellum of mice, as this indicates how active the neurons are. When a puff of air was directed at the eyes of the mice, Najafi et al. saw that the size of the response of the Purkinje cells corresponded with how big the puff of air was. Najafi et al. show that the size of this response, which is based mostly on input from the climbing fibers, is also influenced by input from an additional unknown source.

These findings show that Purkinje cells of the cerebellum receive detailed information about the nature of an event, such as a puff of air. What remains to be seen is whether the cerebellum uses this information to learn the correct response, that is how hard to blink to avoid the expected puff.

DOI:http://dx.doi.org/10.7554/eLife.03663.002

The ever-changing electrical synapse

A wealth of research has revealed that electrical synapses in the central nervous system exhibit a high degree of plasticity. Several recent studies, particularly in the retina and inferior olive, highlight this plasticity. Three classes of mechanisms can alter electrical coupling over time courses ranging from milliseconds to days. Changes of membrane conductance through synaptic input or spiking activity shunt current and decouple neurons on the millisecond time scale. Such activity can also alter coupling symmetry, rectifying electrical synapses. More stable rectification can be accomplished through molecular asymmetry of the synapse itself. On the minutes time scale, changes in connexin phosphorylation can change coupling quasi-stably with an order of magnitude dynamic range. On the hours to days time scale, changes in expression level of connexins alter coupling through the course of circadian time, over developmental time, or in response to tissue injury. Combined, all of these mechanisms allow electrical coupling to be highly dynamic, changing in response to demands at the whole network level, in small portions of a network, or at the level of an individual synapse.

Role of the olivo-cerebellar complex in motor learning and control

How is the cerebellum capable of efficient motor learning and control despite very low firing of the inferior olive (IO) inputs, which are postulated to carry errors needed for learning and contribute to on-line motor control? IO neurons form the largest electrically coupled network in the adult human brain. Here, we discuss how intermediate coupling strengths can lead to chaotic resonance and increase information transmission of the error signal despite the very low IO firing rate. This increased information transmission can then lead to more efficient learning than with weak or strong coupling. In addition, we argue that a dynamic modulation of IO electrical coupling via the Purkinje cell-deep cerebellar neurons – IO triangle could speed up learning and improve on-line control. Initially strong coupling would allow transmission of large errors to multiple functionally related Purkinje cells, resulting in fast but coarse learning as well as significant effects on deep cerebellar nucleus and on-line motor control. In the late phase of learning decreased coupling would allow desynchronized IO firing, allowing high-fidelity transmission of error, resulting in slower but fine learning, and little on-line motor control effects.