New insights into olivo-cerebellar circuits for learning from a small training sample

Dynamic organization of cerebellar climbing fiber response and synchrony in multiple functional components reduces dimensions for reinforcement learning

Cerebellar climbing fibers convey diverse signals, but how they are organized in the compartmental structure of the cerebellar cortex during learning remains largely unclear. We analyzed a large amount of coordinate-localized two-photon imaging data from cerebellar Crus II in mice undergoing 'Go/No-go' reinforcement learning. Tensor component analysis revealed that a majority of climbing fiber inputs to Purkinje cells were reduced to only four functional components, corresponding to accurate timing control of motor initiation related to a Go cue, cognitive error-based learning, reward processing, and inhibition of erroneous behaviors after a No-go cue. Changes in neural activities during learning of the first two components were correlated with corresponding changes in timing control and error learning across animals, indirectly suggesting causal relationships. Spatial distribution of these components coincided well with boundaries of Aldolase-C/zebrin II expression in Purkinje cells, whereas several components are mixed in single neurons. Synchronization within individual components was bidirectionally regulated according to specific task contexts and learning stages. These findings suggest that, in close collaborations with other brain regions including the inferior olive nucleus, the cerebellum, based on anatomical compartments, reduces dimensions of the learning space by dynamically organizing multiple functional components, a feature that may inspire new-generation AI designs.

C1ql1-Bai3 signaling is necessary for climbing fiber synapse formation in mature Purkinje cells in coordination with neuronal activity

Changes in neural activity induced by learning and novel environments have been reported to lead to the formation of new synapses in the adult brain. However, the underlying molecular mechanism is not well understood. Here, we show that Purkinje cells (PCs), which have established adult-type monosynaptic innervation by climbing fibers (CFs) after elimination of weak CFs during development, can be reinnervated by multiple CFs by increased expression of the synaptic organizer C1ql1 in CFs or Bai3, a receptor for C1ql1, in PCs. In the adult cerebellum, CFs are known to have transverse branches that run in a mediolateral direction without forming synapses with PCs. Electrophysiological, Ca²⁺-imaging and immunohistochemical studies showed that overexpression of C1ql1 or Bai3 caused these CF transverse branches to elongate and synapse on the distal dendrites of mature PCs. Mature PCs were also reinnervated by multiple CFs when the glutamate receptor GluD2, which is essential for the maintenance of synapses between granule cells and PCs, was deleted. Interestingly, the effect of GluD2 knockout was not observed in Bai3 knockout PCs. In addition, C1ql1 levels were significantly upregulated in CFs of GluD2 knockout mice, suggesting that endogenous, not overexpressed, C1ql1-Bai3 signaling could regulate the reinnervation of mature PCs by CFs. Furthermore, the effects of C1ql1 and Bai3 overexpression required neuronal activity in the PC and CF, respectively. C1ql1 immunoreactivity at CF-PC synapses was reduced when the neuronal activity of CFs was suppressed. These results suggest that C1ql1-Bai3 signaling may mediate CF synaptogenesis in mature PCs, potentially in concert with neuronal activity.

Chaos may enhance expressivity in cerebellar granular layer

Recent evidence suggests that Golgi cells in the cerebellar granular layer are densely connected to each other with massive gap junctions. Here, we propose that the massive gap junctions between the Golgi cells contribute to the representational complexity of the granular layer of the cerebellum by inducing chaotic dynamics. We construct a model of cerebellar granular layer with diffusion coupling through gap junctions between the Golgi cells, and evaluate the representational capability of the network with the reservoir computing framework. First, we show that the chaotic dynamics induced by diffusion coupling results in complex output patterns containing a wide range of frequency components. Second, the long non-recursive time series of the reservoir represents the passage of time from an external input. These properties of the reservoir enable mapping different spatial inputs into different temporal patterns.

Unconscious reinforcement learning of hidden brain states supported by confidence

Can humans be trained to make strategic use of latent representations in their own brains? We investigate how human subjects can derive reward-maximizing choices from intrinsic high-dimensional information represented stochastically in neural activity. Reward contingencies are defined in real-time by fMRI multivoxel patterns; optimal action policies thereby depend on multidimensional brain activity taking place below the threshold of consciousness, by design. We find that subjects can solve the task within two hundred trials and errors, as their reinforcement learning processes interact with metacognitive functions (quantified as the meaningfulness of their decision confidence). Computational modelling and multivariate analyses identify a frontostriatal neural mechanism by which the brain may untangle the 'curse of dimensionality': synchronization of confidence representations in prefrontal cortex with reward prediction errors in basal ganglia support exploration of latent task representations. These results may provide an alternative starting point for future investigations into unconscious learning and functions of metacognition.

Electrical coupling controls dimensionality and chaotic firing of inferior olive neurons

We previously proposed, on theoretical grounds, that the cerebellum must regulate the dimensionality of its neuronal activity during motor learning and control to cope with the low firing frequency of inferior olive neurons, which form one of two major inputs to the cerebellar cortex. Such dimensionality regulation is possible via modulation of electrical coupling through the gap junctions between inferior olive neurons by inhibitory GABAergic synapses. In addition, we previously showed in simulations that intermediate coupling strengths induce chaotic firing of inferior olive neurons and increase their information carrying capacity. However, there is no in vivo experimental data supporting these two theoretical predictions. Here, we computed the levels of synchrony, dimensionality, and chaos of the inferior olive code by analyzing in vivo recordings of Purkinje cell complex spike activity in three different coupling conditions: carbenoxolone (gap junctions blocker), control, and picrotoxin (GABA-A receptor antagonist). To examine the effect of electrical coupling on dimensionality and chaotic dynamics, we first determined the physiological range of effective coupling strengths between inferior olive neurons in the three conditions using a combination of a biophysical network model of the inferior olive and a novel Bayesian model averaging approach. We found that effective coupling co-varied with synchrony and was inversely related to the dimensionality of inferior olive firing dynamics, as measured via a principal component analysis of the spike trains in each condition. Furthermore, for both the model and the data, we found an inverted U-shaped relationship between coupling strengths and complexity entropy, a measure of chaos for spiking neural data. These results are consistent with our hypothesis according to which electrical coupling regulates the dimensionality and the complexity in the inferior olive neurons in order to optimize both motor learning and control of high dimensional motor systems by the cerebellum.

Computational theory suggests that the cerebellum must decrease the dimensionality of its neuronal activity to learn and control high dimensional motor systems effectively, while being constrained by the low firing frequency of inferior olive neurons, one of the two major source of input signals to the cerebellum. We previously proposed that the cerebellum adaptively controls the dimensionality of inferior olive firing by adjusting the level of synchrony and that such control is made possible by modulating the electrical coupling strength between inferior olive neurons. Here, we developed a novel method that uses a biophysical model of the inferior olive to accurately estimate the effective coupling strengths between inferior olive neurons from in vivo recordings of spike activity in three different coupling conditions. We found that high coupling strengths induce synchronous firing and decrease the dimensionality of inferior olive firing dynamics. In contrast, intermediate coupling strengths lead to chaotic firing and increase the dimensionality of the firing dynamics. Thus, electrical coupling is a feasible mechanism to control dimensionality and chaotic firing of inferior olive neurons. In sum, our results provide insights into possible mechanisms underlying cerebellar function and, in general, a biologically plausible framework to control the dimensionality of neural coding.

50 Years Since the Marr, Ito, and Albus Models of the Cerebellum

Fifty years have passed since David Marr, Masao Ito, and James Albus proposed seminal models of cerebellar functions. These models share the essential concept that parallel-fiber-Purkinje-cell synapses undergo plastic changes, guided by climbing-fiber activities during sensorimotor learning. However, they differ in several important respects, including holistic versus complementary roles of the cerebellum, pattern recognition versus control as computational objectives, potentiation versus depression of synaptic plasticity, teaching signals versus error signals transmitted by climbing-fibers, sparse expansion coding by granule cells, and cerebellar internal models. In this review, we evaluate different features of the three models based on recent computational and experimental studies. While acknowledging that the three models have greatly advanced our understanding of cerebellar control mechanisms in eye movements and classical conditioning, we propose a new direction for computational frameworks of the cerebellum, that is, hierarchical reinforcement learning with multiple internal models.

A practical method for estimating coupling functions in complex dynamical systems

A foremost challenge in modern network science is the inverse problem of reconstruction (inference) of coupling equations and network topology from the measurements of the network dynamics. Of particular interest are the methods that can operate on real (empirical) data without interfering with the system. One such earlier attempt (Tokuda et al. 2007 Phys. Rev. Lett. 99, 064101. (doi:10.1103/PhysRevLett.99.064101)) was a method suited for general limit-cycle oscillators, yielding both oscillators' natural frequencies and coupling functions between them (phase equations) from empirically measured time series. The present paper reviews the above method in a way comprehensive to domain-scientists other than physics. It also presents applications of the method to (i) detection of the network connectivity, (ii) inference of the phase sensitivity function, (iii) approximation of the interaction among phase-coherent chaotic oscillators, and (iv) experimental data from a forced Van der Pol electric circuit. This reaffirms the range of applicability of the method for reconstructing coupling functions and makes it accessible to a much wider scientific community. This article is part of the theme issue 'Coupling functions: dynamical interaction mechanisms in the physical, biological and social sciences'.

A Biomimetic Control Method Increases the Adaptability of a Humanoid Robot Acting in a Dynamic Environment

One of the big challenges in robotics is to endow agents with autonomous and adaptive capabilities. With this purpose, we embedded a cerebellum-based control system into a humanoid robot that becomes capable of handling dynamical external and internal complexity. The cerebellum is the area of the brain that coordinates and predicts the body movements throughout the body-environment interactions. Different biologically plausible cerebellar models are available in literature and have been employed for motor learning and control of simplified objects. We built the canonical cerebellar microcircuit by combining machine learning and computational neuroscience techniques. The control system is composed of the adaptive cerebellar module and a classic control method; their combination allows a fast adaptive learning and robust control of the robotic movements when external disturbances appear. The control structure is built offline, but the dynamic parameters are learned during an online-phase training. The aforementioned adaptive control system has been tested in the Neuro-robotics Platform with the virtual humanoid robot iCub. In the experiment, the robot iCub has to balance with the hand a table with a ball running on it. In contrast with previous attempts of solving this task, the proposed neural controller resulted able to quickly adapt when the internal and external conditions change. Our bio-inspired and flexible control architecture can be applied to different robotic configurations without an excessive tuning of the parameters or customization. The cerebellum-based control system is indeed able to deal with changing dynamics and interactions with the environment. Important insights regarding the relationship between the bio-inspired control system functioning and the complexity of the task to be performed are obtained.

Corticotropin-releasing factor depolarizes rat lateral vestibular nuclear neurons through activation of CRF receptors 1 and 2

Corticotropin-releasing factor (CRF) is a neuropeptide mainly synthesized in the hypothalamic paraventricular nucleus and has been traditionally implicated in stress and anxiety. Intriguingly, genetic or pharmacological manipulation of CRF receptors affects locomotor activity as well as motor coordination and balance in rodents, suggesting an active involvement of the central CRFergic system in motor control. Yet little is known about the exact role of CRF in central motor structures and the underlying mechanisms. Therefore, in the present study, we focused on the effect of CRF on the lateral vestibular nucleus (LVN) in the brainstem vestibular nuclear complex, an important center directly contributing to adjustment of muscle tone for both postural maintenance and the alternative change from the extensor to the flexor phase during locomotion. The results show that CRF depolarizes and increases the firing rate of neurons in the LVN. Tetrodotoxin does not block the CRF-induced depolarization and inward current on LVN neurons, suggesting a direct postsynaptic action of the neuropeptide. The CRF-induced depolarization on LVN neurons was partly blocked by antalarmin or antisauvagine-30, selective antagonists for CRF receptors 1 (CRFR1) and 2 (CRFR2), respectively. Furthermore, combined application of antalarmin and antisauvagine-30 totally abolished the CRF-induced depolarization. Immunofluorescence results show that CRFR1 and CRFR2 are co-localized in the rat LVN. These results demonstrate that CRF excites the LVN neurons by co-activation of both CRFR1 and CRFR2, suggesting that via the direct modulation on the LVN, the central CRFergic system may actively participate in the central vestibular-mediated postural and motor control.

The neural and cognitive architecture for learning from a small sample

Artificial intelligence algorithms are capable of fantastic exploits, yet they are still grossly inefficient compared with the brain's ability to learn from few exemplars or solve problems that have not been explicitly defined. What is the secret that the evolution of human intelligence has unlocked? Generalization is one answer, but there is more to it. The brain does not directly solve difficult problems, it is able to recast them into new and more tractable problems. Here, we propose a model whereby higher cognitive functions profoundly interact with reinforcement learning to drastically reduce the degrees of freedom of the search space, simplifying complex problems, and fostering more efficient learning.

Controlling Chaotic Resonance using External Feedback Signals in Neural Systems

Stochastic resonance is a phenomenon in which the signal response of a non-linear system is enhanced by appropriate external noise. Likewise, a similar phenomenon can be caused by deterministic chaos; this is called chaotic resonance. Devices that employ stochastic resonance have been proposed for the purpose of enhancing tactile sensitivity. However, no applications of chaotic resonance have been reported so far, even though chaotic resonance exhibits a higher sensitivity than stochastic resonance. This contrast in applications could be attributed to the fact that chaotic resonance is induced by adjusting internal parameters. In many cases, especially in biological systems, these parameters are difficult to adjust. In this study, by applying our proposed reduced region of orbit method to a neural system consisting of excitatory and inhibitory neurons, we induce chaotic resonance with signal frequency dependency against weak input signals. Furthermore, the external noise exhibits effects for both diminishing and enhancing signal responses in chaotic resonance. The outcome of this study might facilitate the development of devices utilising the mechanism of chaotic resonance.