Adaptive robotic control driven by a versatile spiking cerebellar network

Mesoscale simulations predict the role of synergistic cerebellar plasticity during classical eyeblink conditioning

According to the motor learning theory by Albus and Ito, synaptic depression at the parallel fibre to Purkinje cells synapse (pf-PC) is the main substrate responsible for learning sensorimotor contingencies under climbing fibre control. However, recent experimental evidence challenges this relatively monopolistic view of cerebellar learning. Bidirectional plasticity appears crucial for learning, in which different microzones can undergo opposite changes of synaptic strength (e.g. downbound microzones-more likely depression, upbound microzones-more likely potentiation), and multiple forms of plasticity have been identified, distributed over different cerebellar circuit synapses. Here, we have simulated classical eyeblink conditioning (CEBC) using an advanced spiking cerebellar model embedding downbound and upbound modules that are subject to multiple plasticity rules. Simulations indicate that synaptic plasticity regulates the cascade of precise spiking patterns spreading throughout the cerebellar cortex and cerebellar nuclei. CEBC was supported by plasticity at the pf-PC synapses as well as at the synapses of the molecular layer interneurons (MLIs), but only the combined switch-off of both sites of plasticity compromised learning significantly. By differentially engaging climbing fibre information and related forms of synaptic plasticity, both microzones contributed to generate a well-timed conditioned response, but it was the downbound module that played the major role in this process. The outcomes of our simulations closely align with the behavioural and electrophysiological phenotypes of mutant mice suffering from cell-specific mutations that affect processing of their PC and/or MLI synapses. Our data highlight that a synergy of bidirectional plasticity rules distributed across the cerebellum can facilitate finetuning of adaptive associative behaviours at a high spatiotemporal resolution.

Human Brain Project Partnering Projects Meeting: Status Quo and Outlook

As the European Flagship Human Brain Project (HBP) ends in September 2023, a meeting dedicated to the Partnering Projects (PPs), a collective of independent research groups that partnered with the HBP, was held on September 4-7, 2022. The purpose of this meeting was to allow these groups to present their results, reflect on their collaboration with the HBP and discuss future interactions with the European Research Infrastructure (RI) EBRAINS that has emerged from the HBP. In this report, we share the tour-de-force that the Partnering Projects that were present in the meeting have made in furthering knowledge concerning various aspects of Brain Research with the HBP. We describe briefly major achievements of the HBP Partnering Projects in terms of a systems-level understanding of the functional architecture of the brain and its possible emulation in artificial systems. We then recapitulate open discussions with EBRAINS representatives about the evolution of EBRAINS as a sustainable Research Infrastructure for the Partnering Projects after the HBP, and also for the wider scientific community.

The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

A cerebellum inspired spiking neural network as a multi-model for pattern classification and robotic trajectory prediction

Spiking neural networks were introduced to understand spatiotemporal information processing in neurons and have found their application in pattern encoding, data discrimination, and classification. Bioinspired network architectures are considered for event-driven tasks, and scientists have looked at different theories based on the architecture and functioning. Motor tasks, for example, have networks inspired by cerebellar architecture where the granular layer recodes sparse representations of the mossy fiber (MF) inputs and has more roles in motor learning. Using abstractions from cerebellar connections and learning rules of deep learning network (DLN), patterns were discriminated within datasets, and the same algorithm was used for trajectory optimization. In the current work, a cerebellum-inspired spiking neural network with dynamics of cerebellar neurons and learning mechanisms attributed to the granular layer, Purkinje cell (PC) layer, and cerebellar nuclei interconnected by excitatory and inhibitory synapses was implemented. The model’s pattern discrimination capability was tested for two tasks on standard machine learning (ML) datasets and on following a trajectory of a low-cost sensor-free robotic articulator. Tuned for supervised learning, the pattern classification capability of the cerebellum-inspired network algorithm has produced more generalized models than data-specific precision models on smaller training datasets. The model showed an accuracy of 72%, which was comparable to standard ML algorithms, such as MLP (78%), Dl4jMlpClassifier (64%), RBFNetwork (71.4%), and libSVM-linear (85.7%). The cerebellar model increased the network’s capability and decreased storage, augmenting faster computations. Additionally, the network model could also implicitly reconstruct the trajectory of a 6-degree of freedom (DOF) robotic arm with a low error rate by reconstructing the kinematic parameters. The variability between the actual and predicted trajectory points was noted to be ± 3 cm (while moving to a position in a cuboid space of 25 × 30 × 40 cm). Although a few known learning rules were implemented among known types of plasticity in the cerebellum, the network model showed a generalized processing capability for a range of signals, modulating the data through the interconnected neural populations. In addition to potential use on sensor-free or feed-forward based controllers for robotic arms and as a generalized pattern classification algorithm, this model adds implications to motor learning theory.

Dual STDP processes at Purkinje cells contribute to distinct improvements in accuracy and speed of saccadic eye movements

Saccadic eye-movements play a crucial role in visuo-motor control by allowing rapid foveation onto new targets. However, the neural processes governing saccades adaptation are not fully understood. Saccades, due to the short-time of execution (20-100 ms) and the absence of sensory information for online feedback control, must be controlled in a ballistic manner. Incomplete measurements of the movement trajectory, such as the visual endpoint error, are supposedly used to form internal predictions about the movement kinematics resulting in predictive control. In order to characterize the synaptic and neural circuit mechanisms underlying predictive saccadic control, we have reconstructed the saccadic system in a digital controller embedding a spiking neural network of the cerebellum with spike timing-dependent plasticity (STDP) rules driving parallel fiber-Purkinje cell long-term potentiation and depression (LTP and LTD). This model implements a control policy based on a dual plasticity mechanism, resulting in the identification of the roles of LTP and LTD in regulating the overall quality of saccade kinematics: it turns out that LTD increases the accuracy by decreasing visual error and LTP increases the peak speed. The control policy also required cerebellar PCs to be divided into two subpopulations, characterized by burst or pause responses. To our knowledge, this is the first model that explains in mechanistic terms the visual error and peak speed regulation of ballistic eye movements in forward mode exploiting spike-timing to regulate firing in different populations of the neuronal network. This elementary model of saccades could be extended and applied to other more complex cases in which single jerks are concatenated to compose articulated and coordinated movements.

The quest for multiscale brain modeling

Addressing the multiscale organization of the brain, which is fundamental to the dynamic repertoire of the organ, remains challenging. In principle, it should be possible to model neurons and synapses in detail and then connect them into large neuronal assemblies to explain the relationship between microscopic phenomena, large-scale brain functions, and behavior. It is more difficult to infer neuronal functions from ensemble measurements such as those currently obtained with brain activity recordings. In this article we consider theories and strategies for combining bottom-up models, generated from principles of neuronal biophysics, with top-down models based on ensemble representations of network activity and on functional principles. These integrative approaches are hoped to provide effective multiscale simulations in virtual brains and neurorobots, and pave the way to future applications in medicine and information technologies.

Bayesian Integration in a Spiking Neural System for Sensorimotor Control

The brain continuously estimates the state of body and environment, with specific regions that are thought to act as Bayesian estimator, optimally integrating noisy and delayed sensory feedback with sensory predictions generated by the cerebellum. In control theory, Bayesian estimators are usually implemented using high-level representations. In this work, we designed a new spike-based computational model of a Bayesian estimator. The state estimator receives spiking activity from two neural populations encoding the sensory feedback and the cerebellar prediction, and it continuously computes the spike variability within each population as a reliability index of the signal these populations encode. The state estimator output encodes the current state estimate. We simulated a reaching task at different stages of cerebellar learning. The activity of the sensory feedback neurons encoded a noisy version of the trajectory after actual movement, with an almost constant intrapopulation spiking variability. Conversely, the activity of the cerebellar output neurons depended on the phase of the learning process. Before learning, they fired at their baseline not encoding any relevant information, and the variability was set to be higher than that of the sensory feedback (more reliable, albeit delayed). When learning was complete, their activity encoded the trajectory before the actual execution, providing an accurate sensory prediction; in this case, the variability was set to be lower than that of the sensory feedback. The state estimator model optimally integrated the neural activities of the afferent populations, so that the output state estimate was primarily driven by sensory feedback in prelearning and by the cerebellar prediction in postlearning. It was able to deal even with more complex scenarios, for example, by shifting the dominant source during the movement execution if information availability suddenly changed. The proposed tool will be a critical block within integrated spiking, brain-inspired control systems for simulations of sensorimotor tasks.

Cerebellum Involvement in Dystonia During Associative Motor Learning: Insights From a Data-Driven Spiking Network Model

Dystonia is a movement disorder characterized by sustained or intermittent muscle contractions causing abnormal, often repetitive movements, postures, or both. Although dystonia is traditionally associated with basal ganglia dysfunction, recent evidence has been pointing to a role of the cerebellum, a brain area involved in motor control and learning. Cerebellar abnormalities have been correlated with dystonia but their potential causative role remains elusive. Here, we simulated the cerebellar input-output relationship with high-resolution computational modeling. We used a data-driven cerebellar Spiking Neural Network and simulated a cerebellum-driven associative learning task, Eye-Blink Classical Conditioning (EBCC), which is characteristically altered in relation to cerebellar lesions in several pathologies. In control simulations, input stimuli entrained characteristic network dynamics and induced synaptic plasticity along task repetitions, causing a progressive spike suppression in Purkinje cells with consequent facilitation of deep cerebellar nuclei cells. These neuronal processes caused a progressive acquisition of eyelid Conditioned Responses (CRs). Then, we modified structural or functional local neural features in the network reproducing alterations reported in dystonic mice. Either reduced olivocerebellar input or aberrant Purkinje cell burst-firing resulted in abnormal learning curves imitating the dysfunctional EBCC motor responses (in terms of CR amount and timing) of dystonic mice. These behavioral deficits might be due to altered temporal processing of sensorimotor information and uncoordinated control of muscle contractions. Conversely, an imbalance of excitatory and inhibitory synaptic densities on Purkinje cells did not reflect into significant EBCC deficit. The present work suggests that only certain types of alterations, including reduced olivocerebellar input and aberrant PC burst-firing, are compatible with the EBCC changes observed in dystonia, indicating that some cerebellar lesions can have a causative role in the pathogenesis of symptoms.

Brain-Inspired Spiking Neural Network Controller for a Neurorobotic Whisker System

It is common for animals to use self-generated movements to actively sense the surrounding environment. For instance, rodents rhythmically move their whiskers to explore the space close to their body. The mouse whisker system has become a standard model for studying active sensing and sensorimotor integration through feedback loops. In this work, we developed a bioinspired spiking neural network model of the sensorimotor peripheral whisker system, modeling trigeminal ganglion, trigeminal nuclei, facial nuclei, and central pattern generator neuronal populations. This network was embedded in a virtual mouse robot, exploiting the Human Brain Project's Neurorobotics Platform, a simulation platform offering a virtual environment to develop and test robots driven by brain-inspired controllers. Eventually, the peripheral whisker system was adequately connected to an adaptive cerebellar network controller. The whole system was able to drive active whisking with learning capability, matching neural correlates of behavior experimentally recorded in mice.

A review of motor neural system robotic modeling approaches and instruments

In this review, we are considering an actively developing tool in neuroscience-robotic modeling. The new perspective and existing application fields, tools, and methods are discussed. We try to determine starting positions and approaches that are useful at the beginning of new research in this field. Among multiple directions of the research is robotic modeling on the level of muscles fibers and their afferents, skin surface sensors, muscles, and joints proprioceptors. Some examples of technical implementation for physical modeling are reviewed. They are software and hardware tools like event-related modeling algorithms, reduced neuron models, robotic drives constructions. We observe existing drives technologies and prospective electric motor types: switched reluctance and transverse flux motors. Next, we look at the existing examples and approaches for robotic modeling of the cerebellum and spinal cord neural networks. These examples show practical methods for the model neural network architecture and adaptation. Those methods allow the use of cortical and spinal cord reflexes for the network training and apply additional artificial blocks for data processing in other brain structures that transmit and receive data from biologically realistic models.

An experimental comparison of different hierarchical self-tuning regulatory control procedures for under-actuated mechatronic systems

This paper presents an experimental comparison of four different hierarchical self-tuning regulatory control procedures in enhancing the robustness of the under-actuated systems against bounded exogenous disturbances. The proposed hierarchical control procedure augments the ubiquitous Linear-Quadratic-Regulator (LQR) with an online reconfiguration block that acts as a superior regulator to dynamically adjust the critical weighting-factors of LQR’s quadratic-performance-index (QPI). The Algebraic-Riccati-Equation (ARE) uses these updated weighting-factors to re-compute the optimal control problem, after every sampling interval, to deliver time-varying state-feedback gains. This article experimentally compares four state-of-the-art rule-based online adaptation mechanisms that dynamically restructure the constituent blocks of the ARE. The proposed hierarchical control procedures are synthesized by self-adjusting the (i) controller’s degree-of-stability, (ii) the control-weighting-factor of QPI, (iii) the state-weighting-factors of QPI as a function of “state-error-phases”, and (iv) the state-weighting-factors of QPI as a function of “state-error-magnitudes”. Each adaptation mechanism is formulated via pre-calibrated hyperbolic scaling functions that are driven by state-error-variations. The implications of each mechanism on the controller’s behaviour are analyzed in real-time by conducting credible hardware-in-the-loop experiments on the QNET Rotary-Pendulum setup. The rotary pendulum is chosen as the benchmark platform owing to its under-actuated configuration and kinematic instability. The experimental outcomes indicate that the latter self-adaptive controller demonstrates superior adaptability and disturbances-rejection capability throughout the operating regime.

A multizone cerebellar chip for bioinspired adaptive robot control and sensorimotor processing

The cerebellum is a neural structure essential for learning, which is connected via multiple zones to many different regions of the brain, and is thought to improve human performance in a large range of sensory, motor and even cognitive processing tasks. An intriguing possibility for the control of complex robotic systems would be to develop an artificial cerebellar chip with multiple zones that could be similarly connected to a variety of subsystems to optimize performance. The novel aim of this paper, therefore, is to propose and investigate a multizone cerebellar chip applied to a range of tasks in robot adaptive control and sensorimotor processing. The multizone cerebellar chip was evaluated using a custom robotic platform consisting of an array of tactile sensors driven by dielectric electroactive polymers mounted upon a standard industrial robot arm. The results demonstrate that the performance in each task was improved by the concurrent, stable learning in each cerebellar zone. This paper, therefore, provides the first empirical demonstration that a synthetic, multizone, cerebellar chip could be embodied within existing robotic systems to improve performance in a diverse range of tasks, much like the cerebellum in a biological system.

How the Cerebellum and Prefrontal Cortex Cooperate During Trace Eyeblinking Conditioning

Several data have demonstrated that during the widely used experimental paradigm for studying associative learning, trace eye blinking conditioning (TEBC), there is a strong interaction between cerebellum and medial prefrontal cortex (mPFC). Despite this evidence, the neural mechanisms underlying this interaction are still not clear. Here, we propose a neurophysiologically plausible computational model to address this issue. The model is constrained on the basis of two critical anatomo-physiological features: (i) the cerebello-cortical organization through two circuits, respectively, targeting M1 and mPFC; (ii) the different timing in the plasticity mechanisms of these parallel circuits produced by the granule cells time sensitivity according to which different subpopulations are active at different moments during conditioned stimuli. The computer simulations run with the model suggest that these features are critical to understand how the cooperation between cerebellum and mPFC supports motor areas during TEBC. In particular, a greater trace interval produces greater plasticity changes at the slow path synapses involving mPFC with respect to plasticity changes at the fast path involving M1. As a consequence, the greater is the trace interval, the stronger is the mPFC involvement. The model has been validated by reproducing data collected through recent real mice experiments.

Implementation of an Advanced Frequency-Based Hebbian Spike Timing Dependent Plasticity

The brain is provided with an enormous computing capability and exploits neural plasticity to store and elaborate complex information. One of the multiple mechanisms that neural circuits express is the Spike-timing-dependent plasticity (STDP), a form of long-term synaptic plasticity exploiting the time relationship between pre- and post-synaptic action potentials (i.e., neuron spikes). It has been found that in certain cases, for instance at the input stage of the cerebellum, between mossy fibers and granular neurons, the plasticity is not only driven by the timing of the spikes, but also by the oscillation frequency of the inputs. This complex behaviour has been implemented in this work, where we developed a novel form of advanced synaptic plasticity model to be used in a well-established neural network simulator (NEST). The subsequent tests proved the proper functioning of the plasticity and its range of applicability, demonstrating the possibility to adopt it in noisy and variable conditions, similar to the biological settings.

A Cerebellum-Inspired Learning Approach for Adaptive and Anticipatory Control

The cerebellum, which is responsible for motor control and learning, has been suggested to act as a Smith predictor for compensation of time-delays by means of internal forward models. However, insights about how forward model predictions are integrated in the Smith predictor have not yet been unveiled. To fill this gap, a novel bio-inspired modular control architecture that merges a recurrent cerebellar-like loop for adaptive control and a Smith predictor controller is proposed. The goal is to provide accurate anticipatory corrections to the generation of the motor commands in spite of sensory delays and to validate the robustness of the proposed control method to input and physical dynamic changes. The outcome of the proposed architecture with other two control schemes that do not include the Smith control strategy or the cerebellar-like corrections are compared. The results obtained on four sets of experiments confirm that the cerebellum-like circuit provides more effective corrections when only the Smith strategy is adopted and that minor tuning in the parameters, fast adaptation and reproducible configuration are enabled.

Response Dynamics in an Olivocerebellar Spiking Neural Network With Non-linear Neuron Properties

Sensorimotor signals are integrated and processed by the cerebellar circuit to predict accurate control of actions. In order to investigate how single neuron dynamics and geometrical modular connectivity affect cerebellar processing, we have built an olivocerebellar Spiking Neural Network (SNN) based on a novel simplification algorithm for single point models (Extended Generalized Leaky Integrate and Fire, EGLIF) capturing essential non-linear neuronal dynamics (e.g., pacemaking, bursting, adaptation, oscillation and resonance). EGLIF models specifically tuned for each neuron type were embedded into an olivocerebellar scaffold reproducing realistic spatial organization and physiological convergence and divergence ratios of connections. In order to emulate the circuit involved in an eye blink response to two associated stimuli, we modeled two adjacent olivocerebellar microcomplexes with a common mossy fiber input but different climbing fiber inputs (either on or off). EGLIF-SNN model simulations revealed the emergence of fundamental response properties in Purkinje cells (burst-pause) and deep nuclei cells (pause-burst) similar to those reported in vivo. The expression of these properties depended on the specific activation of climbing fibers in the microcomplexes and did not emerge with scaffold models using simplified point neurons. This result supports the importance of embedding SNNs with realistic neuronal dynamics and appropriate connectivity and anticipates the scale-up of EGLIF-SNN and the embedding of plasticity rules required to investigate cerebellar functioning at multiple scales.

Real-time cerebellar neuroprosthetic system based on a spiking neural network model of motor learning

OBJECTIVE

Damage to the brain, as a result of various medical conditions, impacts the everyday life of patients and there is still no complete cure to neurological disorders. Neuroprostheses that can functionally replace the damaged neural circuit have recently emerged as a possible solution to these problems. Here we describe the development of a real-time cerebellar neuroprosthetic system to substitute neural function in cerebellar circuitry for learning delay eyeblink conditioning (DEC).

APPROACH

The system was empowered by a biologically realistic spiking neural network (SNN) model of the cerebellar neural circuit, which considers the neuronal population and anatomical connectivity of the network. The model simulated synaptic plasticity critical for learning DEC. This SNN model was carefully implemented on a field programmable gate array (FPGA) platform for real-time simulation. This hardware system was interfaced in in vivo experiments with anesthetized rats and it used neural spikes recorded online from the animal to learn and trigger conditioned eyeblink in the animal during training.

MAIN RESULTS

This rat-FPGA hybrid system was able to process neuronal spikes in real-time with an embedded cerebellum model of ~10 000 neurons and reproduce learning of DEC with different inter-stimulus intervals. Our results validated that the system performance is physiologically relevant at both the neural (firing pattern) and behavioral (eyeblink pattern) levels.

SIGNIFICANCE

This integrated system provides the sufficient computation power for mimicking the cerebellar circuit in real-time. The system interacts with the biological system naturally at the spike level and can be generalized for including other neural components (neuron types and plasticity) and neural functions for potential neuroprosthetic applications.

A Biomimetic Control Method Increases the Adaptability of a Humanoid Robot Acting in a Dynamic Environment

One of the big challenges in robotics is to endow agents with autonomous and adaptive capabilities. With this purpose, we embedded a cerebellum-based control system into a humanoid robot that becomes capable of handling dynamical external and internal complexity. The cerebellum is the area of the brain that coordinates and predicts the body movements throughout the body-environment interactions. Different biologically plausible cerebellar models are available in literature and have been employed for motor learning and control of simplified objects. We built the canonical cerebellar microcircuit by combining machine learning and computational neuroscience techniques. The control system is composed of the adaptive cerebellar module and a classic control method; their combination allows a fast adaptive learning and robust control of the robotic movements when external disturbances appear. The control structure is built offline, but the dynamic parameters are learned during an online-phase training. The aforementioned adaptive control system has been tested in the Neuro-robotics Platform with the virtual humanoid robot iCub. In the experiment, the robot iCub has to balance with the hand a table with a ball running on it. In contrast with previous attempts of solving this task, the proposed neural controller resulted able to quickly adapt when the internal and external conditions change. Our bio-inspired and flexible control architecture can be applied to different robotic configurations without an excessive tuning of the parameters or customization. The cerebellum-based control system is indeed able to deal with changing dynamics and interactions with the environment. Important insights regarding the relationship between the bio-inspired control system functioning and the complexity of the task to be performed are obtained.

Long-Lasting Response Changes in Deep Cerebellar Nuclei in vivo Correlate With Low-Frequency Oscillations

The deep cerebellar nuclei (DCN) have been suggested to play a critical role in sensorimotor learning and some forms of long-term synaptic plasticity observed in vitro have been proposed as a possible substrate. However, till now it was not clear whether and how DCN neuron responses manifest long-lasting changes in vivo. Here, we have characterized DCN unit responses to tactile stimulation of the facial area in anesthetized mice and evaluated the changes induced by theta-sensory stimulation (TSS), a 4 Hz stimulation pattern that is known to induce plasticity in the cerebellar cortex in vivo. DCN units responded to tactile stimulation generating bursts and pauses, which reflected combinations of excitatory inputs most likely relayed by mossy fiber collaterals, inhibitory inputs relayed by Purkinje cells, and intrinsic rebound firing. Interestingly, initial bursts and pauses were often followed by stimulus-induced oscillations in the peri-stimulus time histograms (PSTH). TSS induced long-lasting changes in DCN unit responses. Spike-related potentiation and suppression (SR-P and SR-S), either in units initiating the response with bursts or pauses, were correlated with stimulus-induced oscillations. Fitting with resonant functions suggested the existence of peaks in the theta-band (burst SR-P at 9 Hz, pause SR-S at 5 Hz). Optogenetic stimulation of the cerebellar cortex altered stimulus-induced oscillations suggesting that Purkinje cells play a critical role in the circuits controlling DCN oscillations and plasticity. This observation complements those reported before on the granular and molecular layers supporting the generation of multiple distributed plasticities in the cerebellum following naturally patterned sensory entrainment. The unique dependency of DCN plasticity on circuit oscillations discloses a potential relationship between cerebellar learning and activity patterns generated in the cerebellar network.

Modeling the Encoding of Saccade Kinematic Metrics in the Purkinje Cell Layer of the Cerebellar Vermis

Recent electrophysiological observations related to saccadic eye movements in rhesus monkeys, suggest a prediction of the sensory consequences of movement in the Purkinje cell layer of the cerebellar oculomotor vermis (OMV). A definite encoding of real-time motion of the eye has been observed in simple-spike responses of the combined burst-pause Purkinje cell populations, organized based upon their complex-spike directional tuning. However, the underlying control mechanisms that could lead to such action encoding are still unclear. We propose a saccade control model, with emphasis on the structure of the OMV and its interaction with the extra-cerebellar components. In the simulated bilateral organization of the OMV, each caudal fastigial nucleus is arranged to receive incoming projections from combined burst-pause Purkinje cell populations. The OMV, through the caudal fastigial nuclei, interacts with the brainstem to provide adaptive saccade gain corrections that minimize the visual error in reaching a given target location. The simulation results corroborate the experimental Purkinje cell population activity patterns and their relation with saccade kinematic metrics. The Purkinje layer activity that emerges from the proposed organization, precisely predicted the speed of the eye at different target eccentricities. Simulated granular layer activity suggests no separate dynamics with respect to shaping the bilateral Purkine layer activity. We further examine the validity of the simulated OMV in maintaining the accuracy of saccadic eye movements in the presence of signal dependent variabilities, that can occur in extra-cerebellar pathways.

Control of a Humanoid NAO Robot by an Adaptive Bioinspired Cerebellar Module in 3D Motion Tasks

A bioinspired adaptive model, developed by means of a spiking neural network made of thousands of artificial neurons, has been leveraged to control a humanoid NAO robot in real time. The learning properties of the system have been challenged in a classic cerebellum-driven paradigm, a perturbed upper limb reaching protocol. The neurophysiological principles used to develop the model succeeded in driving an adaptive motor control protocol with baseline, acquisition, and extinction phases. The spiking neural network model showed learning behaviours similar to the ones experimentally measured with human subjects in the same task in the acquisition phase, while resorted to other strategies in the extinction phase. The model processed in real-time external inputs, encoded as spikes, and the generated spiking activity of its output neurons was decoded, in order to provide the proper correction on the motor actuators. Three bidirectional long-term plasticity rules have been embedded for different connections and with different time scales. The plasticities shaped the firing activity of the output layer neurons of the network. In the perturbed upper limb reaching protocol, the neurorobot successfully learned how to compensate for the external perturbation generating an appropriate correction. Therefore, the spiking cerebellar model was able to reproduce in the robotic platform how biological systems deal with external sources of error, in both ideal and real (noisy) environments.

Complex Dynamics in Simplified Neuronal Models: Reproducing Golgi Cell Electroresponsiveness

Brain neurons exhibit complex electroresponsive properties – including intrinsic subthreshold oscillations and pacemaking, resonance and phase-reset – which are thought to play a critical role in controlling neural network dynamics. Although these properties emerge from detailed representations of molecular-level mechanisms in “realistic” models, they cannot usually be generated by simplified neuronal models (although these may show spike-frequency adaptation and bursting). We report here that this whole set of properties can be generated by the extended generalized leaky integrate-and-fire (E-GLIF) neuron model. E-GLIF derives from the GLIF model family and is therefore mono-compartmental, keeps the limited computational load typical of a linear low-dimensional system, admits analytical solutions and can be tuned through gradient-descent algorithms. Importantly, E-GLIF is designed to maintain a correspondence between model parameters and neuronal membrane mechanisms through a minimum set of equations. In order to test its potential, E-GLIF was used to model a specific neuron showing rich and complex electroresponsiveness, the cerebellar Golgi cell, and was validated against experimental electrophysiological data recorded from Golgi cells in acute cerebellar slices. During simulations, E-GLIF was activated by stimulus patterns, including current steps and synaptic inputs, identical to those used for the experiments. The results demonstrate that E-GLIF can reproduce the whole set of complex neuronal dynamics typical of these neurons – including intensity-frequency curves, spike-frequency adaptation, post-inhibitory rebound bursting, spontaneous subthreshold oscillations, resonance, and phase-reset – providing a new effective tool to investigate brain dynamics in large-scale simulations.

Cerebellar Theta-Burst Stimulation Impairs Memory Consolidation in Eyeblink Classical Conditioning

Associative learning of sensorimotor contingences, as it occurs in eyeblink classical conditioning (EBCC), is known to involve the cerebellum, but its mechanism remains controversial. EBCC involves a sequence of learning processes which are thought to occur in the cerebellar cortex and deep cerebellar nuclei. Recently, the extinction phase of EBCC has been shown to be modulated after one week by cerebellar continuous theta-burst stimulation (cTBS). Here, we asked whether cerebellar cTBS could affect retention and reacquisition of conditioned responses (CRs) tested immediately after conditioning. We also investigated a possible lateralized cerebellar control of EBCC by applying cTBS on both the right and left cerebellar hemispheres. Both right and left cerebellar cTBSs induced a statistically significant impairment in retention and new acquisition of conditioned responses (CRs), the disruption effect being marginally more effective when the left cerebellar hemisphere was stimulated. These data support a model in which cTBS impairs retention and reacquisition of CR in the cerebellum, possibly by interfering with the transfer of memory to the deep cerebellar nuclei.

A Survey of Robotics Control Based on Learning-Inspired Spiking Neural Networks

Biological intelligence processes information using impulses or spikes, which makes those living creatures able to perceive and act in the real world exceptionally well and outperform state-of-the-art robots in almost every aspect of life. To make up the deficit, emerging hardware technologies and software knowledge in the fields of neuroscience, electronics, and computer science have made it possible to design biologically realistic robots controlled by spiking neural networks (SNNs), inspired by the mechanism of brains. However, a comprehensive review on controlling robots based on SNNs is still missing. In this paper, we survey the developments of the past decade in the field of spiking neural networks for control tasks, with particular focus on the fast emerging robotics-related applications. We first highlight the primary impetuses of SNN-based robotics tasks in terms of speed, energy efficiency, and computation capabilities. We then classify those SNN-based robotic applications according to different learning rules and explicate those learning rules with their corresponding robotic applications. We also briefly present some existing platforms that offer an interaction between SNNs and robotics simulations for exploration and exploitation. Finally, we conclude our survey with a forecast of future challenges and some associated potential research topics in terms of controlling robots based on SNNs.

Dynamic Redistribution of Plasticity in a Cerebellar Spiking Neural Network Reproducing an Associative Learning Task Perturbed by TMS

During natural learning, synaptic plasticity is thought to evolve dynamically and redistribute within and among subcircuits. This process should emerge in plastic neural networks evolving under behavioral feedback and should involve changes distributed across multiple synaptic sites. In eyeblink classical conditioning (EBCC), the cerebellum learns to predict the precise timing between two stimuli, hence EBCC represents an elementary yet meaningful paradigm to investigate the cerebellar network functioning. We have simulated EBCC mechanisms by reconstructing a realistic cerebellar microcircuit model and embedding multiple plasticity rules imitating those revealed experimentally. The model was tuned to fit experimental EBCC human data, estimating the underlying learning time-constants. Learning started rapidly with plastic changes in the cerebellar cortex followed by slower changes in the deep cerebellar nuclei. This process was characterized by differential development of long-term potentiation and depression at individual synapses, with a progressive accumulation of plasticity distributed over the whole network. The experimental data included two EBCC sessions interleaved by a trans-cranial magnetic stimulation (TMS). The experimental and the model response data were not significantly different in each learning phase, and the model goodness-of-fit was [Formula: see text] for all the experimental conditions. The models fitted on TMS data revealed a slowed down re-acquisition (sessions-2) compared to the control condition ([Formula: see text]). The plasticity parameters characterizing each model significantly differ among conditions, and thus mechanistically explain these response changes. Importantly, the model was able to capture the alteration in EBCC consolidation caused by TMS and showed that TMS affected plasticity at cortical synapses thereby altering the fast learning phase. This, secondarily, also affected plasticity in deep cerebellar nuclei altering learning dynamics in the entire sensory-motor loop. This observation reveals dynamic redistribution of changes over the entire network and suggests how TMS affects local circuit computation and memory processing in the cerebellum.

A Multiple-Plasticity Spiking Neural Network Embedded in a Closed-Loop Control System to Model Cerebellar Pathologies

The cerebellum plays a crucial role in sensorimotor control and cerebellar disorders compromise adaptation and learning of motor responses. However, the link between alterations at network level and cerebellar dysfunction is still unclear. In principle, this understanding would benefit of the development of an artificial system embedding the salient neuronal and plastic properties of the cerebellum and operating in closed-loop. To this aim, we have exploited a realistic spiking computational model of the cerebellum to analyze the network correlates of cerebellar impairment. The model was modified to reproduce three different damages of the cerebellar cortex: (i) a loss of the main output neurons (Purkinje Cells), (ii) a lesion to the main cerebellar afferents (Mossy Fibers), and (iii) a damage to a major mechanism of synaptic plasticity (Long Term Depression). The modified network models were challenged with an Eye-Blink Classical Conditioning test, a standard learning paradigm used to evaluate cerebellar impairment, in which the outcome was compared to reference results obtained in human or animal experiments. In all cases, the model reproduced the partial and delayed conditioning typical of the pathologies, indicating that an intact cerebellar cortex functionality is required to accelerate learning by transferring acquired information to the cerebellar nuclei. Interestingly, depending on the type of lesion, the redistribution of synaptic plasticity and response timing varied greatly generating specific adaptation patterns. Thus, not only the present work extends the generalization capabilities of the cerebellar spiking model to pathological cases, but also predicts how changes at the neuronal level are distributed across the network, making it usable to infer cerebellar circuit alterations occurring in cerebellar pathologies.

A shared synapse architecture for efficient FPGA implementation of autoencoders

This paper proposes a shared synapse architecture for autoencoders (AEs), and implements an AE with the proposed architecture as a digital circuit on a field-programmable gate array (FPGA). In the proposed architecture, the values of the synapse weights are shared between the synapses of an input and a hidden layer, and between the synapses of a hidden and an output layer. This architecture utilizes less of the limited resources of an FPGA than an architecture which does not share the synapse weights, and reduces the amount of synapse modules used by half. For the proposed circuit to be implemented into various types of AEs, it utilizes three kinds of parameters; one to change the number of layers’ units, one to change the bit width of an internal value, and a learning rate. By altering a network configuration using these parameters, the proposed architecture can be used to construct a stacked AE. The proposed circuits are logically synthesized, and the number of their resources is determined. Our experimental results show that single and stacked AE circuits utilizing the proposed shared synapse architecture operate as regular AEs and as regular stacked AEs. The scalability of the proposed circuit and the relationship between the bit widths and the learning results are also determined. The clock cycles of the proposed circuits are formulated, and this formula is used to estimate the theoretical performance of the circuit when the circuit is used to construct arbitrary networks.

Physiology of the cerebellum

The cerebellum is a central brain structure deeply integrated into major loops with the cerebral cortex, brainstem, and spinal cord. The cerebellum shows a complex regional organization consisting of modules with sagittal orientation. The cerebellum takes part in motor control and its lesions cause a movement incoordination syndrome called ataxia. Recent observations also imply involvement of the cerebellum in cognition and executive control, with an impact on pathologies like dyslexia and autism. The cerebellum operates as a forward controller learning to predict the precise timing of correlated events. The physiologic mechanisms of cerebellar functioning are still the object of intense research. The signals entering the cerebellum through the mossy fibers are processed in the granular layer and transmitted to Purkinje cells, while a collateral pathway activates the deep cerebellar nuclei (DCN). Purkinje cells in turn inhibit DCN, so that the cerebellar cortex operates as a side loop controlling the DCN. Learning is now known to occur through synaptic plasticity at multiple synapses in the granular layer, molecular layer, and DCN, extending the original concept of the Motor Learning Theory that predicted a single form of plasticity at the synapse between parallel fibers and Purkinje cells under the supervision of climbing fibers deriving from the inferior olive. Coordination derives from the precise regulation of timing and gain in the different cerebellar modules. The investigation of cerebellar dynamics using advanced physiologic recordings and computational models is now providing new clues on how the cerebellar network performs its internal computations.

Model-Driven Analysis of Eyeblink Classical Conditioning Reveals the Underlying Structure of Cerebellar Plasticity and Neuronal Activity

The cerebellum plays a critical role in sensorimotor control. However, how the specific circuits and plastic mechanisms of the cerebellum are engaged in closed-loop processing is still unclear. We developed an artificial sensorimotor control system embedding a detailed spiking cerebellar microcircuit with three bidirectional plasticity sites. This proved able to reproduce a cerebellar-driven associative paradigm, the eyeblink classical conditioning (EBCC), in which a precise time relationship between an unconditioned stimulus (US) and a conditioned stimulus (CS) is established. We challenged the spiking model to fit an experimental data set from human subjects. Two subsequent sessions of EBCC acquisition and extinction were recorded and transcranial magnetic stimulation (TMS) was applied on the cerebellum to alter circuit function and plasticity. Evolutionary algorithms were used to find the near-optimal model parameters to reproduce the behaviors of subjects in the different sessions of the protocol. The main finding is that the optimized cerebellar model was able to learn to anticipate (predict) conditioned responses with accurate timing and success rate, demonstrating fast acquisition, memory stabilization, rapid extinction, and faster reacquisition as in EBCC in humans. The firing of Purkinje cells (PCs) and deep cerebellar nuclei (DCN) changed during learning under the control of synaptic plasticity, which evolved at different rates, with a faster acquisition in the cerebellar cortex than in DCN synapses. Eventually, a reduced PC activity released DCN discharge just after the CS, precisely anticipating the US and causing the eyeblink. Moreover, a specific alteration in cortical plasticity explained the EBCC changes induced by cerebellar TMS in humans. In this paper, for the first time, it is shown how closed-loop simulations, using detailed cerebellar microcircuit models, can be successfully used to fit real experimental data sets. Thus, the changes of the model parameters in the different sessions of the protocol unveil how implicit microcircuit mechanisms can generate normal and altered associative behaviors.The cerebellum plays a critical role in sensorimotor control. However, how the specific circuits and plastic mechanisms of the cerebellum are engaged in closed-loop processing is still unclear. We developed an artificial sensorimotor control system embedding a detailed spiking cerebellar microcircuit with three bidirectional plasticity sites. This proved able to reproduce a cerebellar-driven associative paradigm, the eyeblink classical conditioning (EBCC), in which a precise time relationship between an unconditioned stimulus (US) and a conditioned stimulus (CS) is established. We challenged the spiking model to fit an experimental data set from human subjects. Two subsequent sessions of EBCC acquisition and extinction were recorded and transcranial magnetic stimulation (TMS) was applied on the cerebellum to alter circuit function and plasticity. Evolutionary algorithms were used to find the near-optimal model parameters to reproduce the behaviors of subjects in the different sessions of the protocol. The main finding is that the optimized cerebellar model was able to learn to anticipate (predict) conditioned responses with accurate timing and success rate, demonstrating fast acquisition, memory stabilization, rapid extinction, and faster reacquisition as in EBCC in humans. The firing of Purkinje cells (PCs) and deep cerebellar nuclei (DCN) changed during learning under the control of synaptic plasticity, which evolved at different rates, with a faster acquisition in the cerebellar cortex than in DCN synapses. Eventually, a reduced PC activity released DCN discharge just after the CS, precisely anticipating the US and causing the eyeblink. Moreover, a specific alteration in cortical plasticity explained the EBCC changes induced by cerebellar TMS in humans. In this paper, for the first time, it is shown how closed-loop simulations, using detailed cerebellar microcircuit models, can be successfully used to fit real experimental data sets. Thus, the changes of the model parameters in the different sessions of the protocol unveil how implicit microcircuit mechanisms can generate normal and altered associative behaviors.

Proprioceptive Feedback through a Neuromorphic Muscle Spindle Model

Connecting biologically inspired neural simulations to physical or simulated embodiments can be useful both in robotics, for the development of a new kind of bio-inspired controllers, and in neuroscience, to test detailed brain models in complete action-perception loops. The aim of this work is to develop a fully spike-based, biologically inspired mechanism for the translation of proprioceptive feedback. The translation is achieved by implementing a computational model of neural activity of type Ia and type II afferent fibers of muscle spindles, the primary source of proprioceptive information, which, in mammals is regulated through fusimotor activation and provides necessary adjustments during voluntary muscle contractions. As such, both static and dynamic γ-motoneurons activities are taken into account in the proposed model. Information from the actual proprioceptive sensors (i.e., motor encoders) is then used to simulate the spindle contraction and relaxation, and therefore drive the neural activity. To assess the feasibility of this approach, the model is implemented on the NEST spiking neural network simulator and on the SpiNNaker neuromorphic hardware platform and tested on simulated and physical robotic platforms. The results demonstrate that the model can be used in both simulated and real-time robotic applications to translate encoder values into a biologically plausible neural activity. Thus, this model provides a completely spike-based building block, suitable for neuromorphic platforms, that will enable the development of sensory-motor closed loops which could include neural simulations of areas of the central nervous system or of low-level reflexes.

Event- and Time-Driven Techniques Using Parallel CPU-GPU Co-processing for Spiking Neural Networks

Modeling and simulating the neural structures which make up our central neural system is instrumental for deciphering the computational neural cues beneath. Higher levels of biological plausibility usually impose higher levels of complexity in mathematical modeling, from neural to behavioral levels. This paper focuses on overcoming the simulation problems (accuracy and performance) derived from using higher levels of mathematical complexity at a neural level. This study proposes different techniques for simulating neural models that hold incremental levels of mathematical complexity: leaky integrate-and-fire (LIF), adaptive exponential integrate-and-fire (AdEx), and Hodgkin-Huxley (HH) neural models (ranged from low to high neural complexity). The studied techniques are classified into two main families depending on how the neural-model dynamic evaluation is computed: the event-driven or the time-driven families. Whilst event-driven techniques pre-compile and store the neural dynamics within look-up tables, time-driven techniques compute the neural dynamics iteratively during the simulation time. We propose two modifications for the event-driven family: a look-up table recombination to better cope with the incremental neural complexity together with a better handling of the synchronous input activity. Regarding the time-driven family, we propose a modification in computing the neural dynamics: the bi-fixed-step integration method. This method automatically adjusts the simulation step size to better cope with the stiffness of the neural model dynamics running in CPU platforms. One version of this method is also implemented for hybrid CPU-GPU platforms. Finally, we analyze how the performance and accuracy of these modifications evolve with increasing levels of neural complexity. We also demonstrate how the proposed modifications which constitute the main contribution of this study systematically outperform the traditional event- and time-driven techniques under increasing levels of neural complexity.

Distributed Circuit Plasticity: New Clues for the Cerebellar Mechanisms of Learning

The cerebellum is involved in learning and memory of sensory motor skills. However, the way this process takes place in local microcircuits is still unclear. The initial proposal, casted into the Motor Learning Theory, suggested that learning had to occur at the parallel fiber-Purkinje cell synapse under supervision of climbing fibers. However, the uniqueness of this mechanism has been questioned, and multiple forms of long-term plasticity have been revealed at various locations in the cerebellar circuit, including synapses and neurons in the granular layer, molecular layer and deep-cerebellar nuclei. At present, more than 15 forms of plasticity have been reported. There has been a long debate on which plasticity is more relevant to specific aspects of learning, but this question turned out to be hard to answer using physiological analysis alone. Recent experiments and models making use of closed-loop robotic simulations are revealing a radically new view: one single form of plasticity is insufficient, while altogether, the different forms of plasticity can explain the multiplicity of properties characterizing cerebellar learning. These include multi-rate acquisition and extinction, reversibility, self-scalability, and generalization. Moreover, when the circuit embeds multiple forms of plasticity, it can easily cope with multiple behaviors endowing therefore the cerebellum with the properties needed to operate as an effective generalized forward controller.

Modeling the Cerebellar Microcircuit: New Strategies for a Long-Standing Issue

The cerebellar microcircuit has been the work bench for theoretical and computational modeling since the beginning of neuroscientific research. The regular neural architecture of the cerebellum inspired different solutions to the long-standing issue of how its circuitry could control motor learning and coordination. Originally, the cerebellar network was modeled using a statistical-topological approach that was later extended by considering the geometrical organization of local microcircuits. However, with the advancement in anatomical and physiological investigations, new discoveries have revealed an unexpected richness of connections, neuronal dynamics and plasticity, calling for a change in modeling strategies, so as to include the multitude of elementary aspects of the network into an integrated and easily updatable computational framework. Recently, biophysically accurate “realistic” models using a bottom-up strategy accounted for both detailed connectivity and neuronal non-linear membrane dynamics. In this perspective review, we will consider the state of the art and discuss how these initial efforts could be further improved. Moreover, we will consider how embodied neurorobotic models including spiking cerebellar networks could help explaining the role and interplay of distributed forms of plasticity. We envisage that realistic modeling, combined with closed-loop simulations, will help to capture the essence of cerebellar computations and could eventually be applied to neurological diseases and neurorobotic control systems.

Healthy and pathological cerebellar Spiking Neural Networks in Vestibulo-Ocular Reflex

Since the Marr-Albus model, computational neuroscientists have been developing a variety of models of the cerebellum, with different approaches and features. In this work, we developed and tested realistic artificial Spiking Neural Networks inspired to this brain region. We tested in computational simulations of the Vestibulo-Ocular Reflex protocol three different models: a network equipped with a single plasticity site, at the cortical level; a network equipped with a distributed plasticity, at both cortical and nuclear levels; a network with a pathological plasticity mechanism at the cortical level. We analyzed the learning performance of the three different models, highlighting the behavioral differences among them. We proved that the model with a distributed plasticity produces a faster and more accurate cerebellar response, especially during a second session of acquisition, compared with the single plasticity model. Furthermore, the pathological model shows an impaired learning capability in Vestibulo-Ocular Reflex acquisition, as found in neurophysiological studies. The effect of the different plasticity conditions, which change fast and slow dynamics, memory consolidation and, in general, learning capabilities of the cerebellar network, explains differences in the behavioral outcome.

Integrated plasticity at inhibitory and excitatory synapses in the cerebellar circuit

The way long-term potentiation (LTP) and depression (LTD) are integrated within the different synapses of brain neuronal circuits is poorly understood. In order to progress beyond the identification of specific molecular mechanisms, a system in which multiple forms of plasticity can be correlated with large-scale neural processing is required. In this paper we take as an example the cerebellar network, in which extensive investigations have revealed LTP and LTD at several excitatory and inhibitory synapses. Cerebellar LTP and LTD occur in all three main cerebellar subcircuits (granular layer, molecular layer, deep cerebellar nuclei) and correspondingly regulate the function of their three main neurons: granule cells (GrCs), Purkinje cells (PCs) and deep cerebellar nuclear (DCN) cells. All these neurons, in addition to be excited, are reached by feed-forward and feed-back inhibitory connections, in which LTP and LTD may either operate synergistically or homeostatically in order to control information flow through the circuit. Although the investigation of individual synaptic plasticities in vitro is essential to prove their existence and mechanisms, it is insufficient to generate a coherent view of their impact on network functioning in vivo. Recent computational models and cell-specific genetic mutations in mice are shedding light on how plasticity at multiple excitatory and inhibitory synapses might regulate neuronal activities in the cerebellar circuit and contribute to learning and memory and behavioral control.

How the credit assignment problems in motor control could be solved after the cerebellum predicts increases in error

We present a cerebellar architecture with two main characteristics. The first one is that complex spikes respond to increases in sensory errors. The second one is that cerebellar modules associate particular contexts where errors have increased in the past with corrective commands that stop the increase in error. We analyze our architecture formally and computationally for the case of reaching in a 3D environment. In the case of motor control, we show that there are synergies of this architecture with the Equilibrium-Point hypothesis, leading to novel ways to solve the motor error and distal learning problems. In particular, the presence of desired equilibrium lengths for muscles provides a way to know when the error is increasing, and which corrections to apply. In the context of Threshold Control Theory and Perceptual Control Theory we show how to extend our model so it implements anticipative corrections in cascade control systems that span from muscle contractions to cognitive operations.

Distributed cerebellar plasticity implements generalized multiple-scale memory components in real-robot sensorimotor tasks

The cerebellum plays a crucial role in motor learning and it acts as a predictive controller. Modeling it and embedding it into sensorimotor tasks allows us to create functional links between plasticity mechanisms, neural circuits and behavioral learning. Moreover, if applied to real-time control of a neurorobot, the cerebellar model has to deal with a real noisy and changing environment, thus showing its robustness and effectiveness in learning. A biologically inspired cerebellar model with distributed plasticity, both at cortical and nuclear sites, has been used. Two cerebellum-mediated paradigms have been designed: an associative Pavlovian task and a vestibulo-ocular reflex, with multiple sessions of acquisition and extinction and with different stimuli and perturbation patterns. The cerebellar controller succeeded to generate conditioned responses and finely tuned eye movement compensation, thus reproducing human-like behaviors. Through a productive plasticity transfer from cortical to nuclear sites, the distributed cerebellar controller showed in both tasks the capability to optimize learning on multiple time-scales, to store motor memory and to effectively adapt to dynamic ranges of stimuli.