Cerebellar learning of bio-mechanical functions of extra-ocular muscles: modeling by artificial neural networks

Dual STDP processes at Purkinje cells contribute to distinct improvements in accuracy and speed of saccadic eye movements

Saccadic eye-movements play a crucial role in visuo-motor control by allowing rapid foveation onto new targets. However, the neural processes governing saccades adaptation are not fully understood. Saccades, due to the short-time of execution (20-100 ms) and the absence of sensory information for online feedback control, must be controlled in a ballistic manner. Incomplete measurements of the movement trajectory, such as the visual endpoint error, are supposedly used to form internal predictions about the movement kinematics resulting in predictive control. In order to characterize the synaptic and neural circuit mechanisms underlying predictive saccadic control, we have reconstructed the saccadic system in a digital controller embedding a spiking neural network of the cerebellum with spike timing-dependent plasticity (STDP) rules driving parallel fiber-Purkinje cell long-term potentiation and depression (LTP and LTD). This model implements a control policy based on a dual plasticity mechanism, resulting in the identification of the roles of LTP and LTD in regulating the overall quality of saccade kinematics: it turns out that LTD increases the accuracy by decreasing visual error and LTP increases the peak speed. The control policy also required cerebellar PCs to be divided into two subpopulations, characterized by burst or pause responses. To our knowledge, this is the first model that explains in mechanistic terms the visual error and peak speed regulation of ballistic eye movements in forward mode exploiting spike-timing to regulate firing in different populations of the neuronal network. This elementary model of saccades could be extended and applied to other more complex cases in which single jerks are concatenated to compose articulated and coordinated movements.

A system-level mathematical model of Basal Ganglia motor-circuit for kinematic planning of arm movements

In this paper, a novel system-level mathematical model of the Basal Ganglia (BG) for kinematic planning, is proposed. An arm composed of several segments presents a geometric redundancy. Thus, selecting one trajectory among an infinite number of possible ones requires overcoming redundancy, according to some kinds of optimization. Solving this optimization is assumed to be the function of BG in planning. In the proposed model, first, a mathematical solution of kinematic planning is proposed for movements of a redundant arm in a plane, based on minimizing energy consumption. Next, the function of each part in the model is interpreted as a possible role of a nucleus of BG. Since the kinematic variables are considered as vectors, the proposed model is presented based on the vector calculus. This vector model predicts different neuronal populations in BG which is in accordance with some recent experimental studies. According to the proposed model, the function of the direct pathway is to calculate the necessary rotation of each joint, and the function of the indirect pathway is to control each joint rotation considering the movement of the other joints. In the proposed model, the local feedback loop between Subthalamic Nucleus and Globus Pallidus externus is interpreted as a local memory to store the previous amounts of movements of the other joints, which are utilized by the indirect pathway. In this model, activities of dopaminergic neurons would encode, at short-term, the error between the desired and actual positions of the end-effector. The short-term modulating effect of dopamine on Striatum is also modeled as cross product. The model is simulated to generate the commands of a redundant manipulator. The performance of the model is studied for different reaching movements between 8 points in a plane. Finally, some symptoms of Parkinson's disease such as bradykinesia and akinesia are simulated by modifying the model parameters, inspired by the dopamine depletion.

Fuzzy neuronal model of motor control inspired by cerebellar pathways to online and gradually learn inverse biomechanical functions in the presence of delay

Contrary to forward biomechanical functions, which are deterministic, inverse biomechanical functions are generally not. Calculating an inverse biomechanical function is an ill-posed problem, which has no unique solution for a manipulator with several degrees of freedom. Studies of the command and control of biological movements suggest that the cerebellum takes part in the computation of approximate inverse functions, and this ability can control fast movements by predicting the consequence of current motor command. Limb movements toward a goal are defined as fast if they last less than the total duration of the processing and transmission delays in the motor and sensory pathways. Because of these delays, fast movements cannot be continuously controlled in a closed loop by use of sensory signals. Thus, fast movements must be controlled by some open loop controller, of which cerebellar pathways constitute an important part. This article presents a system-level fuzzy neuronal motor control circuit, inspired by the cerebellar pathways. The cerebellar cortex (CC) is assumed to embed internal models of the biomechanical functions of the limb segments. Such neural models are able to predict the consequences of motor commands and issue predictive signals encoding movement variables, which are sent to the controller via internal feedback loops. Differences between desired and expected values of variables of movements are calculated in the deep cerebellar nuclei (DCN). After motor learning, the whole circuit can approximate the inverse function of the biomechanical function of a limb and acts as a controller. In this research, internal models of direct biomechanical functions are learned and embedded in the connectivity of the cerebellar pathways. Two fuzzy neural networks represent the two parts of the cerebellum, and an online gradual learning drives the acquisition of the internal models in CC and the controlling rules in DCN. As during real learning, exercise and repetition increase skill and speed. The learning procedure is started by a simple and slow movement, controlled in the presence of delays by a simple closed loop controller comparable to the spinal reflexes. The speed of the movements is then increased gradually, and output error signals are used to compute teaching signals and drive learning. Repetition of movements at each speed level allows to properly set the two neural networks, and progressively learn the movement. Finally, conditions of stability of the proposed model as an inverter are identified. Next, the control of a single segment arm, moved by two muscles, is simulated. After proper setting by motor learning, the circuit is able to reject perturbations.

Cerebellum-inspired neural network solution of the inverse kinematics problem

The demand today for more complex robots that have manipulators with higher degrees of freedom is increasing because of technological advances. Obtaining the precise movement for a desired trajectory or a sequence of arm and positions requires the computation of the inverse kinematic (IK) function, which is a major problem in robotics. The solution of the IK problem leads robots to the precise position and orientation of their end-effector. We developed a bioinspired solution comparable with the cerebellar anatomy and function to solve the said problem. The proposed model is stable under all conditions merely by parameter determination, in contrast to recursive model-based solutions, which remain stable only under certain conditions. We modified the proposed model for the simple two-segmented arm to prove the feasibility of the model under a basic condition. A fuzzy neural network through its learning method was used to compute the parameters of the system. Simulation results show the practical feasibility and efficiency of the proposed model in robotics. The main advantage of the proposed model is its generalizability and potential use in any robot.

Evaluating the adaptive-filter model of the cerebellum

The adaptive-filter model of the cerebellar microcircuit is in widespread use, combining as it does an explanation of key microcircuit features with well-specified computational power. Here we consider two methods for its evaluation. One is to test its predictions concerning relations between cerebellar inputs and outputs. Where the relevant experimental data are available, e.g. for the floccular role in image stabilization, the predictions appear to be upheld. However, for the majority of cerebellar microzones these data have yet to be obtained. The second method is to test model predictions about details of the microcircuit. We focus on features apparently incompatible with the model, in particular non-linear patterns in Purkinje cell simple-spike firing. Analysis of these patterns suggests the following three conclusions. (i) It is important to establish whether they can be observed during task-related behaviour. (ii) Highly non-linear models based on these patterns are unlikely to be universal, because they would be incompatible with the (approximately) linear nature of floccular function. (iii) The control tasks for which these models are computationally suited need to be identified. At present, therefore, the adaptive filter remains a candidate model of at least some cerebellar microzones, and its evaluation suggests promising lines for future enquiry.

Integration of gravitational torques in cerebellar pathways allows for the dynamic inverse computation of vertical pointing movements of a robot arm

BACKGROUND

Several authors suggested that gravitational forces are centrally represented in the brain for planning, control and sensorimotor predictions of movements. Furthermore, some studies proposed that the cerebellum computes the inverse dynamics (internal inverse model) whereas others suggested that it computes sensorimotor predictions (internal forward model).

METHODOLOGY/PRINCIPAL FINDINGS

This study proposes a model of cerebellar pathways deduced from both biological and physical constraints. The model learns the dynamic inverse computation of the effect of gravitational torques from its sensorimotor predictions without calculating an explicit inverse computation. By using supervised learning, this model learns to control an anthropomorphic robot arm actuated by two antagonists McKibben artificial muscles. This was achieved by using internal parallel feedback loops containing neural networks which anticipate the sensorimotor consequences of the neural commands. The artificial neural networks architecture was similar to the large-scale connectivity of the cerebellar cortex. Movements in the sagittal plane were performed during three sessions combining different initial positions, amplitudes and directions of movements to vary the effects of the gravitational torques applied to the robotic arm. The results show that this model acquired an internal representation of the gravitational effects during vertical arm pointing movements.

CONCLUSIONS/SIGNIFICANCE

This is consistent with the proposal that the cerebellar cortex contains an internal representation of gravitational torques which is encoded through a learning process. Furthermore, this model suggests that the cerebellum performs the inverse dynamics computation based on sensorimotor predictions. This highlights the importance of sensorimotor predictions of gravitational torques acting on upper limb movements performed in the gravitational field.

From brainstem to cortex: computational models of saccade generation circuitry

The brain circuitry of saccadic eye movements, from brainstem to cortex, has been extensively studied during the last 30 years. The wealth of data gathered allowed the conception of numerous computational models. These models proposed descriptions of the putative mechanisms generating this data, and, in turn, made predictions and helped to plan new experiments. In this article, we review the computational models of the five main brain regions involved in saccade generation: reticular formation saccadic burst generators, superior colliculus, cerebellum, basal ganglia and premotor cortical areas. We present the various topics these models are concerned with: location of the feedback loop, multimodal saccades, long-term adaptation, on the fly trajectory correction, strategy and metrics selection, short-term spatial memory, transformations between retinocentric and craniocentric reference frames, sequence learning, to name the principle ones. Our objective is to provide a global view of the whole system. Indeed, narrowing too much the modelled areas while trying to explain too much data is a recurrent problem that should be avoided. Moreover, beyond the multiple research topics remaining to be solved locally, questions regarding the operation of the whole structure can now be addressed by building on the existing models.

Animal models of focal dystonia

Animal models indicate that the abnormal movements of focal dystonia result from disordered sensorimotor integration. Sensorimotor integration involves a comparison of sensory information resulting from a movement with the sensory information expected from the movement. Unanticipated sensory signals identified by sensorimotor processing serve as signals to modify the ongoing movement or the planning for subsequent movements. Normally, this process is an effective mechanism to modify neural commands for ongoing movement or for movement planning. Animal models of the focal dystonias spasmodic torticollis, writer's cramp, and benign essential blepharospasm reveal different dysfunctions of sensorimotor integration through which dystonia can arise. Animal models of spasmodic torticollis demonstrate that modifications in a variety of regions are capable of creating abnormal head postures. These data indicate that disruption of neural signals in one structure may mutate the activity pattern of other elements of the neural circuits for movement. The animal model of writer's cramp demonstrates the importance of abnormal sensory processing in generating dystonic movements. Animal models of blepharospasm illustrate how disrupting motor adaptation can produce dystonia. Together, these models show mechanisms by which disruptions in sensorimotor integration can create dystonic movements.

Feed-forward associative learning for volitional movement control

One of the most difficult problems in motor learning is determining the source of a learning signal, sometimes called an error signal. This problem is hidden in the adaptations of simple reflexive movements by attributing its source to sensory organs. The feed-forward associative motor learning theory proposed here attributes the source to the movement system itself. When a subject performs a corrective movement after his primary movement, the proposed neural learning device learns to associate the primary motor command with the corrective motor command by using a place-coding system. In the subsequent trials, the primary movement will involve a correction due to the participation of this mechanism, thus resulting in better performance. The theory assumes three conditions, namely, that a motor center and the learning device share the same place-encoded motor information; the motor center issues a command and a learning signal simultaneously from the same unit; and a learning signal issued with a corrective command has a heterosynaptic interaction with the previous primary command. The cerebellum is a reasonable candidate for the device satisfying these conditions. The reaction time of a corrective movement, usually 100-300 ms, almost satisfies the coincidence condition for long-term depression of the granule-to-Purkinje synapses. As an application, this theory is demonstrated to account for behavioral results regarding saccadic adaptation.

Computation of inverse functions in a model of cerebellar and reflex pathways allows to control a mobile mechanical segment

The command and control of limb movements by the cerebellar and reflex pathways are modeled by means of a circuit whose structure is deduced from functional constraints. One constraint is that fast limb movements must be accurate although they cannot be continuously controlled in closed loop by use of sensory signals. Thus, the pathways which process the motor orders must contain approximate inverse functions of the bio-mechanical functions of the limb and of the muscles. This can be achieved by means of parallel feedback loops, whose pattern turns out to be comparable to the anatomy of the cerebellar pathways. They contain neural networks able to anticipate the motor consequences of the motor orders, modeled by artificial neural networks whose connectivity is similar to that of the cerebellar cortex. These networks learn the direct biomechanical functions of the limbs and muscles by means of a supervised learning process. Teaching signals calculated from motor errors are sent to the learning sites, as, in the cerebellum, complex spikes issued from the inferior olive are conveyed to the Purkinje cells by climbing fibers. Learning rules are deduced by a differential calculation, as classical gradient rules, and they account for the long term depression which takes place in the dendritic arborizations of the Purkinje cells. Another constraint is that reflexes must not impede voluntary movements while remaining at any instant ready to oppose perturbations. Therefore, efferent copies of the motor orders are sent to the interneurones of the reflexes, where they cancel the sensory-motor consequences of the voluntary movements. After learning, the model is able to drive accurately, both in velocity and position, angular movements of a rod actuated by two pneumatic McKibben muscles. Reflexes comparable to the myotatic and tendinous reflexes, and stabilizing reactions comparable to the cerebellar sensory-motor reactions, reduce efficiently the effects of perturbing torques. These results allow to link the behavioral concepts of the equilibrium-point "lambda model" [J Motor Behav 18 (1986) 17] with anatomical and physiological features: gains of reflexes and sensori-motor reactions set the slope of the "invariant characteristic," and efferent copies set the "threshold of the stretch reflex." Thus, mathematical and physical laws account for the raison d'etre of the inhibitory nature of Purkinje cells and for the conspicuous anatomical pattern of the cerebellar pathways. These properties of these pathways allow to perform approximate inverse calculations after learning of direct functions, and insure also the coordination of voluntary and reflex motor orders.