An internal model architecture for novelty detection: implications for cerebellar and collicular roles in sensory processing

Automatic learning mechanisms for flexible human locomotion

Movement flexibility and automaticity are necessary to successfully navigate different environments. When encountering difficult terrains such as a muddy trail, we can change how we step almost immediately so that we can continue walking. This flexibility comes at a cost since we initially must pay deliberate attention to how we are moving. Gradually, after a few minutes on the trail, stepping becomes automatic so that we do not need to think about our movements. Canonical theory indicates that different adaptive motor learning mechanisms confer these essential properties to movement: explicit control confers rapid flexibility, while forward model recalibration confers automaticity. Here we uncover a distinct mechanism of treadmill walking adaptation - an automatic stimulus-response mapping - that confers both properties to movement. The mechanism is flexible as it learns stepping patterns that can be rapidly changed to suit a range of treadmill configurations. It is also automatic as it can operate without deliberate control or explicit awareness by the participants. Our findings reveal a tandem architecture of forward model recalibration and automatic stimulus-response mapping mechanisms for walking, reconciling different findings of motor adaptation and perceptual realignment.

Adaptive Filter Model of Cerebellum for Biological Muscle Control With Spike Train Inputs

Prior applications of the cerebellar adaptive filter model have included a range of tasks within simulated and robotic systems. However, this has been limited to systems driven by continuous signals. Here, the adaptive filter model of the cerebellum is applied to the control of a system driven by spiking inputs by considering the problem of controlling muscle force. The performance of the standard adaptive filter algorithm is compared with the algorithm with a modified learning rule that minimizes inputs and a simple proportional-integral-derivative (PID) controller. Control performance is evaluated in terms of the number of spikes, the accuracy of spike input locations, and the accuracy of muscle force output. Results show that the cerebellar adaptive filter model can be applied without change to the control of systems driven by spiking inputs. The cerebellar algorithm results in good agreement between input spikes and force outputs and significantly improves on a PID controller. Input minimization can be used to reduce the number of spike inputs, but at the expense of a decrease in accuracy of spike input location and force output. This work extends the applications of the cerebellar algorithm and demonstrates the potential of the adaptive filter model to be used to improve functional electrical stimulation muscle control.

Neural mechanisms for the localization of unexpected external motion

To localize objects during active sensing, animals must differentiate stimuli caused by volitional movement from real-world object motion. To determine a neural basis for this ability, we examined the mouse superior colliculus (SC), which contains multiple egocentric maps of sensorimotor space. By placing mice in a whisker-guided virtual reality, we discovered a rapidly adapting tactile response that transiently emerged during externally generated gains in whisker contact. Responses to self-generated touch that matched self-generated history were significantly attenuated, revealing that transient response magnitude is controlled by sensorimotor predictions. The magnitude of the transient response gradually decreased with repetitions in external motion, revealing a slow habituation based on external history. The direction of external motion was accurately encoded in the firing rates of transiently responsive neurons. These data reveal that whisker-specific adaptation and sensorimotor predictions in SC neurons enhance the localization of unexpected, externally generated changes in tactile space.

Neural responses to sensory novelty with and without conscious access

Detection of novel stimuli that violate statistical regularities in the sensory scene is of paramount importance for the survival of biological organisms. Event-related potentials, phasic increases in pupil size, and evoked changes in oscillatory power have been proposed as markers of sensory novelty detection. However, how conscious access to novelty modulates these different brain responses is not well understood. Here, we studied the neural responses to sensory novelty in the auditory modality with and without conscious access. We identified individual thresholds for conscious auditory discrimination and presented to our participants sequences of tones, where the last stimulus could be another standard, a subthreshold target or a suprathreshold target. Participants were instructed to report whether the last tone of each sequence was the same or different from those preceding it. Results indicate that attentional orientation to behaviorally relevant stimuli and overt decision-making mechanisms, indexed by the P3 event-related response and reaction times, best predict whether a novel stimulus will be consciously accessed. Theta power and pupil size do not predict conscious access to novelty, but instead reflect information maintenance and unexpected sensory uncertainty. These results highlight the interplay between bottom-up and top-down mechanisms and how the brain weights neural responses to novelty and uncertainty during perception and goal-directed behavior.

A multizone cerebellar chip for bioinspired adaptive robot control and sensorimotor processing

The cerebellum is a neural structure essential for learning, which is connected via multiple zones to many different regions of the brain, and is thought to improve human performance in a large range of sensory, motor and even cognitive processing tasks. An intriguing possibility for the control of complex robotic systems would be to develop an artificial cerebellar chip with multiple zones that could be similarly connected to a variety of subsystems to optimize performance. The novel aim of this paper, therefore, is to propose and investigate a multizone cerebellar chip applied to a range of tasks in robot adaptive control and sensorimotor processing. The multizone cerebellar chip was evaluated using a custom robotic platform consisting of an array of tactile sensors driven by dielectric electroactive polymers mounted upon a standard industrial robot arm. The results demonstrate that the performance in each task was improved by the concurrent, stable learning in each cerebellar zone. This paper, therefore, provides the first empirical demonstration that a synthetic, multizone, cerebellar chip could be embodied within existing robotic systems to improve performance in a diverse range of tasks, much like the cerebellum in a biological system.

World Statistics Drive Learning of Cerebellar Internal Models in Adaptive Feedback Control: A Case Study Using the Optokinetic Reflex

The cerebellum is widely implicated in having an important role in adaptive motor control. Many of the computational studies on cerebellar motor control to date have focused on the associated architecture and learning algorithms in an effort to further understand cerebellar function. In this paper we switch focus to the signals driving cerebellar adaptation that arise through different motor behavior. To do this, we investigate computationally the contribution of the cerebellum to the optokinetic reflex (OKR), a visual feedback control scheme for image stabilization. We develop a computational model of the adaptation of the cerebellar response to the world velocity signals that excite the OKR (where world velocity signals are used to emulate head velocity signals when studying the OKR in head-fixed experimental laboratory conditions). The results show that the filter learnt by the cerebellar model is highly dependent on the power spectrum of the colored noise world velocity excitation signal. Thus, the key finding here is that the cerebellar filter is determined by the statistics of the OKR excitation signal.

Event-Related Desynchronization/Synchronization in Spinocerebellar Ataxia Type 3

Introduction: Spinocerebellar ataxia type 3 (SCA3) is an autosomal dominant, cerebellar degeneration predominant disease caused by excessive CAG repeats. We examined event-related dysynchronization/synchronization (ERD/ERS) in patients with SCA3.

Methods: We assessed ERD/ERS of self-paced voluntary hand movements in 15 patients with genetically proven SCA3 in comparison with healthy controls.

Results: In ERS, a significant interaction effect between group, frequency, and period (F = 1.591; p = 0.005; ρI = 0.86) was observed. The post-hoc two-tailed independent t-test showed significant differences in high beta and low beta ERS. By contrast, in ERD, no apparent differences were observed in the pattern of patients with SCA3 in comparison with healthy controls (F = 1.01; p = 0.442).

Conclusion: The study revealed a decreased ERS in patients with SCA3, especially at the frequency of 20–30 Hz. This study elucidates the significant role of cerebellum in motor control.

Prominent Changes in Cerebro-Cerebellar Functional Connectivity During Continuous Cognitive Processing

While task-dependent responses of specific brain areas during cognitive tasks are well established, much less is known about the changes occurring in resting state networks (RSNs) in relation to continuous cognitive processing. In particular, the functional involvement of cerebro-cerebellar loops connecting the posterior cerebellum to associative cortices, remains unclear. In this study, 22 healthy volunteers underwent a multi-session functional magnetic resonance imaging (fMRI) protocol composed of four consecutive 8-min resting state fMRI (rs-fMRI) scans. After a first control scan, participants listened to a narrated story for the entire duration of the second rs-fMRI scan; two further rs-fMRI scans followed the end of story listening. The story plot was purposely designed to stimulate specific cognitive processes that are known to involve the cerebro-cerebellar loops. Almost all of the identified 15 RSNs showed changes in functional connectivity (FC) during and for several minutes after the story. The FC changes mainly occurred in the frontal and prefrontal cortices and in the posterior cerebellum, especially in Crus I-II and lobule VI. The FC changes occurred in cerebellar clusters belonging to different RSNs, including the cerebellar network (CBLN), sensory networks (lateral visual network, LVN; medial visual network, MVN) and cognitive networks (default mode network, DMN; executive control network, ECN; right and left ventral attention networks, RVAN and LVAN; salience network, SN; language network, LN; and working memory network, WMN). Interestingly, a k-means analysis of FC changes revealed clustering of FCN, ECN, and WMN, which are all involved in working memory functions, CBLN, DMN, and SN, which play a key-role in attention switching, and RSNs involved in visual imagery. These results show that the cerebellum is deeply entrained in well-structured network clusters, which reflect multiple aspects of cognitive processing, during and beyond the conclusion of auditory stimulation.

A comparative approach to cerebellar function: insights from electrosensory systems

Despite its simple and highly-ordered circuitry the function of the cerebellum remains a topic of vigorous debate. This review explores connections between the cerebellum and sensory processing structures that closely resemble the cerebellum in terms of their evolution, development, patterns of gene expression, and circuitry. Recent studies of cerebellum-like structures involved in electrosensory processing in fish have provided insights into the functions of granule cells and unipolar brush cells-cell types shared with the cerebellum. We also discuss the possibility, supported by recent studies, that generating and subtracting predictions of the sensory consequences of motor commands may be core functions shared by both cerebellum-like structures and the cerebellum.

Perception as a closed-loop convergence process

Perception of external objects involves sensory acquisition via the relevant sensory organs. A widely-accepted assumption is that the sensory organ is the first station in a serial chain of processing circuits leading to an internal circuit in which a percept emerges. This open-loop scheme, in which the interaction between the sensory organ and the environment is not affected by its concurrent downstream neuronal processing, is strongly challenged by behavioral and anatomical data. We present here a hypothesis in which the perception of external objects is a closed-loop dynamical process encompassing loops that integrate the organism and its environment and converging towards organism-environment steady-states. We discuss the consistency of closed-loop perception (CLP) with empirical data and show that it can be synthesized in a robotic setup. Testable predictions are proposed for empirical distinction between open and closed loop schemes of perception.

Biohybrid Control of General Linear Systems Using the Adaptive Filter Model of Cerebellum

The adaptive filter model of the cerebellar microcircuit has been successfully applied to biological motor control problems, such as the vestibulo-ocular reflex (VOR), and to sensory processing problems, such as the adaptive cancelation of reafferent noise. It has also been successfully applied to problems in robotics, such as adaptive camera stabilization and sensor noise cancelation. In previous applications to inverse control problems, the algorithm was applied to the velocity control of a plant dominated by viscous and elastic elements. Naive application of the adaptive filter model to the displacement (as opposed to velocity) control of this plant results in unstable learning and control. To be more generally useful in engineering problems, it is essential to remove this restriction to enable the stable control of plants of any order. We address this problem here by developing a biohybrid model reference adaptive control (MRAC) scheme, which stabilizes the control algorithm for strictly proper plants. We evaluate the performance of this novel cerebellar-inspired algorithm with MRAC scheme in the experimental control of a dielectric electroactive polymer, a class of artificial muscle. The results show that the augmented cerebellar algorithm is able to accurately control the displacement response of the artificial muscle. The proposed solution not only greatly extends the practical applicability of the cerebellar-inspired algorithm, but may also shed light on cerebellar involvement in a wider range of biological control tasks.

Plasticity of cerebellar Purkinje cells in behavioral training of body balance control

Neural responses to sensory inputs caused by self-generated movements (reafference) and external passive stimulation (exafference) differ in various brain regions. The ability to differentiate such sensory information can lead to movement execution with better accuracy. However, how sensory responses are adjusted in regard to this distinguishability during motor learning is still poorly understood. The cerebellum has been hypothesized to analyze the functional significance of sensory information during motor learning, and is thought to be a key region of reafference computation in the vestibular system. In this study, we investigated Purkinje cell (PC) spike trains as cerebellar cortical output when rats learned to balance on a suspended dowel. Rats progressively reduced the amplitude of body swing and made fewer foot slips during a 5-min balancing task. Both PC simple (SSs; 17 of 26) and complex spikes (CSs; 7 of 12) were found to code initially on the angle of the heads with respect to a fixed reference. Using periods with comparable degrees of movement, we found that such SS coding of information in most PCs (10 of 17) decreased rapidly during balance learning. In response to unexpected perturbations and under anesthesia, SS coding capability of these PCs recovered. By plotting SS and CS firing frequencies over 15-s time windows in double-logarithmic plots, a negative correlation between SS and CS was found in awake, but not anesthetized, rats. PCs with prominent SS coding attenuation during motor learning showed weaker SS-CS correlation. Hence, we demonstrate that neural plasticity for filtering out sensory reafference from active motion occurs in the cerebellar cortex in rats during balance learning. SS-CS interaction may contribute to this rapid plasticity as a form of receptive field plasticity in the cerebellar cortex between two receptive maps of sensory inputs from the external world and of efference copies from the will center for volitional movements.

Redefining the cerebellar cortex as an assembly of non-uniform Purkinje cell microcircuits

The adult mammalian cerebellar cortex is generally assumed to have a uniform cytoarchitecture. Differences in cerebellar function are thought to arise primarily through distinct patterns of input and output connectivity rather than as a result of variations in cortical microcircuitry. However, evidence from anatomical, physiological and genetic studies is increasingly challenging this orthodoxy, and there are now various lines of evidence indicating that the cerebellar cortex is not uniform. Here, we develop the hypothesis that regional differences in properties of cerebellar cortical microcircuits lead to important differences in information processing.

How the cerebellum may monitor sensory information for spatial representation

The cerebellum has already been shown to participate in the navigation function. We propose here that this structure is involved in maintaining a sense of direction and location during self-motion by monitoring sensory information and interacting with navigation circuits to update the mental representation of space. To better understand the processing performed by the cerebellum in the navigation function, we have reviewed: the anatomical pathways that convey self-motion information to the cerebellum; the computational algorithm(s) thought to be performed by the cerebellum from these multi-source inputs; the cerebellar outputs directed toward navigation circuits and the influence of self-motion information on space-modulated cells receiving cerebellar outputs. This review highlights that the cerebellum is adequately wired to combine the diversity of sensory signals to be monitored during self-motion and fuel the navigation circuits. The direct anatomical projections of the cerebellum toward the head-direction cell system and the parietal cortex make those structures possible relays of the cerebellum influence on the hippocampal spatial map. We describe computational models of the cerebellar function showing that the cerebellum can filter out the components of the sensory signals that are predictable, and provides a novelty output. We finally speculate that this novelty output is taken into account by the navigation structures, which implement an update over time of position and stabilize perception during navigation.

Cerebellum involvement in cortical sensorimotor circuits for the control of voluntary movements

Sensorimotor integration is crucial to perception and motor control. How and where this process takes place in the brain is still largely unknown. Here we analyze the cerebellar contribution to sensorimotor integration in the whisker system of mice. We identify an area in the cerebellum where cortical sensory and motor inputs converge at the cellular level. Optogenetic stimulation of this area affects thalamic and motor cortex activity, alters parameters of ongoing movements and thereby modifies qualitatively and quantitatively touch events against surrounding objects. These results shed light on the cerebellum as an active component of sensorimotor circuits and show the importance of sensorimotor cortico-cerebellar loops in the fine control of voluntary movements.

Plastic corollary discharge predicts sensory consequences of movements in a cerebellum-like circuit

The capacity to predict the sensory consequences of movements is critical for sensory, motor, and cognitive function. Though it is hypothesized that internal signals related to motor commands, known as corollary discharge, serve to generate such predictions, this process remains poorly understood at the neural circuit level. Here we demonstrate that neurons in the electrosensory lobe (ELL) of weakly electric mormyrid fish generate negative images of the sensory consequences of the fish's own movements based on ascending spinal corollary discharge signals. These results generalize previous findings describing mechanisms for generating negative images of the effects of the fish's specialized electric organ discharge (EOD) and suggest that a cerebellum-like circuit endowed with associative synaptic plasticity acting on corollary discharge can solve the complex and ubiquitous problem of predicting sensory consequences of movements.

Decorrelation learning in the cerebellum: computational analysis and experimental questions

Many cerebellar models use a form of synaptic plasticity that implements decorrelation learning. Parallel fibers carrying signals positively correlated with climbing-fiber input have their synapses weakened (long-term depression), whereas those carrying signals negatively correlated with climbing input have their synapses strengthened (long-term potentiation). Learning therefore ceases when all parallel-fiber signals have been decorrelated from climbing-fiber input. This is a computationally powerful rule for supervised learning and can be cast in a spike-timing dependent plasticity form for comparison with experimental evidence. Decorrelation learning is particularly well suited to sensory prediction, for example, in the reafference problem where external sensory signals are interfered with by reafferent signals from the organism's own movements, and the required circuit appears similar to the one found to mediate classical eye blink conditioning. However, for certain stimuli, avoidance is a much better option than simple prediction, and decorrelation learning can also be used to acquire appropriate avoidance movements. One example of a stimulus to be avoided is retinal slip that degrades visual processing, and decorrelation learning appears to play a role in the vestibulo-ocular reflex that stabilizes gaze in the face of unpredicted head movements. Decorrelation learning is thus suitable for both sensory prediction and motor control. It may also be well suited for generic spatial and temporal coordination, because of its ability to remove the unwanted side effects of movement. Finally, because it can be used with any kind of time-varying signal, the cerebellum could play a role in cognitive processing.

An adaptive filter model of cerebellar zone C3 as a basis for safe limb control?

The review asks how the adaptive filter model of the cerebellum might be relevant to experimental work on zone C3, one of the most extensively studied regions of cerebellar cortex. As far as features of the cerebellar microcircuit are concerned, the model appears to fit very well with electrophysiological discoveries concerning the importance of molecular layer interneurons and their plasticity, the significance of long-term potentiation and the striking number of silent parallel fibre synapses. Regarding external connectivity and functionality, a key feature of the adaptive filter model is its use of the decorrelation algorithm, which renders it uniquely suited to problems of sensory noise cancellation. However, this capacity can be extended to the avoidance of sensory interference, by appropriate movements of, for example, the eyes in the vestibulo-ocular reflex. Avoidance becomes particularly important when painful signals are involved, and as the climbing fibre input to zone C3 is extremely responsive to nociceptive stimuli, it is proposed that one function of this zone is the avoidance of pain by, for example, adjusting movements of the body to avoid self-harm. This hypothesis appears consistent with evidence from humans and animals concerning the role of the intermediate cerebellum in classically conditioned withdrawal reflexes, but further experiments focusing on conditioned avoidance are required to test the hypothesis more stringently. The proposed architecture may also be useful for automatic self-adjusting damage avoidance in robots, an important consideration for next generation 'soft' robots designed to interact with people.

Functional role of the cerebellum in gamma-band synchronization of the sensory and motor cortices

The cerebellum is an essential structure for the control of movement. It sends abundant ascending projections to the cerebral cortex via the thalamus, but its contribution to cortical activity remains largely unknown. Here we studied its influence on cortical neuronal activity in freely moving rats. We demonstrate an excitatory action of the cerebellum on the motor thalamus and the motor cortex. We also show that cerebellar inactivation disrupts the gamma-band coherence of local field potential between the sensory and motor cortices during whisking. In contrast, phase locking of neuronal activities to local gamma oscillations was preserved in the sensory and motor cortices by cerebellar inactivation. These results indicate that the cerebellum contributes to coordinated sensorimotor cortical activities during motor activation and thus participates in the multiregional cortical processing of information.

Adaptive filters and internal models: multilevel description of cerebellar function

Cerebellar function is increasingly discussed in terms of engineering schemes for motor control and signal processing that involve internal models. To address the relation between the cerebellum and internal models, we adopt the chip metaphor that has been used to represent the combination of a homogeneous cerebellar cortical microcircuit with individual microzones having unique external connections. This metaphor indicates that identifying the function of a particular cerebellar chip requires knowledge of both the general microcircuit algorithm and the chip's individual connections. Here we use a popular candidate algorithm as embodied in the adaptive filter, which learns to decorrelate its inputs from a reference ('teaching', 'error') signal. This algorithm is computationally powerful enough to be used in a very wide variety of engineering applications. However, the crucial issue is whether the external connectivity required by such applications can be implemented biologically. We argue that some applications appear to be in principle biologically implausible: these include the Smith predictor and Kalman filter (for state estimation), and the feedback-error-learning scheme for adaptive inverse control. However, even for plausible schemes, such as forward models for noise cancellation and novelty-detection, and the recurrent architecture for adaptive inverse control, there is unlikely to be a simple mapping between microzone function and internal model structure. This initial analysis suggests that cerebellar involvement in particular behaviours is therefore unlikely to have a neat classification into categories such as 'forward model'. It is more likely that cerebellar microzones learn a task-specific adaptive-filter operation which combines a number of signal-processing roles.