Sensory prediction error drives subconscious motor learning outside of the laboratory

Sensorimotor adaptation is supported by at least two parallel learning systems: an intentionally controlled explicit strategy and an involuntary implicit learning system. Past work focused on constrained reaches or finger movements in laboratory environments has shown subconscious learning systems to be driven in part by sensory prediction error (SPE), i.e., the mismatch between the realized and expected outcome of an action. We designed a ball rolling task to explore whether SPEs can drive implicit motor adaptation during complex whole body movements that impart physical motion on external objects. After applying a visual shift, participants rapidly adapted their rolling angles to reduce the error between the ball and the target. We removed all visual feedback and told participants to aim their throw directly toward the primary target, revealing an unintentional 5.06° implicit adjustment to reach angles that decayed over time. To determine whether this implicit adaptation was driven by SPE, we gave participants a second aiming target that would "solve" the visual shift, as in the study by Mazzoni and Krakauer (Mazzoni P, Krakauer JW. J Neurosci 26: 3642-3645, 2006). Remarkably, after rapidly reducing ball-rolling error to zero (due to enhancements in strategic aiming), the additional aiming target caused rolling angles to deviate beyond the primary target by 3.15°. This involuntary overcompensation, which worsened task performance, is a hallmark of SPE-driven implicit learning. These results show that SPE-driven implicit processes, previously observed within simplified finger or planar reaching movements, actively contribute to motor adaptation in more complex naturalistic skill-based tasks.NEW & NOTEWORTHY Implicit and explicit learning systems have been detected using simple, constrained movements inside the laboratory. How these systems impact movements during complex whole body, skill-based tasks has not been established. Here, we demonstrate that sensory prediction errors significantly impact how a person updates their movements, replicating findings from the laboratory in an unconstrained ball-rolling task. This real-world validation is an important step toward explaining how subconscious learning helps humans execute common motor skills in dynamic environments.

Bridging event-related potentials with behavioral studies in motor learning

Behavioral approaches and electrophysiology in understanding human sensorimotor systems have both yielded substantial advancements in past decades. In fact, behavioral neuroscientists have found that motor learning involves the two distinct processes of the implicit and the explicit. Separately, they have also distinguished two kinds of errors that drive motor learning: sensory prediction error and task error. Scientists in electrophysiology, in addition, have discovered two motor-related, event-related potentials (ERPs): error-related negativity (ERN), and feedback-related negativity (FRN). However, there has been a lack of interchange between the two lines of research. This article, therefore, will survey through the literature in both directions, attempting to establish a bridge between these two fruitful lines of research.

Mechanisms of Human Motor Learning Do Not Function Independently

Human motor learning is governed by a suite of interacting mechanisms each one of which modifies behavior in distinct ways and rely on different neural circuits. In recent years, much attention has been given to one type of motor learning, called motor adaptation. Here, the field has generally focused on the interactions of three mechanisms: sensory prediction error SPE-driven, explicit (strategy-based), and reinforcement learning. Studies of these mechanisms have largely treated them as modular, aiming to model how the outputs of each are combined in the production of overt behavior. However, when examined closely the results of some studies also suggest the existence of additional interactions between the sub-components of each learning mechanism. In this perspective, we propose that these sub-component interactions represent a critical means through which different motor learning mechanisms are combined to produce movement; understanding such interactions is critical to advancing our knowledge of how humans learn new behaviors. We review current literature studying interactions between SPE-driven, explicit, and reinforcement mechanisms of motor learning. We then present evidence of sub-component interactions between SPE-driven and reinforcement learning as well as between SPE-driven and explicit learning from studies of people with cerebellar degeneration. Finally, we discuss the implications of interactions between learning mechanism sub-components for future research in human motor learning.

A curtailed task for quantitative evaluation of visuomotor adaptation in the head-mounted display virtual reality environment

To accurately perform a goal-directed movement in continuously changing environments, it is unavoidable for individuals to adapt accordingly. The cerebellum has been known to be responsible for such process, specifically adaptation using sensorimotor information. As shown in previous studies, using HMD-VR technology in an experimental setting has similar advantages as in the real-world environment: researchers can manipulate the experimental environment, precisely control the experiments, and quantitatively analyze errors in real time. Moreover, the HMD-VR environment provides high immersiveness and embodiment which even enhance motor learning and increase engagement and motivation of individuals more than real-world environments do. In our HMD-VR-based task, the subjects were trained to adapt to a condition in which the visual information was artificially 20°clockwise rotated from the actual cursor movement. The subjects used a virtual reality tracker to move the cursor from a starting point to a target that appeared randomly at five locations, 20 cm from the starting point with an interval of 15°. Although no significant side effects were expected from experiencing the HMD-VR environment, we considered the appropriate number of trials for patients with cerebellar disease for future use in clinical settings. To examine the feasibility of our task for analysis of visuomotor adaptation pattern as shown in a real-world-based task, we created and compared two paradigms with a difference in the number of trials. As we expected, the results showed that the heading angle error decreased as the participants of both paradigms continued the task and that there was no significant difference between the two paradigms. Next, we applied our short task paradigm to patients diagnosed with cerebellar ataxia and age-matched controls for further examination of applicability to diagnosis and rehabilitation of the patients. As a result, we observed the distinguishable adaptation pattern of the patient group by using our paradigm. Overall, the results suggest that our paradigm is feasible to analyze the visuomotor adaptation pattern of healthy individuals and patients with cerebellar ataxia so may contribute to the clinical field.

Challenging the Boundaries of the Physical Self: Distal Cues Impact Body Ownership

The unique ability to identify one’s own body and experience it as one’s own is fundamental in goal-oriented behavior and survival. However, the mechanisms underlying the so-called body ownership are yet not fully understood. Evidence based on Rubber Hand Illusion (RHI) paradigms has demonstrated that body ownership is a product of reception and integration of self and externally generated multisensory information, feedforward and feedback processing of sensorimotor signals, and prior knowledge about the body. Crucially, however, these designs commonly involve the processing of proximal modalities while the contribution of distal sensory signals to the experience of ownership remains elusive. Here we propose that, like any robust percept, body ownership depends on the integration and prediction across all sensory modalities, including distal sensory signals pertaining to the environment. To test our hypothesis, we created an embodied goal-oriented Virtual Air Hockey Task, in which participants were to hit a virtual puck into a goal. In two conditions, we manipulated the congruency of distal multisensory cues (auditory and visual) while preserving proximal and action-driven signals entirely predictable. Compared to a fully congruent condition, our results revealed a significant decrease on three dimensions of ownership evaluation when distal signals were incongruent, including the subjective report as well as physiological and kinematic responses to an unexpected threat. Together, these findings support the notion that the way we represent our body is contingent upon all the sensory stimuli, including distal and action-independent signals. The present data extend the current framework of body ownership and may also find applications in rehabilitation scenarios.

Sensory prediction errors in the human midbrain signal identity violations independent of perceptual distance

The firing of dopaminergic midbrain neurons is thought to reflect prediction errors (PE) that depend on the difference between the value of expected and received rewards. However, recent work has demonstrated that unexpected changes in value-neutral outcome features, such as identity, can evoke similar responses. It remains unclear whether the magnitude of these identity PEs scales with the perceptual dissimilarity of expected and received rewards, or whether they are independent of perceptual similarity. We used a Pavlovian transreinforcer reversal task to elicit identity PEs for value-matched food odor rewards, drawn from two perceptual categories (sweet, savory). Replicating previous findings, identity PEs were correlated with fMRI activity in midbrain, OFC, piriform cortex, and amygdala. However, the magnitude of identity PE responses was independent of the perceptual distance between expected and received outcomes, suggesting that identity comparisons underlying sensory PEs may occur in an abstract state space independent of straightforward sensory percepts.

Cerebellum, Predictions and Errors

Making predictions and validating the predictions against actual sensory information is thought to be one of the most fundamental functions of the nervous system. A growing body of evidence shows that the neural mechanisms controlling behavior, both in motor and non-motor domains, rely on prediction errors, the discrepancy between predicted and actual information. The cerebellum has been viewed as a key component of the motor system providing predictions about upcoming movements and receiving feedback about motor errors. Consequentially, studies of cerebellar function have focused on the motor domain with less consideration for the wider context in which movements are generated. However, motor learning experiments show that cognition makes important contributions to motor adaptation that involves the cerebellum. One of the more successful theoretical frameworks for understanding motor control and cerebellar function is the forward internal model which states that the cerebellum predicts the sensory consequences of the motor commands and is involved in computing sensory prediction errors by comparing the predictions to the sensory feedback. The forward internal model was applied and tested mainly for effector movements, raising the question whether cerebellar encoding of behavior reflects task performance measures associated with cognitive involvement. Electrophysiological studies based on pseudo-random tracking in monkeys show that the discharge of Purkinje cell, the sole output neurons of the cerebellar cortex, encodes predictive and feedback signals not only of the effector kinematics but also of task performance. The implications are that the cerebellum implements both effector and task performance forward models and the latter are consistent with the cognitive contributions observed during motor learning. The implications of these findings include insights into recent psychophysical observations on moving with reduced feedback and motor learning. The findings also support the cerebellum's place in hierarchical generative models that work in concert to refine predictions about behavior and the world. Therefore, cerebellar representations bridge motor and non-motor domains and provide a better understanding of cerebellar function within the functional architecture of the brain.

Purkinje Cell Representations of Behavior: Diary of a Busy Neuron

Fundamental for understanding cerebellar function is determining the representations in Purkinje cell activity, the sole output of the cerebellar cortex. Up to the present, the most accurate descriptions of the information encoded by Purkinje cells were obtained in the context of motor behavior and reveal a high degree of heterogeneity of kinematic and performance error signals encoded. The most productive framework for organizing Purkinje cell firing representations is provided by the forward internal model hypothesis. Direct tests of this hypothesis show that individual Purkinje cells encode two different forward models simultaneously, one for effector kinematics and one for task performance. Newer results demonstrate that the timing of simple spike encoding of motor parameters spans an extend interval of up to ±2 seconds. Furthermore, complex spike discharge is not limited to signaling errors, can be predictive, and dynamically controls the information in the simple spike firing to meet the demands of upcoming behavior. These rich, diverse, and changing representations highlight the integrative aspects of cerebellar function and offer the opportunity to generalize the cerebellar computational framework over both motor and non-motor domains.

Somatosensory Cortex Plays an Essential Role in Forelimb Motor Adaptation in Mice

Our motor outputs are constantly re-calibrated to adapt to systematic perturbations. This motor adaptation is thought to depend on the ability to form a memory of a systematic perturbation, often called an internal model. However, the mechanisms underlying the formation, storage, and expression of such models remain unknown. Here, we developed a mouse model to study forelimb adaptation to force field perturbations. We found that temporally precise photoinhibition of somatosensory cortex (S1) applied concurrently with the force field abolished the ability to update subsequent motor commands needed to reduce motor errors. This S1 photoinhibition did not impair basic motor patterns, post-perturbation completion of the action, or their performance in a reward-based learning task. Moreover, S1 photoinhibition after partial adaptation blocked further adaptation, but did not affect the expression of already-adapted motor commands. Thus, S1 is critically involved in updating the memory about the perturbation that is essential for forelimb motor adaptation.

Spinal microcircuits comprising dI3 interneurons are necessary for motor functional recovery following spinal cord transection

The spinal cord has the capacity to coordinate motor activities such as locomotion. Following spinal transection, functional activity can be regained, to a degree, following motor training. To identify microcircuits involved in this recovery, we studied a population of mouse spinal interneurons known to receive direct afferent inputs and project to intermediate and ventral regions of the spinal cord. We demonstrate that while dI3 interneurons are not necessary for normal locomotor activity, locomotor circuits rhythmically inhibit them and dI3 interneurons can activate these circuits. Removing dI3 interneurons from spinal microcircuits by eliminating their synaptic transmission left locomotion more or less unchanged, but abolished functional recovery, indicating that dI3 interneurons are a necessary cellular substrate for motor system plasticity following transection. We suggest that dI3 interneurons compare inputs from locomotor circuits with sensory afferent inputs to compute sensory prediction errors that then modify locomotor circuits to effect motor recovery.

Credit assignment in movement-dependent reinforcement learning

When a person fails to obtain an expected reward from an object in the environment, they face a credit assignment problem: Did the absence of reward reflect an extrinsic property of the environment or an intrinsic error in motor execution? To explore this problem, we modified a popular decision-making task used in studies of reinforcement learning, the two-armed bandit task. We compared a version in which choices were indicated by key presses, the standard response in such tasks, to a version in which the choices were indicated by reaching movements, which affords execution failures. In the key press condition, participants exhibited a strong risk aversion bias; strikingly, this bias reversed in the reaching condition. This result can be explained by a reinforcement model wherein movement errors influence decision-making, either by gating reward prediction errors or by modifying an implicit representation of motor competence. Two further experiments support the gating hypothesis. First, we used a condition in which we provided visual cues indicative of movement errors but informed the participants that trial outcomes were independent of their actual movements. The main result was replicated, indicating that the gating process is independent of participants' explicit sense of control. Second, individuals with cerebellar degeneration failed to modulate their behavior between the key press and reach conditions, providing converging evidence of an implicit influence of movement error signals on reinforcement learning. These results provide a mechanistically tractable solution to the credit assignment problem.