Memory decay and generalization following distinct motor learning mechanisms

Mechanisms of brain self-regulation: psychological factors, mechanistic models and neural substrates

While neurofeedback represents a promising tool for neuroscience and a brain self-regulation approach to psychological rehabilitation, the field faces several problems and challenges. Current research has shown great variability and even failure among human participants in learning to self-regulate target features of brain activity with neurofeedback. A better understanding of cognitive mechanisms, psychological factors and neural substrates underlying self-regulation might help improve neurofeedback's scientific and clinical practices. This article reviews the current understanding of the neural mechanisms of brain self-regulation by drawing on findings from human and animal studies in neurofeedback, brain-computer/machine interfaces and neuroprosthetics. In this article, we look closer at the following topics: cognitive processes and psychophysiological factors affecting self-regulation, theoretical models and neural substrates underlying self-regulation, and finally, we provide an outlook on the outstanding gaps in knowledge and technical challenges. This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Reinforcement Learning during Locomotion

When learning a new motor skill, people often must use trial and error to discover which movement is best. In the reinforcement learning framework, this concept is known as exploration and has been linked to increased movement variability in motor tasks. For locomotor tasks, however, increased variability decreases upright stability. As such, exploration during gait may jeopardize balance and safety, making reinforcement learning less effective. Therefore, we set out to determine if humans could acquire and retain a novel locomotor pattern using reinforcement learning alone. Young healthy male and female participants walked on a treadmill and were provided with binary reward feedback (indicated by a green checkmark on the screen) that was tied to a fixed monetary bonus, to learn a novel stepping pattern. We also recruited a comparison group who walked with the same novel stepping pattern but did so by correcting for target error, induced by providing real-time veridical visual feedback of steps and a target. In two experiments, we compared learning, motor variability, and two forms of motor memories between the groups. We found that individuals in the binary reward group did, in fact, acquire the new walking pattern by exploring (increasing motor variability). Additionally, while reinforcement learning did not increase implicit motor memories, it resulted in more accurate explicit motor memories compared with the target error group. Overall, these results demonstrate that humans can acquire new walking patterns with reinforcement learning and retain much of the learning over 24 h.

Motion state-dependent motor learning based on explicit visual feedback has limited spatiotemporal properties compared with adaptation to physical perturbations

We recently showed that subjects can learn motion state-dependent changes to motor output (temporal force patterns) based on explicit visual feedback of the equivalent force field (i.e., without the physical perturbation). Here, we examined the spatiotemporal properties of this learning compared with learning based on physical perturbations. There were two human subject groups and two experimental paradigms. One group (n = 40) experienced physical perturbations (i.e., a velocity-dependent force field, vFF), whereas the second (n = 40) was given explicit visual feedback (EVF) of the force-velocity relationship. In the latter, subjects moved in force channels and we provided visual feedback of the lateral force exerted during the movement, as well as the required force pattern based on movement velocity. In the first paradigm (spatial generalization), following vFF or EVF training, generalization of learning was tested by requiring subjects to move to 14 untrained target locations (0° to ±135° around the trained location). In the second paradigm (temporal stability), following training, we examined the decay of learning over eight delay periods (0 to 90 s). Results showed that learning based on EVF did not generalize to untrained directions, whereas the generalization for the vFF was significant for targets ≤ 45° away. In addition, the decay of learning for the EVF group was significantly faster than the FF group (a time constant of 2.72 ± 1.74 s vs. 12.53 ± 11.83 s). Collectively, our results suggest that recalibrating motor output based on explicit motion state information, in contrast to physical disturbances, uses learning mechanisms with limited spatiotemporal properties.NEW & NOTEWORTHY Adjustment of motor output based on limb motion state information can be achieved based on explicit information or from physical perturbations. Here, we investigated the spatiotemporal characteristics of short-term motor learning to determine the properties of the respective learning mechanisms. Our results suggest that adjustments based on physical perturbations are more temporally stable and applied over a greater spatial range than the learning based on explicit visual feedback, suggesting largely separate learning mechanisms.

Memory consolidation of sequence learning and dynamic adaptation during wakefulness

Motor learning involves acquiring new movement sequences and adapting motor commands to novel conditions. Labile motor memories, acquired through sequence learning and dynamic adaptation, undergo a consolidation process during wakefulness after initial training. This process stabilizes the new memories, leading to long-term memory formation. However, it remains unclear if the consolidation processes underlying sequence learning and dynamic adaptation are independent and if distinct neural regions underpin memory consolidation associated with sequence learning and dynamic adaptation. Here, we first demonstrated that the initially labile memories formed during sequence learning and dynamic adaptation were stabilized against interference through time-dependent consolidation processes occurring during wakefulness. Furthermore, we found that sequence learning memory was not disrupted when immediately followed by dynamic adaptation and vice versa, indicating distinct mechanisms for sequence learning and dynamic adaptation consolidation. Finally, by applying patterned transcranial magnetic stimulation to selectively disrupt the activity in the primary motor (M1) or sensory (S1) cortices immediately after sequence learning or dynamic adaptation, we found that sequence learning consolidation depended on M1 but not S1, while dynamic adaptation consolidation relied on S1 but not M1. For the first time in a single experimental framework, this study revealed distinct neural underpinnings for sequence learning and dynamic adaptation consolidation during wakefulness, with significant implications for motor skill enhancement and rehabilitation.

Implicit reward-based motor learning

Binary feedback, providing information solely about task success or failure, can be sufficient to drive motor learning. While binary feedback can induce explicit adjustments in movement strategy, it remains unclear if this type of feedback also induces implicit learning. We examined this question in a center-out reaching task by gradually moving an invisible reward zone away from a visual target to a final rotation of 7.5° or 25° in a between-group design. Participants received binary feedback, indicating if the movement intersected the reward zone. By the end of the training, both groups modified their reach angle by about 95% of the rotation. We quantified implicit learning by measuring performance in a subsequent no-feedback aftereffect phase, in which participants were told to forgo any adopted movement strategies and reach directly to the visual target. The results showed a small, but robust (2-3°) aftereffect in both groups, highlighting that binary feedback elicits implicit learning. Notably, for both groups, reaches to two flanking generalization targets were biased in the same direction as the aftereffect. This pattern is at odds with the hypothesis that implicit learning is a form of use-dependent learning. Rather, the results suggest that binary feedback can be sufficient to recalibrate a sensorimotor map.

Implicit reward-based motor learning

Binary feedback, providing information solely about task success or failure, can be sufficient to drive motor learning. While binary feedback can induce explicit adjustments in movement strategy, it remains unclear if this type of feedback also induce implicit learning. We examined this question in a center-out reaching task by gradually moving an invisible reward zone away from a visual target to a final rotation of 7.5° or 25° in a between-group design. Participants received binary feedback, indicating if the movement intersected the reward zone. By the end of the training, both groups modified their reach angle by about 95% of the rotation. We quantified implicit learning by measuring performance in a subsequent no-feedback aftereffect phase, in which participants were told to forgo any adopted movement strategies and reach directly to the visual target. The results showed a small, but robust (2-3°) aftereffect in both groups, highlighting that binary feedback elicits implicit learning. Notably, for both groups, reaches to two flanking generalization targets were biased in the same direction as the aftereffect. This pattern is at odds with the hypothesis that implicit learning is a form of use-dependent learning. Rather, the results suggest that binary feedback can be sufficient to recalibrate a sensorimotor map.