Persistence of reduced neuromotor noise in long-term motor skill learning

Simulated operant reflex conditioning environment reveals effects of feedback parameters

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, biological variability, and reward threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), biological variability (low, high), and reward threshold (easy, moderate, difficult) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

OPERANT REFLEX CONDITIONING SIMULATION ENVIRONMENT REVEALS EFFECTS OF FEEDBACK PARAMETERS

Operant conditioning of neural activation has been researched for decades in humans and animals. Many theories suggest two parallel learning processes, implicit and explicit. The degree to which feedback affects these processes individually remains to be fully understood and may contribute to a large percentage of non-learners. Our goal is to determine the explicit decision-making processes in response to feedback representing an operant conditioning environment. We developed a simulated operant conditioning environment based on a feedback model of spinal reflex excitability, one of the simplest forms of neural operant conditioning. We isolated the perception of the feedback signal from self-regulation of an explicit unskilled visuomotor task, enabling us to quantitatively examine feedback strategy. Our hypothesis was that feedback type, signal quality and success threshold affect operant conditioning performance and operant strategy. Healthy individuals (N = 41) were instructed to play a web application game using keyboard inputs to rotate a virtual knob representative of an operant strategy. The goal was to align the knob with a hidden target. Participants were asked to "down-condition" the amplitude of the virtual feedback signal, which was achieved by placing the knob as close as possible to the hidden target. We varied feedback type (knowledge of performance, knowledge of results), success threshold (easy, moderate, difficult), and biological variability (low, high) in a factorial design. Parameters were extracted from real operant conditioning data. Our main outcomes were the feedback signal amplitude (performance) and the mean change in dial position (operant strategy). We observed that performance was modulated by variability, while operant strategy was modulated by feedback type. These results show complex relations between fundamental feedback parameters and provide the principles for optimizing neural operant conditioning for non-responders.

Effect of Handedness on Learned Controllers and Sensorimotor Noise During Trajectory-Tracking

In human-in-the-loop control systems, operators can learn to manually control dynamic machines with either hand using a combination of reactive (feedback) and predictive (feedforward) control. This article studies the effect of handedness on learned controllers and performance during a trajectory-tracking task. In an experiment with 18 participants, subjects perform an assay of unimanual trajectory-tracking and disturbance-rejection tasks through second-order machine dynamics, first with one hand then the other. To assess how hand preference (or dominance) affects learned controllers, we extend, validate, and apply a nonparametric modeling method to estimate the concurrent feedback and feedforward controllers. We find that performance improves because feedback adapts, regardless of the hand used. We do not detect statistically significant differences in performance or learned controllers between hands. Adaptation to reject disturbances arising exogenously (i.e., applied by the experimenter) and endogenously (i.e., generated by sensorimotor noise) explains observed performance improvements.

Skilled motor control of an inverted pendulum implies low entropy of states but high entropy of actions

The mastery of skills, such as balancing an inverted pendulum, implies a very accurate control of movements to achieve the task goals. Traditional accounts of skilled action control that focus on either routinization or perceptual control make opposite predictions about the ways we achieve mastery. The notion of routinization emphasizes the decrease of the variance of our actions, whereas the notion of perceptual control emphasizes the decrease of the variance of the states we visit, but not of the actions we execute. Here, we studied how participants managed control tasks of varying levels of difficulty, which consisted of controlling inverted pendulums of different lengths. We used information-theoretic measures to compare the predictions of alternative accounts that focus on routinization and perceptual control, respectively. Our results indicate that the successful performance of the control task strongly correlates with the decrease of state variability and the increase of action variability. As postulated by perceptual control theory, the mastery of skilled pendulum control consists in achieving stable control of goals by flexible means.

Back to reality: differences in learning strategy in a simplified virtual and a real throwing task

Virtual environments have been widely used in motor neuroscience and rehabilitation, as they afford tight control of sensorimotor conditions and readily afford visual and haptic manipulations. However, typically, studies have only examined performance in the virtual testbeds, without asking how the simplified and controlled movement in the virtual environment compares to behavior in the real world. To test whether performance in the virtual environment was a valid representation of corresponding behavior in the real world, this study compared throwing in a virtual set-up with realistic throwing, where the task parameters were precisely matched. Even though the virtual task only required a horizontal single-joint arm movement, similar to many simplified movement assays in motor neuroscience, throwing accuracy and precision were significantly worse than in the real task that involved all degrees of freedom of the arm; only after 3 practice days did success rate and error reach similar levels. To gain more insight into the structure of the learning process, movement variability was decomposed into deterministic and stochastic contributions. Using the tolerance-noise-covariation decomposition method, distinct stages of learning were revealed: While tolerance was optimized first in both environments, it was higher in the virtual environment, suggesting that more familiarization and exploration was needed in the virtual task. Covariation and noise showed more contributions in the real task, indicating that subjects reached the stage of fine-tuning of variability only in the real task. These results showed that while the tasks were precisely matched, the simplified movements in the virtual environment required more time to become successful. These findings resonate with the reported problems in transfer of therapeutic benefits from virtual to real environments and alert that the use of virtual environments in research and rehabilitation needs more caution.NEW & NOTEWORTHY This study compared human performance of the same throwing task in a real and a matched virtual environment. With 3 days' practice, subjects improved significantly faster in the real task, even though the arm and hand movements were more complex. Decomposing variability revealed that performance in the virtual environment, despite its simplified hand movements, required more exploration. Additionally, due to fewer constraints in the real task, subjects could modify the geometry of the solution manifold, by shifting the release position, and thereby simplify the task.

Multiple processes independently predict motor learning

Background

Our ability to acquire, refine and adapt skilled limb movements is a hallmark of human motor learning that allows us to successfully perform many daily activities. The capacity to acquire, refine and adapt other features of motor performance, such as visual search, eye-hand coordination and visuomotor decisions, may also contribute to motor learning. However, the extent to which refinements of multiple behavioral features and their underlying neural processes independently contribute to motor learning remains unknown. In the current study, we used an ethological approach to test the hypothesis that practice-related refinements of multiple behavioral features would be independently predictive of motor learning.

Methods

Eighteen healthy, young adults used an upper-limb robot with eye-tracking to practice six trials of a continuous, visuomotor task once a week for six consecutive weeks. Participants used virtual paddles to hit away 200 “Targets” and avoid hitting 100 “Distractors” that continuously moved towards them from the back of the workspace. Motor learning was inferred from trial-by-trial acquisition and week-by-week retention of improvements on two measures of task performance related to motor execution and motor inhibition. Adaptations involving underlying neural processes were inferred from trial-by-trial acquisition and week-by-week retention of refinements on measures of skilled limb movement, visual search, eye-hand coordination and visuomotor decisions. We tested our hypothesis by quantifying the extent to which refinements on measures of multiple behavioral features (predictors) were independently predictive of improvements on our two measures of task performance (outcomes) after removing all shared variance between predictors.

Results

We found that refinements on measures of skilled limb movement, visual search and eye-hand coordination were independently predictive of improvements on our measure of task performance related to motor execution. In contrast, only refinements of eye-hand coordination were independently predictive of improvements on our measure of task performance related to motor inhibition.

Conclusion

Our results provide indirect evidence that refinements involving multiple, neural processes may independently contribute to motor learning, and distinct neural processes may underlie improvements in task performance related to motor execution and motor inhibition. This also suggests that refinements involving multiple, neural processes may contribute to motor recovery after stroke, and rehabilitation interventions should be designed to produce refinements of all behavioral features that may contribute to motor recovery.

Reward modulates the association between sensory noise and brain activity during perceptual decision-making

Perceptual decisions entail the accumulation of evidence until a decision criterion is reached. The amount of noise in this process is inversely related to the behavioral performance of the decision-maker. Hence, reducing the amount of perceived noise could improve performance in perceptual decisions. In this study, we investigated whether providing monetary reward for correct responses in a perceptual decision-making task would enhance performance based on prior research linking noise reduction to the administration of reward. To this end, thirty-one healthy young adults carried out an incentivized dot tracking task (iDT) during recording of functional magnetic resonance imaging (fMRI). Behavioral responses were fitted to a Bayesian version of the drift-diffusion model that, among other parameters, also includes an estimate of sensory noise. Fifty percent of the trials were incentivized to compare rewarded with unrewarded trials regarding behavior, brain responses and estimates of model parameters. In order to establish a link between the noise parameter and fMRI activity, we correlated percent signal change (PSC) values from nucleus accumbens and caudate nucleus with noise levels in rewarded and unrewarded trials respectively. Although reward did not affect behavioral performance and model parameters, the fMRI analyses showed notable differences in nucleus accumbens, caudate nucleus and rostral anterior cingulate cortex in rewarded relative to unrewarded trials. Furthermore, higher PSC within nucleus accumbens was significantly associated with lower sensory noise levels, which was specific to rewarded trials. This work is consistent with previous findings on reward modulation of brain responses and marks a first step towards elucidating the effects of reward-induced noise suppression during perceptual decision-making.

Learning and transfer of complex motor skills in virtual reality: a perspective review

The development of more effective rehabilitative interventions requires a better understanding of how humans learn and transfer motor skills in real-world contexts. Presently, clinicians design interventions to promote skill learning by relying on evidence from experimental paradigms involving simple tasks, such as reaching for a target. While these tasks facilitate stringent hypothesis testing in laboratory settings, the results may not shed light on performance of more complex real-world skills. In this perspective, we argue that virtual environments (VEs) are flexible, novel platforms to evaluate learning and transfer of complex skills without sacrificing experimental control. Specifically, VEs use models of real-life tasks that afford controlled experimental manipulations to measure and guide behavior with a precision that exceeds the capabilities of physical environments. This paper reviews recent insights from VE paradigms on motor learning into two pressing challenges in rehabilitation research: 1) Which training strategies in VEs promote complex skill learning? and 2) How can transfer of learning from virtual to real environments be enhanced? Defining complex skills by having nested redundancies, we outline findings on the role of movement variability in complex skill acquisition and discuss how VEs can provide novel forms of guidance to enhance learning. We review the evidence for skill transfer from virtual to real environments in typically developing and neurologically-impaired populations with a view to understanding how differences in sensory-motor information may influence learning strategies. We provide actionable suggestions for practicing clinicians and outline broad areas where more research is required. Finally, we conclude that VEs present distinctive experimental platforms to understand complex skill learning that should enable transfer from therapeutic practice to the real world.

Gender Differences in Throwing Revisited: Sensorimotor Coordination in a Virtual Ball Aiming Task

Numerous studies have demonstrated that boys throw balls faster, farther and more accurately than girls. This may be largely due to well-known anatomical and muscle-physiological differences that play a central role in overarm throwing. With the objective to understand the potential contribution of the equally essential coordinative aspects in throwing for this gender difference, this large cross-sectional study examined a simplified forearm throw that eliminated the requirements that give males an advantage.While the overall performance error indeed became similar in the age groups younger than 20 years and older than 50 years, it was attenuated for middle-aged individuals. The gender differences remained in individuals who reported no throwing experience, but females with throwing experience reached similar performance as males. Two fine-grained spatiotemporal metrics displayed similar age-dependent gender disparities: while overall, males showed better spatiotemporal coordination of the ball release, age group comparisons specified that it was particularly middle-aged females that made more timing errors and did not develop a noise-tolerant strategy as males did. As throwing experience did not explain this age-dependency, the results are discussed in the context of spatial abilities and video game experience, both more pronounced in males. In contrast, a measure of rhythmicity developed over successive throws only revealed weak gender differences, speaking to the fundamental tendency in humans to fall into rhythmic patterns. Only the youngest individuals between 5 and 9 years of age showed significantly less rhythmicity in their performance. This computational study was performed in a large cohort in the context of an outreach activity, demonstrating that robust quantitative measures can also be obtained in less controlled environments. The findings also alert that motor neuroscience may need to pay more attention to gender differences.

Differential control of task and null space variability in response to changes in task difficulty when learning a bimanual steering task

The presence of motor redundancy means that movement variability can be split into a 'task-space' component that affects task performance, and a 'null space' component which has no effect on task performance. While the control of task-space variability during learning is essential, because it is directly linked to performance, how the nervous system controls null space variability during learning has not been well understood. One factor that has been hypothesized to govern the change in null space variability with learning is task difficulty, but this has not been directly tested. Here, we examined how task difficulty influences the change in null space variability with learning. Healthy, college-aged participants (N = 36) performed a bimanual steering task, where they steered a cursor through a smooth W-shaped track of a certain width as quickly as possible while attempting to keep the cursor within the track. Task difficulty was altered by changing the track width and participants were split into one of the three groups based on the track width that they practiced on-wide, narrow, or progressive (where the width of the track progressively changed from wide to narrow over practice). The redundancy in this task arose from the fact that the position of the cursor was defined as the average position of the two hands. Results showed that movement time depended on task difficulty, but all groups were able to decrease their movement time with practice. Learning was associated with a reduction in null space variability in all groups, but critically, there was no effect of task difficulty. Further analyses showed that while the task-space variability showed an expected speed-accuracy tradeoff with movement time, the null space variability showed a qualitatively different pattern. These results suggest differential control of task and null space variability in response to changes in task difficulty with learning, and may reflect a strong preference to minimize overall movement variability during learning.

It's Not (Only) the Mean that Matters: Variability, Noise and Exploration in Skill Learning

Mastering a motor skill is typified by a decrease in variability. However, variability is much more than the undesired signature of discoordination: structure in both its distributional properties and temporal sequence can reveal control priorities. Extending from the notion that signal-dependent noise corrupts information transmission in the neuromotor system, this review tracks more recent recognitions that the complex dynamic motor system in its interaction with task constraints creates high-dimensional spaces with multiple equivalent solutions. Further analysis differentiates these solutions to have different degrees of noise-sensitivity, goal-relevance or additional costs. Practice proceeds from exploration of these solution spaces to then exploitation with further channeling of noise. Extended practice leads to fine-tuning of skill brought about by reducing noise. These distinct changes in variability are suggested as a way to characterize stages of learning. Capitalizing on the sensitivity of the CNS to noise, interventions can add extrinsic or amplify intrinsic noise to guide (re-)learning desired behaviors. The persistence and generalization of acquired skill is still largely understudied, although an essential element of skill. Consistent with advances in the physical sciences, there is increasing realization that noise can have beneficial effects. Analysis of the non-random structure of variability may reveal more than analysis of only its mean.

Exploiting the geometry of the solution space to reduce sensitivity to neuromotor noise

Throwing is a uniquely human skill that requires a high degree of coordination to successfully hit a target. Timing of ball release appears crucial as previous studies report required timing accuracies as short as 1-2ms, which however appear physiologically challenging. This study mathematically and experimentally demonstrates that humans can overcome these seemingly stringent timing requirements by shaping their hand trajectories to create extended timing windows, where ball releases achieve target hits despite temporal imprecision. Subjects practiced four task variations in a virtual environment, each with a distinct geometry of the solution space and different demands for timing. Model-based analyses of arm trajectories revealed that subjects first decreased timing error, followed by lengthening timing windows in their hand trajectories. This pattern was invariant across solution spaces, except for a control case. Hence, the exquisite skill that humans evolved for throwing is achieved by developing strategies that are less sensitive to temporal variability arising from neuromotor noise. This analysis also provides an explanation why coaches emphasize the "follow-through" in many ball sports.

High variability impairs motor learning regardless of whether it affects task performance

Motor variability plays an important role in motor learning, although the exact mechanisms of how variability affects learning are not well understood. Recent evidence suggests that motor variability may have different effects on learning in redundant tasks, depending on whether it is present in the task space (where it affects task performance) or in the null space (where it has no effect on task performance). We examined the effect of directly introducing null and task space variability using a manipulandum during the learning of a motor task. Participants learned a bimanual shuffleboard task for 2 days, where their goal was to slide a virtual puck as close as possible toward a target. Critically, the distance traveled by the puck was determined by the sum of the left- and right-hand velocities, which meant that there was redundancy in the task. Participants were divided into five groups, based on both the dimension in which the variability was introduced and the amount of variability that was introduced during training. Results showed that although all groups were able to reduce error with practice, learning was affected more by the amount of variability introduced rather than the dimension in which variability was introduced. Specifically, groups with higher movement variability during practice showed larger errors at the end of practice compared with groups that had low variability during learning. These results suggest that although introducing variability can increase exploration of new solutions, this may adversely affect the ability to retain the learned solution.

NEW & NOTEWORTHY We examined the role of introducing variability during motor learning in a redundant task. The presence of redundancy allows variability to be introduced in different dimensions: the task space (where it affects task performance) or the null space (where it does not affect task performance). We found that introducing variability affected learning adversely, but the amount of variability was more critical than the dimension in which variability was introduced.

Influence of Speed and Accuracy Constraints on Motor Learning for a Trajectory-Based Movement

This study investigated the influences of task constraint on motor learning for a trajectory-based movement considering the speed-accuracy relationship. In the experiment, participants practiced trajectory-based movements for five consecutive days. The participants were engaged in training with time-minimization or time-matching constraints. The results demonstrated that the speed-accuracy tradeoff was not apparent or was weak in the training situation. When the participants practiced the movement with a time-minimization constraint, movement errors did not vary, whereas the movement time decreased. With the time-matching constraint, the errors decreased as a session proceeded. These results were discussed in terms of the combination of signal-dependent noises and exploratory search noises. It is suggested that updating spatial and temporal factors does not appear to occur simultaneously in motor learning.