Structure learning in action

Guiding functional reorganization of motor redundancy using a body-machine interface

Background

Body-machine interfaces map movements onto commands to external devices. Redundant motion signals derived from inertial sensors are mapped onto lower-dimensional device commands. Then, the device users face two problems, a) the structural problem of understanding the operation of the interface and b) the performance problem of controlling the external device with high efficiency. We hypothesize that these problems, while being distinct are connected in that aligning the space of body movements with the space encoded by the interface, i.e. solving the structural problem, facilitates redundancy resolution towards increasing efficiency, i.e. solving the performance problem.

Methods

Twenty unimpaired volunteers practiced controlling the movement of a computer cursor by moving their arms. Eight signals from four inertial sensors were mapped onto the two cursor’s coordinates on a screen. The mapping matrix was initialized by asking each user to perform free-form spontaneous upper-limb motions and deriving the two main principal components of the motion signals. Participants engaged in a reaching task for 18 min, followed by a tracking task. One group of 10 participants practiced with the same mapping throughout the experiment, while the other 10 with an adaptive mapping that was iteratively updated by recalculating the principal components based on ongoing movements.

Results

Participants quickly reduced reaching time while also learning to distribute most movement variance over two dimensions. Participants with the fixed mapping distributed movement variance over a subspace that did not match the potent subspace defined by the interface map. In contrast, participant with the adaptive map reduced the difference between the two subspaces, resulting in a smaller amount of arm motions distributed over the null space of the interface map. This, in turn, enhanced movement efficiency without impairing generalization from reaching to tracking.

Conclusions

Aligning the potent subspace encoded by the interface map to the user’s movement subspace guides redundancy resolution towards increasing movement efficiency, with implications for controlling assistive devices. In contrast, in the pursuit of rehabilitative goals, results would suggest that the interface must change to drive the statistics of user’s motions away from the established pattern and toward the engagement of movements to be recovered.

Trial registration

ClinicalTrials.gov, NCT01608438, Registered 16 April 2012.

Self-controlled practice and nudging during structural learning of a novel control interface

Self-controlled practice schedules have been shown to enhance motor learning in several contexts, but their effectiveness in structural learning tasks, where the goal is to eventually learn an underlying structure or rule, is not well known. Here we examined the use of self-controlled practice in a novel control interface requiring structural learning. In addition, we examined the effect of ‘nudging’–i.e., whether altering task difficulty could influence self-selected strategies, and hence facilitate learning. Participants wore four inertial measurement units (IMUs) on their upper body and the goal was to use motions of the upper body to move a screen cursor to different targets presented on the screen. The structure in this task that had to be learned was based on the fact that the signals from the IMUs were linearly mapped to the x- and y- position of the cursor. Participants (N = 62) were split into 3 groups (random, self-selected, nudge) based on whether they had control over the sequence in which they could practice the targets. To test whether participants learned the underlying structure, participants were tested both on the trained targets, as well as novel targets that were not practiced during training. Results showed that during training, the self-selected group showed shorter movement times relative to the random group, and both self-selected and nudge groups adopted a strategy of tending to repeat targets. However, in the test phase, we found no significant differences in task performance between groups, indicating that structural learning was not reliably affected by the type of practice. In addition, nudging participants by adjusting task difficulty did not show any significant benefits to overall learning. These results suggest that although self-controlled practice influenced practice structure and facilitated learning, it did not provide any additional benefits relative to practicing on a random schedule in this task.

Retrospective Inference as a Form of Bounded Rationality, and Its Beneficial Influence on Learning

Probabilistic models of cognition typically assume that agents make inferences about current states by combining new sensory information with fixed beliefs about the past, an approach known as Bayesian filtering. This is computationally parsimonious, but, in general, leads to suboptimal beliefs about past states, since it ignores the fact that new observations typically contain information about the past as well as the present. This is disadvantageous both because knowledge of past states may be intrinsically valuable, and because it impairs learning about fixed or slowly changing parameters of the environment. For these reasons, in offline data analysis it is usual to infer on every set of states using the entire time series of observations, an approach known as (fixed-interval) Bayesian smoothing. Unfortunately, however, this is impractical for real agents, since it requires the maintenance and updating of beliefs about an ever-growing set of states. We propose an intermediate approach, finite retrospective inference (FRI), in which agents perform update beliefs about a limited number of past states (Formally, this represents online fixed-lag smoothing with a sliding window). This can be seen as a form of bounded rationality in which agents seek to optimize the accuracy of their beliefs subject to computational and other resource costs. We show through simulation that this approach has the capacity to significantly increase the accuracy of both inference and learning, using a simple variational scheme applied to both randomly generated Hidden Markov models (HMMs), and a specific application of the HMM, in the form of the widely used probabilistic reversal task. Our proposal thus constitutes a theoretical contribution to normative accounts of bounded rationality, which makes testable empirical predictions that can be explored in future work.

Inference-Based Decisions in a Hidden State Foraging Task: Differential Contributions of Prefrontal Cortical Areas

Essential features of the world are often hidden and must be inferred by constructing internal models based on indirect evidence. Here, to study the mechanisms of inference, we establish a foraging task that is naturalistic and easily learned yet can distinguish inference from simpler strategies such as the direct integration of sensory data. We show that both mice and humans learn a strategy consistent with optimal inference of a hidden state. However, humans acquire this strategy more than an order of magnitude faster than mice. Using optogenetics in mice, we show that orbitofrontal and anterior cingulate cortex inactivation impacts task performance, but only orbitofrontal inactivation reverts mice from an inference-based to a stimulus-bound decision strategy. These results establish a cross-species paradigm for studying the problem of inference-based decision making and begins to dissect the network of brain regions crucial for its performance.

Analyzing Abstraction and Hierarchical Decision-Making in Absolute Identification by Information-Theoretic Bounded Rationality

In the face of limited computational resources, bounded rational decision theory predicts that information-processing should be concentrated on actions that make a significant contribution in terms of the utility achieved. Accordingly, information-processing can be simplified by choosing stereotypic actions that lead to satisfactory performance over a range of different inputs rather than choosing a specific action for each input. Such a set of similar inputs with similar action responses would then correspond to an abstraction that can be harnessed with possibly negligible loss in utility, but with potentially considerable savings in information-processing effort. Here we test this prediction in an identification task, where human subjects were asked to estimate the roundness of ellipses varying from a straight line to a perfect circle. Crucially, when reporting their estimates, subjects could choose between three different levels of precision corresponding to three levels of abstraction in a decision-making hierarchy. To induce changes in level selection, we manipulated the information-processing resources available at the perceptual and action stages by varying the difficulty of identifying the stimulus and by enforcing different response times in the action stage. In line with theoretical predictions, we find that subjects adapt their abstraction level depending on the available resources. We compare subjects' behavior to the maximum efficiency predicated by the bounded rational decision-making model and investigate possible sources of inefficiency.

Search Strategies in the Perceptual-Motor Workspace and the Acquisition of Coordination, Control, and Skill

In this paper we re-visit and elaborate-on the theoretical framework of learning as searching within the perceptual-motor workspace for a solution to the task. The central focus is the nature of search strategies to locate and create stable equilibrium regions in the perceptual-motor workspace and how these strategies relate to the emergent movement forms in the acquisition of coordination, control, and skill. In the ecological theory of perception and action, the enhanced stability of performance occurs through the attunement of the perceptual systems to the task dynamics together with modifications of action as task and intrinsic dynamics cooperate and/or compete. Thus, through practice in this search process, individuals adapt to the pick-up of task relevant perceptual variables and change their movement form according to the stability of the performed action and its outcome in relation to the task demands. Contemporary experimental findings have revealed features of the search process given the interaction of individual intrinsic dynamics in the context of task requirements and principles that drive the change – e.g., exploitation of more tolerant task-space solutions and emergence of compensatory mechanisms. Finally, we outline how the search strategy framework relates to traditional learning-related phenomena: including the dynamical pathways of learning, learning curves, factors of learning, individuality, motor development, and sport and rehabilitation interventions.

Interactions between motor exploration and reinforcement learning

Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning.NEW & NOTEWORTHY Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.

Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies

On-going developments in myoelectric prosthesis control have provided prosthesis users with an assortment of control strategies that vary in reliability and performance. Many studies have focused on improving performance by providing feedback to the user but have overlooked the effect of this feedback on internal model development, which is key to improve long-term performance. In this paper, the strength of internal models developed for two commonly used myoelectric control strategies: raw control with raw feedback (using a regression-based approach) and filtered control with filtered feedback (using a classifier-based approach), were evaluated using two psychometric measures: trial-by-trial adaptation and just-noticeable difference. The performance of both strategies was also evaluated using Schmidt's style target acquisition task. Results obtained from 24 able-bodied subjects showed that although filtered control with filtered feedback had better short-term performance in path efficiency ( ), raw control with raw feedback resulted in stronger internal model development ( ), which may lead to better long-term performance. Despite inherent noise in the control signals of the regression controller, these findings suggest that rich feedback associated with regression control may be used to improve human understanding of the myoelectric control system.

The visual geometry of a tool modulates generalization during adaptation

Knowledge about a tool’s dynamics can be acquired from the visual configuration of the tool and through physical interaction. Here, we examine how visual information affects the generalization of dynamic learning during tool use. Subjects rotated a virtual hammer-like object while we varied the object dynamics separately for two rotational directions. This allowed us to quantify the coupling of adaptation between the directions, that is, how adaptation transferred from one direction to the other. Two groups experienced the same dynamics of the object. For one group, the object’s visual configuration was displayed, while for the other, the visual display was uninformative as to the dynamics. We fit a range of context-dependent state-space models to the data, comparing different forms of coupling. We found that when the object’s visual configuration was explicitly provided, there was substantial coupling, such that 31% of learning in one direction transferred to the other. In contrast, when the visual configuration was ambiguous, despite experiencing the same dynamics, the coupling was reduced to 12%. Our results suggest that generalization of dynamic learning of a tool relies, not only on its dynamic behaviour, but also on the visual configuration with which the dynamics is associated.

Infants differentially update their internal models of a dynamic environment

Unexpected events provide us with opportunities for learning about what to expect from the world around us. Using a saccadic-planning paradigm, we investigated whether and how infants and adults represent the statistics of a changing environment (i.e. build an internal model of the environment). Participants observed differently colored bees that appeared at an unexpected location every few trials. The color cues indicated whether the subsequent bees would appear at this new location (i.e. update trials) or at the same location as previously (i.e. no-update trials). Infants learned the predictive value of the color cues and updated their internal models when necessary. Unlike infants, adults had a tendency to update their models each time they observed a change in the structure. We argue that infants are open to learning from current evidence due to being less influenced by their prior knowledge. This is an advantageous learning strategy to form accurate representations in dynamic environments, which is fundamental for successful adaptation.

Shared neural and cognitive mechanisms in action and language: The multiscale information transfer framework

This review compares how humans process action and language sequences produced by other humans. On the one hand, we identify commonalities between action and language processing in terms of cognitive mechanisms (e.g., perceptual segmentation, predictive processing, integration across multiple temporal scales), neural resources (e.g., the left inferior frontal cortex), and processing algorithms (e.g., comprehension based on changes in signal entropy). On the other hand, drawing on sign language with its particularly strong motor component, we also highlight what differentiates (both oral and signed) linguistic communication from nonlinguistic action sequences. We propose the multiscale information transfer framework (MSIT) as a way of integrating these insights and highlight directions into which future empirical research inspired by the MSIT framework might fruitfully evolve. This article is categorized under: Psychology > Language Linguistics > Language in Mind and Brain Psychology > Motor Skill and Performance Psychology > Prediction.

Motor learning of a dynamic balance task: Influence of lower limb power and prior balance practice

OBJECTIVES

We wanted to verify if the "learning to learn" effect observed in the learning of visuomotor tasks is also present when learning a balance task, i.e., whether the learning rate of a balance task is improved by prior practice of similar balance tasks.

DESIGN

Single centre, parallel group, controlled training study.

METHODS

32 young healthy participants were divided into a control and a training group. The training group's practice consisted of 90 trials of three balance tasks. Forty-eight hours after the training, we recorded performance during the acquisition (90 trials) of a novel balance task in both groups, and 24h thereafter we measured its retention (10 trials).

RESULTS

Mixed models statistical analysis showed that the learning rate of both the acquisition and the retention phase was not influenced by the 90 prior practice trials performed by the training group. However, participants with high lower limb power had a higher balance performance than participants with low power, which can be partly explained by the higher learning rate observed during the acquisition phase for participants with high power.

CONCLUSIONS

Contrary to visuomotor or perceptual tasks, we did not find a "learning to learn" effect for balance tasks. The correlation between learning rate and lower limb power suggests that motor learning of dynamic balance tasks may depend on the physical capability to execute the correct movement. Thus, a prior strength and conditioning program with emphasis on lower limb power should be considered when designing a balance training, especially in fall prevention.

Animal Learning in a Multidimensional Discrimination Task as Explained by Dimension-Specific Allocation of Attention

Reinforcement learning describes the process by which during a series of trial-and-error attempts, actions that culminate in reward are reinforced, becoming more likely to be chosen in similar circumstances. When decisions are based on sensory stimuli, an association is formed between the stimulus, the action and the reward. Computational, behavioral and neurobiological accounts of this process successfully explain simple learning of stimuli that differ in one aspect, or along a single stimulus dimension. However, when stimuli may vary across several dimensions, identifying which features are relevant for the reward is not trivial, and the underlying cognitive process is poorly understood. To study this we adapted an intra-dimensional/ extra-dimensional set-shifting paradigm to train rats on a multi-sensory discrimination task. In our setup, stimuli of different modalities (spatial, olfactory and visual) are combined into complex cues and manipulated independently. In each set, only a single stimulus dimension is relevant for reward. To distinguish between learning and decision-making we suggest a weighted attention model (WAM). Our model learns by assigning a separate learning rule for the values of features of each dimension (e.g., for each color), reinforced after every experience. Decisions are made by comparing weighted averages of the learnt values, factored by dimension specific weights. Based on the observed behavior of the rats we estimated the parameters of the WAM and demonstrated that it outperforms an alternative model, in which a learnt value is assigned to each combination of features. Estimated decision weights of the WAM reveal an experience-based bias in learning. In the first experimental set the weights associated with all dimensions were similar. The extra-dimensional shift rendered this dimension irrelevant. However, its decision weight remained high for the early learning stage in this last set, providing an explanation for the poor performance of the animals. Thus, estimated weights can be viewed as a possible way to quantify the experience-based bias.

Learning and transfer of working memory gating policies

knowledge about the tasks we encounter enables us to rapidly and flexibly adapt to novel task contexts. Previous research has focused primarily on abstract rules that leverage shared structure in stimulus-response (S-R) mappings as the basis of such task knowledge. Here we provide evidence that working memory (WM) gating policies - a type of control policy required for internal control of WM during a task - constitute a form of abstract task knowledge that can be transferred across contexts. In two experiments, we report specific evidence for the transfer of selective WM gating policies across changes of task context. We show that this transfer is not tied to shared structure in S-R mappings, but instead in the dynamic structure of the task. Collectively, our results highlight the importance of WM gating policies in particular, and control policies in general, as a key component of the task knowledge that supports flexible behavior and task generalization.

Dissociable effects of practice variability on learning motor and timing skills

Motor skill acquisition inherently depends on the way one practices the motor task. The amount of motor task variability during practice has been shown to foster transfer of the learned skill to other similar motor tasks. In addition, variability in a learning schedule, in which a task and its variations are interweaved during practice, has been shown to help the transfer of learning in motor skill acquisition. However, there is little evidence on how motor task variations and variability schedules during practice act on the acquisition of complex motor skills such as music performance, in which a performer learns both the right movements (motor skill) and the right time to perform them (timing skill). This study investigated the impact of rate (tempo) variability and the schedule of tempo change during practice on timing and motor skill acquisition. Complete novices, with no musical training, practiced a simple musical sequence on a piano keyboard at different rates. Each novice was assigned to one of four learning conditions designed to manipulate the amount of tempo variability across trials (large or small tempo set) and the schedule of tempo change (randomized or non-randomized order) during practice. At test, the novices performed the same musical sequence at a familiar tempo and at novel tempi (testing tempo transfer), as well as two novel (but related) sequences at a familiar tempo (testing spatial transfer). We found that practice conditions had little effect on learning and transfer performance of timing skill. Interestingly, practice conditions influenced motor skill learning (reduction of movement variability): lower temporal variability during practice facilitated transfer to new tempi and new sequences; non-randomized learning schedule improved transfer to new tempi and new sequences. Tempo (rate) and the sequence difficulty (spatial manipulation) affected performance variability in both timing and movement. These findings suggest that there is a dissociable effect of practice variability on learning complex skills that involve both motor and timing constraints.

Feature-based learning improves adaptability without compromising precision

Learning from reward feedback is essential for survival but can become extremely challenging with myriad choice options. Here, we propose that learning reward values of individual features can provide a heuristic for estimating reward values of choice options in dynamic, multi-dimensional environments. We hypothesize that this feature-based learning occurs not just because it can reduce dimensionality, but more importantly because it can increase adaptability without compromising precision of learning. We experimentally test this hypothesis and find that in dynamic environments, human subjects adopt feature-based learning even when this approach does not reduce dimensionality. Even in static, low-dimensional environments, subjects initially adopt feature-based learning and gradually switch to learning reward values of individual options, depending on how accurately objects’ values can be predicted by combining feature values. Our computational models reproduce these results and highlight the importance of neurons coding feature values for parallel learning of values for features and objects.

Learning about a rewarded outcome is complicated by the fact that a choice often incorporates multiple features with differing association with the reward. Here the authors demonstrate that feature-based learning is an efficient and adaptive strategy in dynamically changing environments.

Multi-day Adaptation and Savings in Manual and Locomotor Tasks

Using an individual differences approach, we evaluated whether manual and locomotor adaptation are associated in terms of adaptation and savings across days, and whether they rely on shared underlying mechanisms involving visuospatial working memory or visual field dependence. Participants performed a manual and a locomotor adaptation task during 4 separate test sessions over a 3-month period. Reliable adaptation and savings were observed for both tasks. It was further found that higher visuospatial working memory performance and lower visual field dependence scores were associated with faster learning in the manual and locomotor tasks, respectively. Moreover, adaptation rates were correlated between the 2 tasks in the final test session, suggesting that people may gradually be learning something generalizable about the adaptation process.

Structural Learning in a Visuomotor Adaptation Task Is Explicitly Accessible

Structural learning is a phenomenon characterized by faster learning in a new situation that shares features of previously experienced situations. One prominent example within the sensorimotor domain is that human participants are faster to counter a novel rotation following experience with a set of variable visuomotor rotations. This form of learning is thought to occur implicitly through the updating of an internal forward model, which predicts the sensory consequences of motor commands. However, recent work has shown that much of rotation learning occurs through an explicitly accessible process, such as movement re-aiming. We sought to determine if structural learning in a visuomotor rotation task is purely implicit (e.g., driven by an internal model) or explicitly accessible (i.e., re-aiming). We found that participants exhibited structural learning: following training with a variable set of rotations, they more quickly learned a novel rotation. This benefit was entirely conferred by the explicit re-aiming of movements. Implicit learning offered little to no contribution. Next, we investigated the specificity of this learning benefit by exposing participants to a novel perturbation drawn from a statistical structure either congruent or incongruent with their prior experience. We found that participants who experienced congruent training and test phase structure (i.e., rotations to rotation) learned more quickly than participants exposed to incongruent training and test phase structure (i.e., gains to rotation) and a control group. These results suggest that structural learning in a visuomotor rotation task is specific to previously experienced statistical structure and expressed via explicit re-aiming of movements.

The neuroscience of body memory: From the self through the space to the others

Our experience of the body is not direct; rather, it is mediated by perceptual information, influenced by internal information, and recalibrated through stored implicit and explicit body representation (body memory). This paper presents an overview of the current investigations related to body memory by bringing together recent studies from neuropsychology, neuroscience, and evolutionary and cognitive psychology. To do so, in the paper, I explore the origin of representations of human body to elucidate their developmental process and, in particular, their relationship with more explicit concepts of self. First, it is suggested that our bodily experience is constructed from early development through the continuous integration of sensory and cultural data from six different representations of the body, i.e., the Sentient Body (Minimal Selfhood), the Spatial Body (Self Location), the Active Body (Agency), the Personal Body (Whole Body Ownership - Me); the Objectified Body (Objectified Self - Mine), and the Social Body (Body Satisfaction - Ideal Me). Then, it is suggested that these six representations can be combined in a coherent supramodal representation, i.e. the "body matrix", through a predictive, multisensory processing activated by central, top-down, attentional processes. From an evolutionary perspective, the main goal of the body matrix is to allow the self to protect and extend its boundaries at both the homeostatic and psychological levels. From one perspective, the self extends its boundaries (peripersonal space) through the enactment and recognition of motor schemas. From another perspective, the body matrix, by defining the boundaries of the body, also defines where the self is present, i.e., in the body that is processed by the body matrix as the most likely to be its one, and in the space surrounding it. In the paper I also introduce and discuss the concept of "embodied medicine": the use of advanced technology for altering the body matrix with the goal of improving our health and well-being.

Learning new movements after paralysis: Results from a home-based study

Body-machine interfaces (BMIs) decode upper-body motion for operating devices, such as computers and wheelchairs. We developed a low-cost portable BMI for survivors of cervical spinal cord injury and investigated it as a means to support personalized assistance and therapy within the home environment. Depending on the specific impairment of each participant, we modified the interface gains to restore a higher level of upper body mobility. The use of the BMI over one month led to increased range of motion and force at the shoulders in chronic survivors. Concurrently, subjects learned to reorganize their body motions as they practiced the control of a computer cursor to perform different tasks and games. The BMI allowed subjects to generate any movement of the cursor with different motions of their body. Through practice subjects demonstrated a tendency to increase the similarity between the body motions used to control the cursor in distinct tasks. Nevertheless, by the end of learning, some significant and persistent differences appeared to persist. This suggests the ability of the central nervous system to concurrently learn operating the BMI while exploiting the possibility to adapt the available mobility to the specific spatio-temporal requirements of each task.

Motor Learning Enhances Use-Dependent Plasticity

Motor behaviors are shaped not only by current sensory signals but also by the history of recent experiences. For instance, repeated movements toward a particular target bias the subsequent movements toward that target direction. This process, called use-dependent plasticity (UDP), is considered a basic and goal-independent way of forming motor memories. Most studies consider movement history as the critical component that leads to UDP (Classen et al., 1998; Verstynen and Sabes, 2011). However, the effects of learning (i.e., improved performance) on UDP during movement repetition have not been investigated. Here, we used transcranial magnetic stimulation in two experiments to assess plasticity changes occurring in the primary motor cortex after individuals repeated reinforced and nonreinforced actions. The first experiment assessed whether learning a skill task modulates UDP. We found that a group that successfully learned the skill task showed greater UDP than a group that did not accumulate learning, but made comparable repeated actions. The second experiment aimed to understand the role of reinforcement learning in UDP while controlling for reward magnitude and action kinematics. We found that providing subjects with a binary reward without visual feedback of the cursor led to increased UDP effects. Subjects in the group that received comparable reward not associated with their actions maintained the previously induced UDP. Our findings illustrate how reinforcing consistent actions strengthens use-dependent memories and provide insight into operant mechanisms that modulate plastic changes in the motor cortex.SIGNIFICANCE STATEMENT Performing consistent motor actions induces use-dependent plastic changes in the motor cortex. This plasticity reflects one of the basic forms of human motor learning. Past studies assumed that this form of learning is exclusively affected by repetition of actions. However, here we showed that success-based reinforcement signals could affect the human use-dependent plasticity (UDP) process. Our results indicate that learning augments and interacts with UDP. This effect is important to the understanding of the interplay between the different forms of motor learning and suggests that reinforcement is not only important to learning new behaviors, but can shape our subsequent behavior via its interaction with UDP.

Laparoscopic Motor Learning and Workspace Exploration

BACKGROUND

Laparoscopic surgery requires operators to learn novel complex movement patterns. However, our understanding of how best to train surgeons' motor skills is inadequate, and research is needed to determine optimal laparoscopic training regimes. This difficulty is confounded by variables inherent in surgical practice, for example, the increasing prevalence of morbidly obese patients presents additional challenges related to restriction of movement because of abdominal wall resistance and reduced intra-abdominal space. The aim of this study was to assess learning of a surgery-related task in constrained and unconstrained conditions using a novel system linking a commercially available robotic arm with specialised software creating the novel kinematic assessment tool (Omni-KAT).

METHODS

We created an experimental tool that records motor performance by linking a commercially available robotic arm with specialized software that presents visual stimuli and objectively measures movement outcome (kinematics). Participants were given the task of generating aiming movements along a horizontal plane to move a visual cursor on a vertical screen. One group received training that constrained movements to the correct plane, whereas the other group was unconstrained and could explore the entire "action space."

RESULTS

The tool successfully generated the requisite force fields and precisely recorded the aiming movements. Consistent with predictions from structural learning theory, the unconstrained group produced better performance after training as indexed by movement duration (p < 0.05).

CONCLUSION

The data showed improved performance for participants who explored the entire action space, highlighting the importance of learning the full dynamics of laparoscopic instruments. These findings, alongside the development of the Omni-KAT, open up exciting prospects for better understanding of the learning processes behind surgical training and investigate ways in which learning can be optimized.

Consolidation of visuomotor adaptation memory with consistent and noisy environments

Our understanding of how we learn and retain motor behaviors is still limited. For instance, there is conflicting evidence as to whether the memory of a learned visuomotor perturbation consolidates; i.e., the motor memory becomes resistant to interference from learning a competing perturbation over time. Here, we sought to determine the factors that influence consolidation during visually guided walking. Subjects learned a novel mapping relationship, created by prism lenses, between the perceived location of two targets and the motor commands necessary to direct the feet to their positions. Subjects relearned this mapping 1 wk later. Different groups experienced protocols with or without a competing mapping (and with and without washout trials), presented either on the same day as initial learning or before relearning on day 2 We tested identical protocols under constant and noisy mapping structures. In the latter, we varied, on a trial-by-trial basis, the strength of prism lenses around a non-zero mean. We found that a novel visuomotor mapping is retained at least 1 wk after initial learning. We also found reduced foot-placement error with relearning in constant and noisy mapping groups, despite learning a competing mapping beforehand, and with the exception of one protocol, with and without washout trials. Exposure to noisy mappings led to similar performance on relearning compared with the equivalent constant mapping groups for most protocols. Overall, our results support the idea of motor memory consolidation during visually guided walking and suggest that constant and noisy practices are effective for motor learning.

NEW & NOTEWORTHY

The adaptation of movement is essential for many daily activities. To interact with targets, this often requires learning the mapping to produce appropriate motor commands based on visual input. Here, we show that a novel visuomotor mapping is retained 1 wk after initial learning in a visually guided walking task. Furthermore, we find that this motor memory consolidates (i.e., becomes more resistant to interference from learning a competing mapping) when learning in constant and noisy mapping environments.

Visuomotor Map Determines How Visually Guided Reaching Movements are Corrected Within and Across Trials

When a visually guided reaching movement is unexpectedly perturbed, it is implicitly corrected in two ways: immediately after the perturbation by feedback control (online correction) and in the next movement by adjusting feedforward motor commands (offline correction or motor adaptation). Although recent studies have revealed a close relationship between feedback and feedforward controls, the nature of this relationship is not yet fully understood. Here, we show that both implicit online and offline movement corrections utilize the same visuomotor map for feedforward movement control that transforms the spatial location of visual objects into appropriate motor commands. First, we artificially distorted the visuomotor map by applying opposite visual rotations to the cursor representing the hand position while human participants reached for two different targets. This procedure implicitly altered the visuomotor map so that changes in the movement direction to the target location were more insensitive or more sensitive. Then, we examined how such visuomotor map distortion influenced online movement correction by suddenly changing the target location. The magnitude of online movement correction was altered according to the shape of the visuomotor map. We also examined offline movement correction; the aftereffect induced by visual rotation in the previous trial was modulated according to the shape of the visuomotor map. These results highlighted the importance of the visuomotor map as a foundation for implicit motor control mechanisms and the intimate relationship between feedforward control, feedback control, and motor adaptation.

Computations underlying sensorimotor learning

The study of sensorimotor learning has a long history. With the advent of innovative techniques for studying learning at the behavioral and computational levels new insights have been gained in recent years into how the sensorimotor system acquires, retains, represents, retrieves and forgets sensorimotor tasks. In this review we highlight recent advances in the field of sensorimotor learning from a computational perspective. We focus on studies in which computational models are used to elucidate basic mechanisms underlying adaptation and skill acquisition in human behavior.

Nonspeech Oral Movements and Oral Motor Disorders: A Narrative Review

PURPOSE

Speech and other oral functions such as swallowing have been compared and contrasted with oral behaviors variously labeled quasispeech, paraspeech, speechlike, and nonspeech, all of which overlap to some degree in neural control, muscles deployed, and movements performed. Efforts to understand the relationships among these behaviors are hindered by the lack of explicit and widely accepted definitions. This review article offers definitions and taxonomies for nonspeech oral movements and for diverse speaking tasks, both overt and covert.

METHOD

Review of the literature included searches of Medline, Google Scholar, HighWire Press, and various online sources. Search terms pertained to speech, quasispeech, paraspeech, speechlike, and nonspeech oral movements. Searches also were carried out for associated terms in oral biology, craniofacial physiology, and motor control.

RESULTS AND CONCLUSIONS

Nonspeech movements have a broad spectrum of clinical applications, including developmental speech and language disorders, motor speech disorders, feeding and swallowing difficulties, obstructive sleep apnea syndrome, trismus, and tardive stereotypies. The role and benefit of nonspeech oral movements are controversial in many oral motor disorders. It is argued that the clinical value of these movements can be elucidated through careful definitions and task descriptions such as those proposed in this review article.

The intentional stance as structure learning: a computational perspective on mindreading

Recent theories of mindreading explain the recognition of action, intention, and belief of other agents in terms of generative architectures that model the causal relations between observables (e.g., observed movements) and their hidden causes (e.g., action goals and beliefs). Two kinds of probabilistic generative schemes have been proposed in cognitive science and robotics that link to a "theory theory" and "simulation theory" of mindreading, respectively. The former compares perceived actions to optimal plans derived from rationality principles and conceptual theories of others' minds. The latter reuses one's own internal (inverse and forward) models for action execution to perform a look-ahead mental simulation of perceived actions. Both theories, however, leave one question unanswered: how are the generative models - including task structure and parameters - learned in the first place? We start from Dennett's "intentional stance" proposal and characterize it within generative theories of action and intention recognition. We propose that humans use an intentional stance as a learning bias that sidesteps the (hard) structure learning problem and bootstraps the acquisition of generative models for others' actions. The intentional stance corresponds to a candidate structure in the generative scheme, which encodes a simplified belief-desire folk psychology and a hierarchical intention-to-action organization of behavior. This simple structure can be used as a proxy for the "true" generative structure of others' actions and intentions and is continuously grown and refined - via state and parameter learning - during interactions. In turn - as our computational simulations show - this can help solve mindreading problems and bootstrap the acquisition of useful causal models of both one's own and others' goal-directed actions.

Learning feedback and feedforward control in a mirror-reversed visual environment

When we learn a novel task, the motor system needs to acquire both feedforward and feedback control. Currently, little is known about how the learning of these two mechanisms relate to each other. In the present study, we tested whether feedforward and feedback control need to be learned separately, or whether they are learned as common mechanism when a new control policy is acquired. Participants were trained to reach to two lateral and one central target in an environment with mirror (left-right)-reversed visual feedback. One group was allowed to make online movement corrections, whereas the other group only received visual information after the end of the movement. Learning of feedforward control was assessed by measuring the accuracy of the initial movement direction to lateral targets. Feedback control was measured in the responses to sudden visual perturbations of the cursor when reaching to the central target. Although feedforward control improved in both groups, it was significantly better when online corrections were not allowed. In contrast, feedback control only adaptively changed in participants who received online feedback and remained unchanged in the group without online corrections. Our findings suggest that when a new control policy is acquired, feedforward and feedback control are learned separately, and that there may be a trade-off in learning between feedback and feedforward controllers.

Neural coding of uncertainty and probability

Organisms must act in the face of sensory, motor, and reward uncertainty stemming from a pandemonium of stochasticity and missing information. In many tasks, organisms can make better decisions if they have at their disposal a representation of the uncertainty associated with task-relevant variables. We formalize this problem using Bayesian decision theory and review recent behavioral and neural evidence that the brain may use knowledge of uncertainty, confidence, and probability.

Structural constraints on learning in the neural network

Recent research suggests the brain can learn almost any brain-computer interface (BCI) configuration; however, contrasting behavioral evidence from structural learning theory argues that previous experience facilitates, or impedes, future learning. A study by Sadtler and colleagues (Nature 512: 423-426, 2014) used BCI to demonstrate that neural network structural characteristics constrain learning, a finding that might also provide insight into how the brain responds to and recovers after injury.

Decoding of human hand actions to handle missing limbs in neuroprosthetics

The only way we can interact with the world is through movements, and our primary interactions are via the hands, thus any loss of hand function has immediate impact on our quality of life. However, to date it has not been systematically assessed how coordination in the hand's joints affects every day actions. This is important for two fundamental reasons. Firstly, to understand the representations and computations underlying motor control "in-the-wild" situations, and secondly to develop smarter controllers for prosthetic hands that have the same functionality as natural limbs. In this work we exploit the correlation structure of our hand and finger movements in daily-life. The novelty of our idea is that instead of averaging variability out, we take the view that the structure of variability may contain valuable information about the task being performed. We asked seven subjects to interact in 17 daily-life situations, and quantified behavior in a principled manner using CyberGlove body sensor networks that, after accurate calibration, track all major joints of the hand. Our key findings are: (1) We confirmed that hand control in daily-life tasks is very low-dimensional, with four to five dimensions being sufficient to explain 80-90% of the variability in the natural movement data. (2) We established a universally applicable measure of manipulative complexity that allowed us to measure and compare limb movements across tasks. We used Bayesian latent variable models to model the low-dimensional structure of finger joint angles in natural actions. (3) This allowed us to build a naïve classifier that within the first 1000 ms of action initiation (from a flat hand start configuration) predicted which of the 17 actions was going to be executed-enabling us to reliably predict the action intention from very short-time-scale initial data, further revealing the foreseeable nature of hand movements for control of neuroprosthetics and tele operation purposes. (4) Using the Expectation-Maximization algorithm on our latent variable model permitted us to reconstruct with high accuracy (<5-6° MAE) the movement trajectory of missing fingers by simply tracking the remaining fingers. Overall, our results suggest the hypothesis that specific hand actions are orchestrated by the brain in such a way that in the natural tasks of daily-life there is sufficient redundancy and predictability to be directly exploitable for neuroprosthetics.

Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning

Concurrent augmented feedback has been shown to be less effective for learning simple motor tasks than for complex tasks. However, as mostly artificial tasks have been investigated, transfer of results to tasks in sports and rehabilitation remains unknown. Therefore, in this study, the effect of different concurrent feedback was evaluated in trunk-arm rowing. It was then investigated whether multimodal audiovisual and visuohaptic feedback are more effective for learning than visual feedback only. Naïve subjects (N = 24) trained in three groups on a highly realistic virtual reality-based rowing simulator. In the visual feedback group, the subject's oar was superimposed to the target oar, which continuously became more transparent when the deviation between the oars decreased. Moreover, a trace of the subject's trajectory emerged if deviations exceeded a threshold. The audiovisual feedback group trained with oar movement sonification in addition to visual feedback to facilitate learning of the velocity profile. In the visuohaptic group, the oar movement was inhibited by path deviation-dependent braking forces to enhance learning of spatial aspects. All groups significantly decreased the spatial error (tendency in visual group) and velocity error from baseline to the retention tests. Audiovisual feedback fostered learning of the velocity profile significantly more than visuohaptic feedback. The study revealed that well-designed concurrent feedback fosters complex task learning, especially if the advantages of different modalities are exploited. Further studies should analyze the impact of within-feedback design parameters and the transferability of the results to other tasks in sports and rehabilitation.

Neural constraints on learning

Motor, sensory, and cognitive learning require networks of neurons to generate new activity patterns. Because some behaviors are easier to learn than others^1,2, we wondered if some neural activity patterns are easier to generate than others. We asked whether the existing network constrains the patterns that a subset of its neurons is capable of exhibiting, and if so, what principles define the constraint. We employed a closed-loop intracortical brain-computer interface (BCI) learning paradigm in which Rhesus monkeys controlled a computer cursor by modulating neural activity patterns in primary motor cortex. Using the BCI paradigm, we could specify and alter how neural activity mapped to cursor velocity. At the start of each session, we observed the characteristic activity patterns of the recorded neural population. These patterns comprise a low-dimensional space (termed the intrinsic manifold, or IM) within the high-dimensional neural firing rate space. They presumably reflect constraints imposed by the underlying neural circuitry. We found that the animals could readily learn to proficiently control the cursor using neural activity patterns that were within the IM. However, animals were less able to learn to proficiently control the cursor using activity patterns that were outside of the IM. This result suggests that the existing structure of a network can shape learning. On the timescale of hours, it appears to be difficult to learn to generate neural activity patterns that are not consistent with the existing network structure. These findings offer a network-level explanation for the observation that we are more readily able to learn new skills when they are related to the skills that we already possess^3,4.

Minimally invasive surgery training using multiple port sites to improve performance

BACKGROUND

Structural learning theory suggests that experiencing motor task variation enables the central nervous system to extract general rules regarding tasks with a similar structure - rules that can subsequently be applied to novel situations. Complex minimally invasive surgery (MIS) requires different port sites, but switching ports alters the limb movements required to produce the same endpoint control of the surgical instrument. The purpose of the present study was to determine if structural learning theory can be applied to MIS to inform training methods.

METHODS

A tablet laptop running bespoke software was placed within a laparoscopic box trainer and connected to a monitor situated at eye level. Participants (right-handed, non-surgeons, mean age = 23.2 years) used a standard laparoscopic grasper to move between locations on the screen. There were two training groups: the M group (n = 10) who trained using multiple port sites, and the S group (n = 10) who trained using a single port site. A novel port site was used as a test of generalization. Performance metrics were a composite of speed and accuracy (SACF) and normalized jerk (NJ; a measure of movement 'smoothness').

RESULTS

The M group showed a statistically significant performance advantage over the S group at test, as indexed by improved SACF (p < 0.05) and NJ (p < 0.05).

CONCLUSIONS

This study has demonstrated the potential benefits of incorporating a structural learning approach within MIS training. This may have practical applications when training junior surgeons and developing surgical simulation devices.

Individual variability in visual discrimination and reversal learning performance in common marmosets

Detailed information about the characteristics of learning behavior in marmosets is useful for future marmoset research. We trained 42 marmosets in visual discrimination and reversal learning. All marmosets could learn visual discrimination, and all but one could complete reversal learning, though some marmosets failed to touch the visual stimuli and were screened out. In 87% of measurements, the final percentage of correct responses was over 95%. We quantified performance with two measures: onset trial and dynamic interval. Onset trial represents the number of trials that elapsed before the marmoset started to learn. Dynamic interval represents the number of trials from the start before reaching the final percentage of correct responses. Both measures decreased drastically as a result of the formation of discrimination learning sets. In reversal learning, both measures worsened, but the effect on onset trial was far greater. The effects of age and sex were not significant as far as we used adolescent or young adult marmosets. Unexpectedly, experimental circumstance (in the colony or isolator) had only a subtle effect on performance. However, we found that marmosets from different families exhibited different learning process characteristics, suggesting some family effect on learning.

Structure learning and the Occam's razor principle: a new view of human function acquisition

We often encounter pairs of variables in the world whose mutual relationship can be described by a function. After training, human responses closely correspond to these functional relationships. Here we study how humans predict unobserved segments of a function that they have been trained on and we compare how human predictions differ to those made by various function-learning models in the literature. Participants' performance was best predicted by the polynomial functions that generated the observations. Further, participants were able to explicitly report the correct generating function in most cases upon a post-experiment survey. This suggests that humans can abstract functions. To understand how they do so, we modeled human learning using an hierarchical Bayesian framework organized at two levels of abstraction: function learning and parameter learning, and used it to understand the time course of participants' learning as we surreptitiously changed the generating function over time. This Bayesian model selection framework allowed us to analyze the time course of function learning and parameter learning in relative isolation. We found that participants acquired new functions as they changed and even when parameter learning was not completely accurate, the probability that the correct function was learned remained high. Most importantly, we found that humans selected the simplest-fitting function with the highest probability and that they acquired simpler functions faster than more complex ones. Both aspects of this behavior, extent and rate of selection, present evidence that human function learning obeys the Occam's razor principle.