Motor planning under uncertainty

Ouvrai opens access to remote virtual reality studies of human behavioural neuroscience

Modern virtual reality (VR) devices record six-degree-of-freedom kinematic data with high spatial and temporal resolution and display high-resolution stereoscopic three-dimensional graphics. These capabilities make VR a powerful tool for many types of behavioural research, including studies of sensorimotor, perceptual and cognitive functions. Here we introduce Ouvrai, an open-source solution that facilitates the design and execution of remote VR studies, capitalizing on the surge in VR headset ownership. This tool allows researchers to develop sophisticated experiments using cutting-edge web technologies such as WebXR to enable browser-based VR, without compromising on experimental design. Ouvrai's features include easy installation, intuitive JavaScript templates, a component library managing front- and backend processes and a streamlined workflow. It integrates with Firebase, Prolific and Amazon Mechanical Turk and provides data processing utilities for analysis. Unlike other tools, Ouvrai remains free, with researchers managing their web hosting and cloud database via personal Firebase accounts. Ouvrai is not limited to VR studies; researchers can also develop and run desktop or touchscreen studies using the same streamlined workflow. Through three distinct motor learning experiments, we confirm Ouvrai's efficiency and viability for conducting remote VR studies.

Subtle Visual Latency Can Profoundly Impair Implicit Sensorimotor Learning

Short sub-100ms visual feedback latencies are common in many types of human-computer interactions yet are known to markedly reduce performance in a wide variety of motor tasks from simple pointing to operating surgical robotics. These latencies are also present in the computer-based experiments used to study the sensorimotor learning that underlies the acquisition of motor performance. Inspired by neurophysiological findings showing that cerebellar LTD and cortical LTP would both be disrupted by sub-100ms latencies, we hypothesized that implicit sensorimotor learning may be particularly sensitive to these short latencies. Remarkably, we find that improving latency by just 60ms, from 85 to 25ms in latency-optimized experiments, increases implicit learning by 50% and proportionally decreases explicit learning, resulting in a dramatic reorganization of sensorimotor memory. We go on to show that implicit sensorimotor learning is considerably more sensitive to latencies in the sub-100ms range than at higher latencies, in line with the latency-specific neural plasticity that has been observed. This suggests a clear benefit for latency reduction in computer-based training that involves implicit sensorimotor learning and that across-study differences in implicit motor learning might often be explained by disparities in feedback latency.

Behavioral Motor Performance

The human sensorimotor control system has exceptional abilities to perform skillful actions. We easily switch between strenuous tasks that involve brute force, such as lifting a heavy sewing machine, and delicate movements such as threading a needle in the same machine. Using a structure with different control architectures, the motor system is capable of updating its ability to perform through our daily interaction with the fluctuating environment. However, there are issues that make this a difficult computational problem for the brain to solve. The brain needs to control a nonlinear, nonstationary neuromuscular system, with redundant and occasionally undesired degrees of freedom, in an uncertain environment using a body in which information transmission is subject to delays and noise. To gain insight into the mechanisms of motor control, here we survey movement laws and invariances that shape our everyday motion. We then examine the major solutions to each of these problems in the three parts of the sensorimotor control system, sensing, planning, and acting. We focus on how the sensory system, the control architectures, and the structure and operation of the muscles serve as complementary mechanisms to overcome deviations and disturbances to motor behavior and give rise to skillful motor performance. We conclude with possible future research directions based on suggested links between the operation of the sensorimotor system across the movement stages. © 2024 American Physiological Society. Compr Physiol 14:5179-5224, 2024.

Continuous evaluation of cost-to-go for flexible reaching control and online decisions

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Muscular reflex gains reflect changes of mind in reaching

Decisions for action are accompanied by a continual processing of sensory information, sometimes resulting in a revision of the initial choice, called a change of mind (CoM). Although the motor system is tuned during the formation of a reach decision, it is unclear whether its preparatory state differs between CoM and non-CoM decisions. To test this, participants (n = 14) viewed a random-dot motion (RDM) stimulus of various coherence levels for a random viewing duration. At the onset of a mechanical perturbation that rapidly stretched the pectoralis muscle, they indicated the perceived motion direction by making a reaching movement to one of two targets. Using electromyography (EMG), we quantified the reflex gains of the pectoralis and posterior deltoid muscles. Results show that reflex gains scaled with both the coherence level and the viewing duration of the stimulus. We fit a drift diffusion model (DDM) to the behavioral choices. The decision variable (DV), derived from the DDM, correlated well with the measured reflex gain at the single-trial level. However, when matched on DV magnitude, reflex gains were significantly lower in CoM than non-CoM trials. We conclude that the internal state of the motor system, as measured by the spinal reflexes, reflects the continual deliberation on sensory evidence for action selection, including the postdecisional evidence that can lead to a change of mind.NEW & NOTEWORTHY Using behavioral findings, EMG, and computational modeling, we show that not only the perceptual decision outcome but also the accumulating evidence for that outcome is continuously sent to the relevant muscles. Moreover, we show that an upcoming change of mind can be detected in the motor periphery, suggesting that a correlate of the internal decision making process is being sent along.

Single-Trial Dynamics of Competing Reach Plans in the Human Motor Periphery

Contemporary motor control theories propose competition between multiple motor plans before the winning command is executed. While most competitions are completed before movement onset, movements are often initiated before the competition has been resolved. An example of this is saccadic averaging, wherein the eyes land at an intermediate location between two visual targets. Behavioral and neurophysiological signatures of competing motor commands have also been reported for reaching movements, but debate remains about whether such signatures attest to an unresolved competition, arise from averaging across many trials, or reflect a strategy to optimize behavior given task constraints. Here, we recorded EMG activity from an upper limb muscle (m. pectoralis) while 12 (8 female) participants performed an immediate response reach task, freely choosing between one of two identical and suddenly presented visual targets. On each trial, muscle recruitment showed two distinct phases of directionally tuned activity. In the first wave, time-locked ∼100 ms of target presentation, muscle activity was clearly influenced by the nonchosen target, reflecting a competition between reach commands that was biased in favor of the ultimately chosen target. This resulted in an initial movement intermediate between the two targets. In contrast, the second wave, time-locked to voluntary reach onset, was not biased toward the nonchosen target, showing that the competition between targets was resolved. Instead, this wave of activity compensated for the averaging induced by the first wave. Thus, single-trial analysis reveals an evolution in how the nonchosen target differentially influences the first and second wave of muscle activity.SIGNIFICANCE STATEMENT Contemporary theories of motor control suggest that multiple motor plans compete for selection before the winning command is executed. Evidence for this is found in intermediate reach movements toward two potential target locations, but recent findings have challenged this notion by arguing that intermediate reaching movements reflect an optimal response strategy. By examining upper limb muscle recruitment during a free-choice reach task, we show early recruitment of a suboptimal averaged motor command to the two targets that subsequently transitions to a single motor command that compensates for the initially averaged motor command. Recording limb muscle activity permits single-trial resolution of the dynamic influence of the nonchosen target through time.

Parallel movement planning is achieved via an optimal preparatory state in motor cortex

How do patterns of neural activity in the motor cortex contribute to the planning of a movement? A recent theory developed for single movements proposes that the motor cortex acts as a dynamical system whose initial state is optimized during the preparatory phase of the movement. This theory makes important yet untested predictions about preparatory dynamics in more complex behavioral settings. Here, we analyze preparatory activity in non-human primates planning not one but two movements simultaneously. As predicted by the theory, we find that parallel planning is achieved by adjusting preparatory activity within an optimal subspace to an intermediate state reflecting a trade-off between the two movements. The theory quantitatively accounts for the relationship between this intermediate state and fluctuations in the animals' behavior down at the trial level. These results uncover a simple mechanism for planning multiple movements in parallel and further point to motor planning as a controlled dynamical process.

The role of consciously timed movements in shaping and improving auditory timing

Our subjective sense of time is intertwined with a plethora of perceptual, cognitive and motor functions, and likewise, the brain is equipped to expertly filter, weight and combine these signals for seamless interactions with a dynamic world. Until relatively recently, the literature on time perception has excluded the influence of simultaneous motor activity, yet it has been found that motor circuits in the brain are at the core of most timing functions. Several studies have now identified that concurrent movements exert robust effects on perceptual timing estimates, but critically have not assessed how humans consciously judge the duration of their own movements. This creates a gap in our understanding of the mechanisms driving movement-related effects on sensory timing. We sought to address this gap by administering a sensorimotor timing task in which we explicitly compared the timing of isolated auditory tones and arm movements, or both simultaneously. We contextualized our findings within a Bayesian cue combination framework, in which separate sources of temporal information are weighted by their reliability and integrated into a unitary time estimate that is more precise than either unisensory estimate. Our results revealed differences in accuracy between auditory, movement and combined trials, and (crucially) that combined trials were the most accurately timed. Under the Bayesian framework, we found that participants' combined estimates were more precise than isolated estimates, yet were sub-optimal when compared with the model's prediction, on average. These findings elucidate previously unknown qualities of conscious motor timing and propose computational mechanisms that can describe how movements combine with perceptual signals to create unified, multimodal experiences of time.

Pain, from perception to action: A computational perspective

Pain is driven by sensation and emotion, and in turn, it motivates decisions and actions. To fully appreciate the multidimensional nature of pain, we formulate the study of pain within a closed-loop framework of sensory-motor prediction. In this closed-loop cycle, prediction plays an important role, as the interaction between prediction and actual sensory experience shapes pain perception and subsequently, action. In this Perspective, we describe the roles of two prominent computational theories-Bayesian inference and reinforcement learning-in modeling adaptive pain behaviors. We show that prediction serves as a common theme between these two theories, and that each of these theories can explain unique aspects of the pain perception-action cycle. We discuss how these computational theories and models can improve our mechanistic understandings of pain-centered processes such as anticipation, attention, placebo hypoalgesia, and pain chronification.

A neurocomputational theory of action regulation predicts motor behavior in neurotypical individuals and patients with Parkinson’s disease

Surviving in an uncertain environment requires not only the ability to select the best action, but also the flexibility to withhold inappropriate actions when the environmental conditions change. Although selecting and withholding actions have been extensively studied in both human and animals, there is still lack of consensus on the mechanism underlying these action regulation functions, and more importantly, how they inter-relate. A critical gap impeding progress is the lack of a computational theory that will integrate the mechanisms of action regulation into a unified framework. The current study aims to advance our understanding by developing a neurodynamical computational theory that models the mechanism of action regulation that involves suppressing responses, and predicts how disruption of this mechanism can lead to motor deficits in Parkinson’s disease (PD) patients. We tested the model predictions in neurotypical individuals and PD patients in three behavioral tasks that involve free action selection between two opposed directions, action selection in the presence of conflicting information and abandoning an ongoing action when a stop signal is presented. Our results and theory suggest an integrated mechanism of action regulation that affects both action initiation and inhibition. When this mechanism is disrupted, motor behavior is affected, leading to longer reaction times and higher error rates in action inhibition.

Anticipatory action planning for stepping onto competing potential targets

The brain plans an anticipatory action for performing tasks successfully and effortlessly even if there are multiple possible options. There is increasing evidence that, when multiple actions are possible, the brain considers two factors when planning an anticipatory action-the probabilistic value and the action cost for each potential action. When the action involves maintaining upright balance, such as standing, stepping, or walking, the action cost for maintaining postural stability could be considered dominantly. We addressed this issue by using a "go-before-you-know" task to step onto a target on the floor. In this task, two potential targets were located on the medial or lateral side of the stepping foot, and the true target was cued only after participants shifted their loads to leave that foot. Participants initiated their stepping actions without knowing which of the potential targets would be the true one. The results showed that, for the majority of participants, lateral displacements of the center of pressure (COP) with two potential targets were similar to those when a single target exists on the individual's medial side. Given that mediolateral postural stability became more destabilized with stepping onto the medial target than stepping onto the lateral target, they were likely to plan their mediolateral components of the postural adjustments for the worst-case scenario (i.e., falling). Additionally, posterior COP movements with two potential targets became smaller than those with a single target, suggesting an effort to create extra time to determine the true target and to adjust the swing foot. Based on these findings, we concluded that action costs for maintaining postural stability were considered dominantly for planning an anticipatory action to accomplish a stepping task successfully while ensuring upright balance.

The biased hand. Mouse-tracking metrics to examine the conflict processing in a race-implicit association test

In this study, we adapted a race-Implicit Association Test (race-IAT) to mouse-tracking (MT) technique to identify the more representative target observed MT-metrics and explore the temporal unfolding of the cognitive conflict emerging during the categorisation task. Participants of Western European descent performed a standard keyboard-response race-IAT (RT-race-IAT) and an MT-race-IAT with the same structure. From a behavioural point of view, our sample showed a typical Congruency Effect, thus a pro-White implicit bias, in the RT-race-IAT. In addition, in the MT-race-IAT, the MT-metrics showed a similar Congruency Effect mirroring the higher attraction of the averaged-trajectories towards the incorrect response button in incongruent than congruent trials. Moreover, these MT-metrics were positively associated with RT-race-IAT scores, strengthening the MT approach’s validity in characterising the implicit bias. Furthermore, the distributional analyses showed that mouse trajectories displayed a smooth profile both in congruent and incongruent trials to indicate that the unfolding of the decision process and the raised conflict is guided by dynamical cognitive processing. This latter continuous competition process was studied using a novel phase-based approach which allowed to temporally dissect an Early, a Mid and a Late phase, each of which may differently reflect the decision conflict between automatic and controlled responses in the evolution of the mouse movement towards the target response. Our results show that the MT approach provides an accurate and finer-grained characterisation of the implicit racial attitude than classical RT-IAT. Finally, our novel phase-based approach can be an effective tool to shed light on the implicit conflict processing emerging in a categorisation task with a promising transferable value in different cognitive and neuropsychological fields.

Symbolic cues enhance express visuomotor responses in human arm muscles at the motor planning rather than the visuospatial processing stage

Humans can produce "express" (~100ms) arm muscle responses that are inflexibly locked in time and space to visual target presentations, consistent with subcortical visuomotor transformations via the tecto-reticulo-spinal pathway. These express visuomotor responses are sensitive to explicit cue-driven expectations, but it is unclear at what stage of sensory-to-motor transformation such modulation occurs. Here, we recorded electromyographic activity from shoulder muscles as participants reached toward one of four virtual targets whose physical location was partially predictable from a symbolic cue. In an experiment in which targets could be veridically reached, express responses were inclusive of the biomechanical requirements for reaching the cued locations and not systematically modulated by cue validity. In a second experiment, movements were restricted to the horizontal plane so that the participants could perform only rightward or leftward reaches, irrespective of target position on the vertical axis. Express muscle responses were almost identical for targets that were validly cued in the horizontal direction, regardless of cue validity in the vertical dimension. Together, these findings suggest that the cue-induced enhancements of express responses are dominated by effects at the level of motor plans and not solely via facilitation of early visuospatial target processing. Notably, direct cortico-tectal and cortico-reticular projections exist that are well-placed to modulate pre-stimulus motor preparation state in subcortical circuits. Our results could reflect a neural mechanism by which contextually relevant motor responses to compatible visual inputs are rapidly released via subcortical circuits that are sufficiently along the sensory- to-motor continuum.

Motor Plans under Uncertainty Reflect a Trade-Off between Maximizing Reward and Success

When faced with multiple potential movement options, individuals either reach directly to one of the options, or initiate a reach intermediate between the options. It remains unclear why people generate these two types of behaviors. Using the go-before-you-know task (commonly used to study behavior under choice uncertainty) in humans, we examined two key questions. First, do these two types of responses actually reflect distinct movement strategies? If so, the relative desirability (i.e., weighing the success likelihood vs the attainable reward) of the two target options would not need to be computed identically for direct and intermediate reaches. We showed that indeed, when reward and success likelihood differed between the two options, reach direction was preferentially biased toward different directions for direct versus intermediate reaches. Importantly, this suggests that the computation of subjective values depends on the choice of movement strategy. Second, what drives individual differences in how people respond under uncertainty? We found that risk/reward-seeking individuals tended to generate more intermediate reaches and were more responsive to changes in reward, suggesting these movements may reflect a strategy to maximize reward versus success. Together, these findings suggest that when faced with choice uncertainty, individuals adopt movement strategies consistent with their risk/reward attitude, preferentially biasing behavior toward exogenous rewards or endogenous success and consequently modulating the relative desirability of the available options.