Temporal Evolution of Spatial Computations for Visuomotor Control

Modulations of stretch reflex by altering visuomotor contexts

Various functional modulations of the stretch reflex help to stabilize actions, but the computational mechanism behind its context-dependent tuning remains unclear. While many studies have demonstrated that motor contexts associated with the task goal cause functional modulation of the stretch reflex of upper limbs, it is not well understood how visual contexts independent of the task requirements affect the stretch reflex. To explore this issue, we conducted two experiments testing 20 healthy human participants (age range 20-45, average 31.3 ± 9.0), in which visual contexts were manipulated in a visually guided reaching task. During wrist flexion movements toward a visual target, a mechanical load was applied to the wrist joint to evoke stretch reflex of wrist flexor muscle (flexor carpi radialis). The first experiment (n = 10) examined the effect of altering the visuomotor transformation on the stretch reflex that was evaluated with surface electromyogram. We found that the amplitude of the stretch reflex decreased (p = 0.024) when a rotational transformation of 90° was introduced between the hand movement and the visual cursor, whereas the amplitude did not significantly change (p = 0.26) when the rotational transformation was accompanied by a head rotation so that the configuration of visual feedback was maintained in visual coordinates. The results suggest that the stretch reflex was regulated depending on whether the visuomotor mapping had already been acquired or not. In the second experiment (n = 10), we examined how uncertainty in the visual target or hand cursor affects the stretch reflex by removing these visual stimuli. We found that the reflex amplitude was reduced by the disappearance of the hand cursor (p = 0.039), but was not affected by removal of the visual target (p = 0.27), suggesting that the visual state of the body and target contribute differently to the reflex tuning. These findings support the idea that visual updating of the body state is crucial for regulation of quick motor control driven by proprioceptive signals.

Rapid Online Corrections for Proprioceptive and Visual Perturbations Recruit Similar Circuits in Primary Motor Cortex

An important aspect of motor function is our ability to rapidly generate goal-directed corrections for disturbances to the limb or behavioral goal. The primary motor cortex (M1) is a key region involved in processing feedback for rapid motor corrections, yet we know little about how M1 circuits are recruited by different sources of sensory feedback to make rapid corrections. We trained two male monkeys (Macaca mulatta) to make goal-directed reaches and on random trials introduced different sensory errors by either jumping the visual location of the goal (goal jump), jumping the visual location of the hand (cursor jump), or applying a mechanical load to displace the hand (proprioceptive feedback). Sensory perturbations evoked a broad response in M1 with ∼73% of neurons (n = 257) responding to at least one of the sensory perturbations. Feedback responses were also similar as response ranges between the goal and cursor jumps were highly correlated (range of r = [0.91, 0.97]) as were the response ranges between the mechanical loads and the visual perturbations (range of r = [0.68, 0.86]). Lastly, we identified the neural subspace each perturbation response resided in and found a strong overlap between the two visual perturbations (range of overlap index, 0.73-0.89) and between the mechanical loads and visual perturbations (range of overlap index, 0.36-0.47) indicating each perturbation evoked similar structure of activity at the population level. Collectively, our results indicate rapid responses to errors from different sensory sources target similar overlapping circuits in M1.

Reach Corrections Toward Moving Objects are Faster Than Reach Corrections Toward Instantaneously Switching Targets

Visually guided reaching is a common motor behavior that engages subcortical circuits to mediate rapid corrections. Although these neural mechanisms have evolved for interacting with the physical world, they are often studied in the context of reaching toward virtual targets on a screen. These targets often change position by disappearing from one place reappearing in another instantaneously. In this study, we instructed participants to perform rapid reaches to physical objects that changed position in different ways. In one condition, the objects moved very rapidly from one place to another. In the other condition, illuminated targets instantaneously switched position by being extinguished in one position and illuminating in another. Participants were consistently faster in correcting their reach trajectories when the object moved continuously.

Continuous use of visual information about the position of the moving hand

People generally look at a target when they want to reach for it. Doing so presumably helps them continuously update their judgments about the target's position and motion. But not looking at their hand does not prevent people from updating judgments about its position on the basis of visual information, because people do respond to experimental perturbations of visual information about the position of their hand. Here, we study such responses by adding jitter to the movement of a cursor that follows participants' fingers. We analyse the response to the jitter in a way that reveals how the vigour of the response depends on the moment during the movement at which the change in cursor position occurs. We compare the change in vigour to that for equivalent jitter in the position of the target. We find that participants respond to jitter in the position of a cursor in much the same way as they respond to jitter in the target's position. The responses are more vigorous late in the movement, when adjustments need to be made within less time, but similarly so for the cursor as for the target. The responses are weaker for the cursor, presumably because of the jitter-free kinaesthetic information about the position of the finger.

When intercepting moving targets, mid-movement error corrections reflect distinct responses to visual and haptic perturbations

We examined a key aspect of sensorimotor skill: the capability to correct performance errors that arise mid-movement. Participants grasped the handle of a robot that imposed a nominal viscous resistance to hand movement. They watched a target move pseudo-randomly just above the horizontal plane of hand motion and initiated quick interception movements when cued. On some trials, the robot's viscosity or the target's speed changed without warning coincident with the GO cue. We fit a sum-of-Gaussians model to mechanical power measured at the handle to determine the number, magnitude, and relative timing of submovements occurring in each interception attempt. When a single submovement successfully intercepted the target, capture times averaged 410 ms. Sometimes, two or more submovements were required. Initial error corrections typically occurred before feedback could indicate the target had been captured or missed. Error corrections occurred sooner after movement onset in response to mechanical viscosity increases (at 154 ms) than to unprovoked errors on control trials (215 ms). Corrections occurred later (272 ms) in response to viscosity decreases. The latency of corrections for target speed changes did not differ from those in control trials. Remarkably, these early error corrections accommodated the altered testing conditions; speed/viscosity increases elicited more vigorous corrections than in control trials with unprovoked errors; speed/viscosity decreases elicited less vigorous corrections. These results suggest that the brain monitors and predicts the outcome of evolving movements, rapidly infers causes of mid-movement errors, and plans and executes corrections-all within 300 ms of movement onset.

Investigating the online control of goal-directed actions to a tactile target on the body

Movement corrections to somatosensory targets have been found to be shorter in latency and larger in magnitude than corrections to external visual targets. Somatosensory targets (e.g., body positions) can be identified using both tactile (i.e., skin receptors) and proprioceptive information (e.g., the sense of body position derived from sensory organs in the muscles and joints). Here, we investigated whether changes in tactile information alone, without changes in proprioception, can elicit shorter correction latencies and larger correction magnitudes than those to external visual targets. Participants made reaching movements to a myofilament touching the index finger of the non-reaching finger (i.e., a tactile target) and a light-emitting diode (i.e., visual target). In one-third of the trials, target perturbations occurred 100 ms after movement onset, such that the target was displaced 3 cm either away or toward the participant. We found that participants demonstrated larger correction magnitudes to visual than tactile target perturbations. Moreover, we found no differences in correction latency between movements to perturbed tactile and visual targets. Further, we found that while participants detected tactile stimuli earlier than visual stimuli, they took longer to initiate reaching movements to an unperturbed tactile target than an unperturbed visual target. These results provide evidence that additional processes may be required when planning movements to tactile versus visual targets and that corrections to changes in tactile target positions alone may not facilitate the latency and magnitude advantages observed for corrections to somatosensory targets (i.e., proprioceptive-tactile targets).

Contextual cues are not unique for motor learning: Task-dependant switching of feedback controllers

The separation of distinct motor memories by contextual cues is a well known and well studied phenomenon of feedforward human motor control. However, there is no clear evidence of such context-induced separation in feedback control. Here we test both experimentally and computationally if context-dependent switching of feedback controllers is possible in the human motor system. Specifically, we probe visuomotor feedback responses of our human participants in two different tasks—stop and hit—and under two different schedules. The first, blocked schedule, is used to measure the behaviour of stop and hit controllers in isolation, showing that it can only be described by two independent controllers with two different sets of control gains. The second, mixed schedule, is then used to compare how such behaviour evolves when participants regularly switch from one task to the other. Our results support our hypothesis that there is contextual switching of feedback controllers, further extending the accumulating evidence of shared features between feedforward and feedback control.

Extensive evidence has demonstrated that humans can learn distinct motor memories (i.e. independent feedforward controllers) using contextual cues. However, there is little evidence that such contextual cues produce similar separation of feedback controllers. As accumulating evidence highlights the connection between feedforward and feedback control, we propose that context may be used to separate feedback controllers as well. It has not been trivial to test experimentally whether a change in context also modulates the feedback control, as the controller output is affected by other non-contextual factors such as movement kinematics, time-to-target or the properties of the perturbation used to probe the control. Here we present a computational approach based on normative modelling where we separate the effects of the context from other non-contextual effects on the visuomotor feedback system. We then show experimentally that task context independently modulates the feedback control in a particular manner that can be reliably predicted using optimal feedback control.

The Effect of Target Velocity on the Fast Corrective Response during Reaching Movement

Online motor control is often required to correct errors in rapid adjustments during reaching movements. It has been established that the initial arm trajectory during reaching is corrected by a target displacement. Since this corrective response occurs without perception of target perturbation, this is regarded as an automatic response. However, an object rarely "jumps" in daily life, rather it often "moves" as a chronological change of the position that causes visual motion. Therefore, the purpose of this study was to investigate whether the implicit visuomotor response is induced by target motion stimuli and to clarify the effects of target motion velocity on initial arm trajectory. Participants were asked to move a cursor from a start circle to a visual target. The target moved either leftward or rightward when the cursor passed 20 mm from the start circle. Four target velocities (10, 20, 30, 40 deg/s) were randomly presented. Our results showed that the initial velocity (first 50 ms) of the fast corrective response increased with the target velocity. Therefore, it is indicated that the fast corrective response is induced by the target motion stimulus with a short latency and its amplitude is dependent on the target velocity.

Eye-hand coordination during online reach corrections is task dependent

To produce accurate movements, the human motor system needs to deal with errors that can occur due to inherent noise, changes in the body, or disturbances in the environment. Here, we investigated the temporal coupling of rapid corrections of the eye and hand in response to a change in visual target location during the movement. In addition to a "classic" double-step task in which the target stepped to a new position, participants performed a set of modified double-step tasks in which the change in movement goal was indicated by the appearance of an additional target, or by a spatial or symbolic cue. We found that both the absolute correction latencies of the eye and hand and the relative eye-hand correction latencies were dependent on the visual characteristics of the target change, with increasingly longer latencies in tasks that required more visual and cognitive processing. Typically, the hand started correcting slightly earlier than the eye, especially when the target change was indicated by a symbolic cue, and in conditions where visual feedback of the hand position was provided during the reach. Our results indicate that the oculomotor and limb-motor system can be differentially influenced by processing requirements of the task and emphasize that temporal eye-hand coupling is flexible rather than rigid.NEW & NOTEWORTHY Eye movements support hand movements in many situations. Here, we used variations of a double-step task to investigate temporal coupling of corrective hand and eye movements in response to target displacements. Correction latency coupling depended on the visual and cognitive processing demands of the task. The hand started correcting before the eye, especially when the task required decoding a symbolic cue. These findings highlight the flexibility and task dependency of eye-hand coordination.

Virtual stick balancing: skill development in Newtonian and Aristotelian dynamics

Human reaction delay significantly limits manual control of unstable systems. It is more difficult to balance a short stick on a fingertip than a long one, because a shorter stick falls faster and therefore requires faster reactions. In this study, a virtual stick balancing environment was developed where the reaction delay can be artificially modulated and the law of motion can be changed between second-order (Newtonian) and first-order (Aristotelian) dynamics. Twenty-four subjects were separated into two groups and asked to perform virtual stick balancing programmed according to either Newtonian or Aristotelian dynamics. The shortest stick length (critical length, L_c) was determined for different added delays in six sessions of balancing trials performed on different days. The observed relation between L_c and the overall reaction delay τ reflected the feature of the underlying mathematical models: (i) for the Newtonian dynamics L_c is proportional to τ²; (ii) for the Aristotelian dynamics L_c is proportional to τ. Deviation of the measured L_c(τ) function from the theoretical one was larger for the Newtonian dynamics for all sessions, which suggests that, at least in virtually controlled tasks, it is more difficult to adopt second-order dynamics than first-order dynamics.

Can ongoing movements be guided by allocentric visual information when the target is visible?

People use both egocentric (object-to-self) and allocentric (object-to-object) spatial information to interact with the world. Evidence for allocentric information guiding ongoing actions stems from studies in which people reached to where targets had previously been seen while other objects were moved. Since egocentric position judgments might fade or change when the target is removed, we sought for conditions in which people might benefit from relying on allocentric information when the target remains visible. We used a task that required participants to intercept targets that moved across a screen using a cursor that represented their finger but that moved by a different amount in a different plane. During each attempt, we perturbed the target, cursor, or background individually or all three simultaneously such that their relative positions did not change and there was no need to adjust the ongoing movement. An obvious way to avoid responding to such simultaneous perturbations is by relying on allocentric information. Relying on egocentric information would give a response that resembles the combined responses to the three isolated perturbations. The hand responded in accordance with the responses to the isolated perturbations despite the differences between how the finger and cursor moved. This response remained when the simultaneous perturbation was repeated many times, suggesting that participants hardly relied upon allocentric spatial information to control their ongoing visually guided actions.

Reaching movements are automatically redirected to nearby options during target split

Motor behavior often occurs in environments with multiple goal options that can vary during the ongoing action. We explored this situation by requiring subjects to select between different target options during an ongoing reach. During split trials the original target was replaced with a left and a right flanking target, and participants had to select between them. This contrasted with the standard jump trials, where the original target would be replaced with a single flanking target, left or right. When participants were instructed to follow their natural tendency, they all tended to select the split target nearest the original. The near-target preference was more prominent with increased spatial disparity between the options and when participants could preview the potential options. Moreover, explicit instruction to obtain the "far" target during split trials resulted many errors compared with a "near" instruction, ~50% vs. ~15%. Online reaction times to target change were delayed in split trials compared with jump trials, ~200 ms vs. ~150 ms, but also highly automatic. Trials in which the instructed far target was correctly obtained were delayed by a further ~50 ms, unlike those in which the near target was incorrectly obtained. We also observed nonspecific responses from arm muscles at the jump trial latency during split trials. Taken together, our results indicate that online selection of reach targets is automatically linked to the spatial distribution of the options, though at greater delays than redirecting to a single target.NEW & NOTEWORTHY This work demonstrates that target selection during an ongoing reach is automatically linked to the option nearest a voided target. Online reaction times for two options are longer than redirection to a single option. Attempts to override the near-target tendency result in a high number of errors at the normal delay and further delays when the attempt is successful.

The Psychology of Reaching: Action Selection, Movement Implementation, and Sensorimotor Learning

The study of motor planning and learning in humans has undergone a dramatic transformation in the 20 years since this journal's last review of this topic. The behavioral analysis of movement, the foundational approach for psychology, has been complemented by ideas from control theory, computer science, statistics, and, most notably, neuroscience. The result of this interdisciplinary approach has been a focus on the computational level of analysis, leading to the development of mechanistic models at the psychological level to explain how humans plan, execute, and consolidate skilled reaching movements. This review emphasizes new perspectives on action selection and motor planning, research that stands in contrast to the previously dominant representation-based perspective of motor programming, as well as an emerging literature highlighting the convergent operation of multiple processes in sensorimotor learning.

Spatiotemporal transformations for gaze control

Sensorimotor transformations require spatiotemporal coordination of signals, that is, through both time and space. For example, the gaze control system employs signals that are time-locked to various sensorimotor events, but the spatial content of these signals is difficult to assess during ordinary gaze shifts. In this review, we describe the various models and methods that have been devised to test this question, and their limitations. We then describe a new method that can (a) simultaneously test between all of these models during natural, head-unrestrained conditions, and (b) track the evolving spatial continuum from target (T) to future gaze coding (G, including errors) through time. We then summarize some applications of this technique, comparing spatiotemporal coding in the primate frontal eye field (FEF) and superior colliculus (SC). The results confirm that these areas preferentially encode eye-centered, effector-independent parameters, and show-for the first time in ordinary gaze shifts-a spatial transformation between visual and motor responses from T to G coding. We introduce a new set of spatial models (T-G continuum) that revealed task-dependent timing of this transformation: progressive during a memory delay between vision and action, and almost immediate without such a delay. We synthesize the results from our studies and supplement it with previous knowledge of anatomy and physiology to propose a conceptual model where cumulative transformation noise is realized as inaccuracies in gaze behavior. We conclude that the spatiotemporal transformation for gaze is both local (observed within and across neurons in a given area) and distributed (with common signals shared across remote but interconnected structures).

Perceptual decisions about object shape bias visuomotor coordination during rapid interception movements

Visual processing for perception and for action is thought to be mediated by two specialized neural pathways. Using a visuomotor decision-making task, we show that participants differentially utilized online perceptual decision-making in reaching and interception and that eye movements necessary for perception influenced motor decision strategies. These results provide evidence that task complexity modulates how pathways processing perception versus action information interact during the visual control of movement.

Time-to-Target Simplifies Optimal Control of Visuomotor Feedback Responses

Visuomotor feedback responses vary in intensity throughout a reach, commonly explained by optimal control. Here, we show that the optimal control for a range of movements with the same goal can be simplified to a time-to-target dependent control scheme. We measure our human participants’ visuomotor responses in five reaching conditions, each with different hand or cursor kinematics. Participants only produced different feedback responses when these kinematic changes resulted in different times-to-target. We complement our experimental data with a range of finite and non-finite horizon optimal feedback control (OFC) models, finding that the model with time-to-target as one of the input parameters best replicates the experimental data. Overall, this suggests that time-to-target is a critical control parameter in online feedback control. Moreover, we propose that for a specific task and known dynamics, humans can instantly produce a control signal without any additional online computation allowing rapid response onset and close to optimal control.

Feedback Delay Changes the Control of an Inverted Pendulum

We recently developed a simulated inverted pendulum in order to examine human sensorimotor control strategies for stabilization. This simulated system allows us to manipulate the visual and haptic feedback independently from the physical dynamics of the task. Here we examine the effect of sensory delay in a balancing task. Human participants attempted to balance an inverted pendulum (simulated on a robotic manipulandum) with three different added delays (25, 50, and 75 ms), where the same delay was added to both the visual and haptic feedback. Increasing sensory delays decreased the ability of the participants to stabilize the pendulum. Investigation into the online control of the pendulum showed that with longer delays participants reduced their movement frequency but increased the amplitudes of their corrections.

Decision-making in sensorimotor control

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

Visual Feedback Processing of the Limb Involves Two Distinct Phases

Muscle responses to mechanical disturbances exhibit two distinct phases: a response starting at ~20 ms that is fairly stereotyped, and a response starting at ~60 ms modulated by many behavioral contexts including goal-redundancy and environmental obstacles. Muscle responses to disturbances of visual feedback of the hand arise within ~90 ms. However, little is known whether these muscle responses are sensitive to behavioral contexts. We had 49 human participants (27 male) execute goal-directed reaches with visual feedback of their hand presented as a cursor. On random trials, the cursor jumped laterally to the reach direction, and thus, required a correction to attain the goal. The first experiment demonstrated that the response amplitude starting at 90 ms scaled with jump magnitude, but only for jumps <2 cm. For larger jumps, the duration of the muscle response scaled with the jump size starting after 120 ms. The second experiment demonstrated that the early response was sensitive to goal redundancy as wider targets evoked a smaller corrective response. The third experiment demonstrated that the early response did not consider the presence of obstacles, as this response routinely drove participants directly to the goal even though this path was blocked by an obstacle. Instead, the appropriate muscle response to navigate around the obstacle started after 120 ms. Our findings highlight that visual feedback of the limb involves two distinct phases: a response starting at 90 ms with limited sensitivity to jump magnitude and sensitive to goal-redundancy, and a response starting at 120 ms with increased sensitivity to jump magnitude and environmental factors.SIGNIFICANCE STATEMENT The motor system can integrate proprioceptive feedback to guide an ongoing action in ~60 ms and is flexible to a broad range of behavioral contexts. In contrast, the present study identified that the motor response to a visual disturbance exhibits two distinct phases: an early response starting at 90 ms with limited scaling with disturbance size and sensitivity to goal-redundancy, and a slower response starting after 120 ms with increased sensitivity to disturbance size and sensitive to environmental obstacles. These data suggest visual feedback of the hand is processed through two distinct feedback processes.

A Technique for Measuring Visuomotor Feedback Contributions to the Control of an Inverted Pendulum

We developed a new technique to measure the contributions of rapid visuomotor feedback responses to the stabilization of a simulated inverted pendulum. Human participants balanced an inverted pendulum simulated on a robotic manipulandum. At a random time during the balancing task, the visual representation of the tip of the pendulum was shifted by a small displacement to the left or right while the motor response was measured. This response was either the exerted force against a fixation position, or the motion to re-stabilize the pendulum in the free condition. Our results demonstrate that rapid involuntary visuomotor feedback responses contribute to the stabilization of the pendulum.

Rapid Visuomotor Responses Reflect Value-Based Decisions

Cognitive decision-making is known to be sensitive to the values of potential options, which are the probability and size of rewards associated with different choices. Here, we examine whether rapid motor responses to perturbations of visual feedback about movement, which mediate low-level and involuntary feedback control loops, reflect computations associated with high-level value-based decision-making. In three experiments involving human participants, we varied the value associated with different potential targets for reaching movements by controlling the distributions of rewards across the targets (Experiment 1), the probability with which each target could be specified (Experiment 2), or both (Experiment 3). We found that the size of rapid and involuntary feedback responses to movement perturbations was strongly influenced by the relative value between targets. A statistical model of relative value that includes a term for risk sensitivity provided the best fit to the visuomotor response data, illustrating that feedback control policies are biased to favor more frequent task success at the expense of the overall extrinsic reward accumulated through movement. Importantly however, the regulation of rapid feedback responses was associated with successful pursuit of high-value task outcomes. This implies that when we move, the brain specifies a set of feedback control gains that enable low-level motor areas not only to generate efficient and accurate movement, but also to rapidly and adaptively respond to evolving sensory information in a manner consistent with value-based decision-making.SIGNIFICANCE STATEMENT Current theories of sensorimotor control suggest that, rather than selecting and planning the details of movements in advance, the role of the brain is to set time-varying feedback gains that continuously transform sensory information into motor commands by feedback control. Here, we examine whether the fastest motor responses to perturbations of movement, which mediate low-level and involuntary feedback control loops (i.e., reflexes), reflect computations associated with high-level, value-based decision-making. We find that rapid feedback responses during reaching reflect the relative probabilities and rewards associated with target options. This suggests that low-order components of the sensorimotor control hierarchy, which generate rapid and automatic responses, can continuously evaluate evolving sensory evidence and initiate responses according to the prospect of reward.

A Simulated Inverted Pendulum to Investigate Human Sensorimotor Control

Sensorimotor control regulates balance and stability as well as adaptation to the external environment. We introduce the use of a simulated inverted pendulum to study human sensorimotor control, demonstrating that this system introduces similar control challenges to human subjects as a physical inverted pendulum. Participants exhibited longer stabilization of the system as the pendulum length between the hand and the center of mass increased while the required control input varied in a non-monotonic, yet predictable manner. Finally, we show that the experimental results can be modelled as a PD controller with a time delay of $\tau = 140$ ms, matching the human visuomotor delay. Our results provide evidence of the importance of vision in a control of unstable systems and serve as a proof of concept of a simulated inverted pendulum.

State Estimation for Early Feedback Responses in Reaching: Intramodal or Multimodal?

Humans are highly skilled in controlling their reaching movements, making fast and task-dependent movement corrections to unforeseen perturbations. To guide these corrections, the neural control system requires a continuous, instantaneous estimate of the current state of the arm and body in the world. According to Optimal Feedback Control theory, this estimate is multimodal and constructed based on the integration of forward motor predictions and sensory feedback, such as proprioceptive, visual and vestibular information, modulated by context, and shaped by past experience. But how can a multimodal estimate drive fast movement corrections, given that the involved sensory modalities have different processing delays, different coordinate representations, and different noise levels? We develop the hypothesis that the earliest online movement corrections are based on multiple single modality state estimates rather than one combined multimodal estimate. We review studies that have investigated online multimodal integration for reach control and offer suggestions for experiments to test for the existence of intramodal state estimates. If proven true, the framework of Optimal Feedback Control needs to be extended with a stage of intramodal state estimation, serving to drive short-latency movement corrections.

Done in 100 ms: path-dependent visuomotor transformation in the human upper limb

A core assumption underlying mental chronometry is that more complex tasks increase cortical processing, prolonging reaction times. In this study we show that increases in task complexity alter the magnitude, rather than the latency, of the output for a circuit that rapidly transforms visual information into motor actions. We quantified visual stimulus-locked responses (SLRs), which are changes in upper limb muscle recruitment that evolve at a fixed latency ~100 ms after novel visual stimulus onset. First, we studied the underlying reference frame of the SLR by dissociating the initial eye and hand position. Despite its quick latency, we found that the SLR was expressed in a hand-centric reference frame, suggesting that the circuit mediating the SLR integrated retinotopic visual information with body configuration. Next, we studied the influence of planned movement trajectory, requiring participants to prepare and generate either curved or straight reaches in the presence of obstacles to attain the same visual stimulus location. We found that SLR magnitude was influenced by the planned movement trajectory to the same visual stimulus. On the basis of these results, we suggest that the circuit mediating the SLR lies in parallel to other well-studied corticospinal pathways. Although the fixed latency of the SLR precludes extensive cortical processing, inputs conveying information relating to task complexity, such as body configuration and planned movement trajectory, can preset nodes within the circuit underlying the SLR to modulate its magnitude. NEW & NOTEWORTHY We studied stimulus-locked responses (SLRs), which are changes in human upper limb muscle recruitment that evolve at a fixed latency ~100 ms after novel visual stimulus onset. We showed that despite its quick latency, the circuitry mediating the SLR transformed a retinotopic visual signal into a hand-centric motor command that is modulated by the planned movement trajectory. We suggest that the circuit generating the SLR is mediated through a tectoreticulospinal, rather than a corticospinal, pathway.

Rapid visuomotor feedback gains are tuned to the task dynamics

Here, we test whether rapid visuomotor feedback responses are selectively tuned to the task dynamics. The responses do not exhibit gain scaling, but they do vary with the level and stability of task dynamics. Moreover, these feedback gains are independently tuned to perturbations to the left and right, depending on these dynamics. Our results demonstrate that the sensorimotor control system regulates the feedback gain as part of the adaptation process, tuning them appropriately to the environment.

Adaptation to novel dynamics requires learning a motor memory, or a new pattern of predictive feedforward motor commands. Recently, we demonstrated the upregulation of rapid visuomotor feedback gains early in curl force field learning, which decrease once a predictive motor memory is learned. However, even after learning is complete, these feedback gains are higher than those observed in the null field trials. Interestingly, these upregulated feedback gains in the curl field were not observed in a constant force field. Therefore, we suggest that adaptation also involves selectively tuning the feedback sensitivity of the sensorimotor control system to the environment. Here, we test this hypothesis by measuring the rapid visuomotor feedback gains after subjects adapt to a variety of novel dynamics generated by a robotic manipulandum in three experiments. To probe the feedback gains, we measured the magnitude of the motor response to rapid shifts in the visual location of the hand during reaching. While the feedback gain magnitude remained similar over a larger than a fourfold increase in constant background load, the feedback gains scaled with increasing lateral resistance and increasing instability. The third experiment demonstrated that the feedback gains could also be independently tuned to perturbations to the left and right, depending on the lateral resistance, demonstrating the fractionation of feedback gains to environmental dynamics. Our results show that the sensorimotor control system regulates the gain of the feedback system as part of the adaptation process to novel dynamics, appropriately tuning them to the environment.

NEW & NOTEWORTHY Here, we test whether rapid visuomotor feedback responses are selectively tuned to the task dynamics. The responses do not exhibit gain scaling, but they do vary with the level and stability of task dynamics. Moreover, these feedback gains are independently tuned to perturbations to the left and right, depending on these dynamics. Our results demonstrate that the sensorimotor control system regulates the feedback gain as part of the adaptation process, tuning them appropriately to the environment.

Distinct and flexible rates of online control

Elliott et al. (Hum Mov Sci 10:393-418, 1991) proposed a pseudocontinuous model of online control whereby overlapping corrections lead to the appearance of smooth kinematic profiles in the presence of online feedback. More recently, it was also proposed that online control is not a singular process [see Elliott et al. (Psychol Bull 136(6):1023-1044, 2010)]. However, support for contemporary models of online control were based on methodologies that were not designed to be sensitive to different online control sub-processes. The current study sought to evaluate the possibility of multiple distinct (i.e., visual and non-visual) mechanisms contributing to the control of reaching movements completed in either a full-vision, a no-vision, or a no-vision memory-guided condition. Frequency domain analysis was applied to the acceleration traces of reaching movements. In an attempt to elicit a modulation in the online control mechanisms, these movements were completed at two levels of spatio-temporal constraint, namely with 10 and 30 cm target distances. One finding was that performance in the full-vision relative to both no-vision conditions could be distinguished via two distinct frequency peaks. Increases in the peak magnitude at the lower frequencies were associated with visuomotor mechanisms and increases in the peak magnitude at the higher frequencies were associated with non-visual mechanisms. In addition, performance to the 30-cm target led to a lower peak at a lower frequency relative to the 10 cm target, indicating that the iterative rates of visuomotor control mechanisms are flexible and sensitive to the spatio-temporal constraints of the associated movement.

Parallel Specification of Visuomotor Feedback Gains during Bimanual Reaching to Independent Goals

During goal-directed reaching, rapid visuomotor feedback processes enable the human motor system to quickly correct for errors in the trajectory of the hand that arise from motor noise and, in some cases, external perturbations. To date, these visuomotor responses, the gain of which is sensitive to features of the task and environment, have primarily been examined in the context of unimanual reaching movements toward a single target. However, many natural tasks involve moving both hands together, often to separate targets, such that errors can occur in parallel and at different spatial locations. Here, we examined the resource capacity of automatic visuomotor corrective mechanisms by comparing feedback gains during bimanual reaches, toward two targets, to feedback gains during unimanual reaches toward single targets. To investigate the sensitivity of the feedback gains and their relation to visual-spatial processing, we manipulated the widths of the targets and participants’ gaze location. We found that the gain of corrective responses to cursor displacements, while strongly modulated by target width and gaze position, were only slightly reduced during bimanual control. Our results show that automatic visuomotor corrective mechanisms can efficiently operate in parallel across multiple spatial locations.

When Optimal Feedback Control Is Not Enough: Feedforward Strategies Are Required for Optimal Control with Active Sensing

Movement planning is thought to be primarily determined by motor costs such as inaccuracy and effort. Solving for the optimal plan that minimizes these costs typically leads to specifying a time-varying feedback controller which both generates the movement and can optimally correct for errors that arise within a movement. However, the quality of the sensory feedback during a movement can depend substantially on the generated movement. We show that by incorporating such state-dependent sensory feedback, the optimal solution incorporates active sensing and is no longer a pure feedback process but includes a significant feedforward component. To examine whether people take into account such state-dependency in sensory feedback we asked people to make movements in which we controlled the reliability of sensory feedback. We made the visibility of the hand state-dependent, such that the visibility was proportional to the component of hand velocity in a particular direction. Subjects gradually adapted to such a sensory perturbation by making curved hand movements. In particular, they appeared to control the late visibility of the movement matching predictions of the optimal controller with state-dependent sensory noise. Our results show that trajectory planning is not only sensitive to motor costs but takes sensory costs into account and argues for optimal control of movement in which feedforward commands can play a significant role.

The dominant theory of how movements are planned suggests that a task specifies a motor cost in terms of effort and accuracy and the motor system chooses a movement to minimize this cost. The pre-eminent theory is that this results an optimal feedback controller that both generates the movement and can correct online for disturbances that arise due to factors such as noise. However, current formulations of the optimal feedback control problem have assumed that sensory information has noise independent of the action, or noise that simply scales with movement speed. In reality, visual and proprioceptive noise of limb position vary substantially with the state of the body. We show that including such variation in sensory feedback qualitatively changes the optimal solution so that feedback control alone is no longer optimal for the task and an additional feedforward controller must be used. To test this idea, we develop a novel motor learning paradigm in which we can vary the sensory noise on visual feedback of the limb and show that human participants adapt to such a perturbation to produce complex reach paths which are well accounted for by our model.

Visuomotor Map Determines How Visually Guided Reaching Movements are Corrected Within and Across Trials

When a visually guided reaching movement is unexpectedly perturbed, it is implicitly corrected in two ways: immediately after the perturbation by feedback control (online correction) and in the next movement by adjusting feedforward motor commands (offline correction or motor adaptation). Although recent studies have revealed a close relationship between feedback and feedforward controls, the nature of this relationship is not yet fully understood. Here, we show that both implicit online and offline movement corrections utilize the same visuomotor map for feedforward movement control that transforms the spatial location of visual objects into appropriate motor commands. First, we artificially distorted the visuomotor map by applying opposite visual rotations to the cursor representing the hand position while human participants reached for two different targets. This procedure implicitly altered the visuomotor map so that changes in the movement direction to the target location were more insensitive or more sensitive. Then, we examined how such visuomotor map distortion influenced online movement correction by suddenly changing the target location. The magnitude of online movement correction was altered according to the shape of the visuomotor map. We also examined offline movement correction; the aftereffect induced by visual rotation in the previous trial was modulated according to the shape of the visuomotor map. These results highlighted the importance of the visuomotor map as a foundation for implicit motor control mechanisms and the intimate relationship between feedforward control, feedback control, and motor adaptation.