Precis of Vigor: Neuroeconomics of movement control

The expression of decision and learning variables in movement patterns related to decision actions

Decisions are not necessarily easy to separate into a planning and an execution phase and the decision-making process can often be reflected in the movement associated with the decision. Here, we used formalized definitions of concepts relevant in decision-making and learning to explore if and how these concepts correlate with decision-related movement paths, both during and after a choice is made. To this end, we let 120 participants (46 males, mean age = 24.5 years) undergo a repeated probabilistic two-choice task with changing probabilities where we used mouse-tracking, a simple non-invasive technique, to study the movements related to decisions. The decisions of the participants were modelled using Bayesian inference which enabled the computation of variables related to decision-making and learning. Analyses of the movement during the decision showed effects of relevant decision variables, such as confidence, on aspects related to, for instance, timing and pausing, range of movement and deviation from the shortest distance. For the movements after a decision there were some effects of relevant learning variables, mainly related to timing and speed. We believe our findings can be of interest for researchers within several fields, spanning from social learning to experimental methods and human-machine/robot interaction.

Motivational Vigor in Parkinson's Disease Requires the Short and Long Duration Response to Levodopa

BACKGROUND

Impaired movement vigor (bradykinesia) is a cardinal feature of Parkinson's disease (PD) and hypothesized to result from abnormal motivational processes-impaired motivation-vigor coupling. Dopamine replacement therapy (DRT) improves bradykinesia, but the response to DRT is multifaceted, comprising a short-duration response (SDR) and a long-duration response (LDR) only manifesting with chronic treatment. Prior experiments assessing motivation-vigor coupling in PD used chronically treated subjects, obscuring the roles of the SDR and LDR.

METHODS

To disambiguate the SDR and LDR, 11 de novo PD subjects (6 male [M]:5 female [F]; mean age, 67) were studied before treatment, after an acute levodopa (l-dopa) dose, and in both the practical "off" (LDR) and "on" (LDR + SDR) states after chronic stable treatment. At each visit, subjects were characterized with a standard battery including the Movement Disorder Society-Sponsored Revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS) and an incentivized joystick task to assess motor performance in response to varying rewards.

RESULTS

l-Dopa induced a robust SDR and LDR, with further improvement in the combined SDR + LDR state. At baseline, after acute treatment (SDR), and after LDR induction, subjects did not exhibit the normal increase in movement speed with increasing reward. Only in the combined SDR + LDR state was there restoration of motivation-vigor coupling.

CONCLUSIONS

Although consistent with prior results in chronically treated PD subjects, the significant improvement in motor performance observed with the SDR and LDR suggests that bradykinesia is not solely secondary to deficient modulation of motivational processes. © 2023 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

From neural noise to co-adaptability: Rethinking the multifaceted architecture of motor variability

In the last decade, the source and the functional meaning of motor variability have attracted considerable attention in behavioral and brain sciences. This construct classically combined different levels of description, variable internal robustness or coherence, and multifaceted operational meanings. We provide here a comprehensive review of the literature with the primary aim of building a precise lexicon that goes beyond the generic and monolithic use of motor variability. In the pars destruens of the work, we model three domains of motor variability related to peculiar computational elements that influence fluctuations in motor outputs. Each domain is in turn characterized by multiple sub-domains. We begin with the domains of noise and differentiation. However, the main contribution of our model concerns the domain of adaptability, which refers to variation within the same exact motor representation. In particular, we use the terms learning and (social)fitting to specify the portions of motor variability that depend on our propensity to learn and on our largely constitutive propensity to be influenced by external factors. A particular focus is on motor variability in the context of the sub-domain named co-adaptability. Further groundbreaking challenges arise in the modeling of motor variability. Therefore, in a separate pars construens, we attempt to characterize these challenges, addressing both theoretical and experimental aspects as well as potential clinical implications for neurorehabilitation. All in all, our work suggests that motor variability is neither simply detrimental nor beneficial, and that studying its fluctuations can provide meaningful insights for future research.

The value of time in the invigoration of human movements when interacting with a robotic exoskeleton

Time and effort are thought to be subjectively balanced during the planning of goal-directed actions, thereby setting the vigor of volitional movements. Theoretical models predicted that the value of time should then amount to high levels of effort. However, the time-effort trade-off has so far only been studied for a narrow range of efforts. To investigate the extent to which humans can invest in a time-saving effort, we used a robotic exoskeleton to substantially vary the energetic cost associated with a certain vigor during reaching movements. In this situation, minimizing the time-effort trade-off should lead to high and low human efforts for upward and downward movements, respectively. Consistently, all participants expended substantial amounts of energy upward and remained essentially inactive by harnessing the work of gravity downward, while saving time in both cases. A common time-effort trade-off may therefore determine the vigor of reaching movements for a wide range of efforts.

Neural substrates of continuous and discrete inhibitory control

Inhibitory control dysfunctions play an important role in psychiatric disorders but the precise nature of these dysfunctions is still not well understood. Advances in computational modeling of real-time motor control using a proportion-integral-derivative (PID) control framework have parsed continuous motor inhibition into a preemptive drive component (signified by the K_p parameter) and a reactive damping component (signified by the K_d parameter). This investigation examined the relationship between inhibitory control processing during a stop signal task and continuous motor control during a simulated one-dimensional driving task in a transdiagnostic sample of participants. A transdiagnostic psychiatric sample of 492 individuals completed a stop signal task during functional magnetic resonance imaging and a simple behavioral motor control task, which was modeled using the PID framework. We examined associations between the K_p and K_d parameters and behavioral indices as well as neural activation on the stop signal task. Individuals with higher damping, controlling for a drive, on the driving task exhibited relatively less strategic adjustment after a stop trial (indexed by the difference in go trial reaction time and by stop trial accuracy) on the stop signal task. Individuals with higher damping, controlling for a drive, additionally exhibited increased activity in the frontal and parietal regions as well as the insula and caudate during response inhibition on the stop signal task. The results suggest that computational indices of motor control performance may serve as behavioral markers of the functioning of neural systems involved in inhibitory control.

From thinking fast to moving fast: motor control of fast limb movements in healthy individuals

The ability to produce high movement speeds is a crucial factor in human motor performance, from the skilled athlete to someone avoiding a fall. Despite this relevance, there remains a lack of both an integrative brain-to-behavior analysis of these movements and applied studies linking the known dependence on open-loop, central control mechanisms of these movements to their real-world implications, whether in the sports, performance arts, or occupational setting. In this review, we cover factors associated with the planning and performance of fast limb movements, from the generation of the motor command in the brain to the observed motor output. At each level (supraspinal, peripheral, and motor output), the influencing factors are presented and the changes brought by training and fatigue are discussed. The existing evidence of more applied studies relevant to practical aspects of human performance is also discussed. Inconsistencies in the existing literature both in the definitions and findings are highlighted, along with suggestions for further studies on the topic of fast limb movement control. The current heterogeneity in what is considered a fast movement and in experimental protocols makes it difficult to compare findings in the existing literature. We identified the role of the cerebellum in movement prediction and of surround inhibition in motor slowing, as well as the effects of fatigue and training on central motor control, as possible avenues for further research, especially in performance-driven populations.

The Promise of Behavioral Tracking Systems for Advancing Primate Animal Welfare

Simple Summary

Computerized tracking systems for primates and other animals are one of the great inventions of the 21st century. These systems have already revolutionized the study of primatology, psychology, neuroscience, and biomedicine. Less discussed is that they also promise to greatly enhance animal welfare. Their potential benefits include identifying and reducing pain, suffering, and distress in captive populations, improving laboratory animal welfare, and applying our understanding of animal behavior to increase the “natural” behaviors in captive and wild populations, especially those under threat. We are optimistic that these changes will greatly increase the welfare of primates, including those in laboratories, zoos, primate centers, and in the wild.

Abstract

Recent years have witnessed major advances in the ability of computerized systems to track the positions of animals as they move through large and unconstrained environments. These systems have so far been a great boon in the fields of primatology, psychology, neuroscience, and biomedicine. Here, we discuss the promise of these technologies for animal welfare. Their potential benefits include identifying and reducing pain, suffering, and distress in captive populations, improving laboratory animal welfare within the context of the three Rs of animal research (reduction, refinement, and replacement), and applying our understanding of animal behavior to increase the “natural” behaviors in captive and wild populations facing human impact challenges. We note that these benefits are often incidental to the designed purpose of these tracking systems, a reflection of the fact that animal welfare is not inimical to research progress, but instead, that the aligned interests between basic research and welfare hold great promise for improvements to animal well-being.

Evidence for a cost of time in the invigoration of isometric reaching movements

How the brain determines the vigor of goal-directed movements is a fundamental question in neuroscience. Recent evidence has suggested that vigor results from a trade-off between a cost related to movement production (cost of movement) and a cost related to our brain's tendency to temporally discount the value of future reward (cost of time). However, whether it is critical to hypothesize a cost of time to explain the vigor of basic reaching movements with intangible reward is unclear because the cost of movement may be theoretically sufficient for this purpose. Here we directly address this issue by designing an isometric reaching task whose completion can be accurate and effortless in prefixed durations. The cost of time hypothesis predicts that participants should be prone to spend energy to save time even if the task can be accomplished at virtually no motor cost. Accordingly, we found that all participants generated substantial amounts of force to invigorate task accomplishment, especially when the prefixed duration was long enough. Remarkably, the time saved by each participant was linked to their original vigor in the task and predicted by an optimal control model balancing out movement and time costs. Taken together, these results supports the existence of an idiosyncratic, cognitive cost of time that underlies the invigoration of basic isometric reaching movements.

Effort, success, and side of lesion determine arm choice in individuals with chronic stroke

In neurotypical individuals, arm choice in reaching movements depends on expected biomechanical effort, expected success, and a handedness bias. Following a stroke, does arm choice change to account for the decreased motor performance, or does it follow a preinjury habitual preference pattern? Participants with mild-to-moderate chronic stroke who were right-handed before stroke performed reaching movements in both spontaneous and forced-choice blocks, under no-time, medium-time, and fast-time constraint conditions designed to modulate reaching success. Mixed-effects logistic regression models of arm choice revealed that expected effort predicted choices. However, expected success only strongly predicted choice in left-hemiparetic individuals. In addition, reaction times decreased in left-hemiparetic individuals between the no-time and the fast-time constraint conditions but showed no changes in right-hemiparetic individuals. Finally, arm choice in the no-time constraint condition correlated with a clinical measure of spontaneous arm use for right-, but not for left-hemiparetic individuals. Our results are consistent with the view that right-hemiparetic individuals show a habitual pattern of arm choice for reaching movements relatively independent of failures. In contrast, left-hemiparetic individuals appear to choose their paretic left arm more optimally: that is, if a movement with the paretic arm is predicted to be not successful in the upcoming movement, the nonparetic right arm is chosen instead.NEW & NOTEWORTHY Although we are seldom aware of it, we constantly make decisions to use one arm or the other in daily activities. Here, we studied whether these decisions change following stroke. Our results show that effort, success, and side of lesion determine arm choice in a reaching task: whereas left-paretic individuals modified their arm choice in response to failures in reaching the target, right-paretic individuals showed a pattern of choice independent of failures.

Dissociating the Impact of Movement Time and Energy Costs on Decision-Making and Action Initiation in Humans

Recent theories and data suggest that adapted behavior involves economic computations during which multiple trade-offs between reward value, accuracy requirement, energy expenditure, and elapsing time are solved so as to obtain rewards as soon as possible while spending the least possible amount of energy. However, the relative impact of movement energy and duration costs on perceptual decision-making and movement initiation is poorly understood. Here, we tested 31 healthy subjects on a perceptual decision-making task in which they executed reaching movements to report probabilistic choices. In distinct blocks of trials, the reaching duration (“Time” condition) and energy (“Effort” condition) costs were independently varied compared to a “Reference” block, while decision difficulty was maintained similar at the block level. Participants also performed a simple delayed-reaching (DR) task aimed at estimating movement initiation duration in each motor condition. Results in that DR task show that long duration movements extended reaction times (RTs) in most subjects, whereas energy-consuming movements led to mixed effects on RTs. In the decision task, about half of the subjects decreased their decision durations (DDs) in the Time condition, while the impact of energy on DDs were again mixed across subjects. Decision accuracy was overall similar across motor conditions. These results indicate that movement duration and, to a lesser extent, energy expenditure, idiosyncratically affect perceptual decision-making and action initiation. We propose that subjects who shortened their choices in the time-consuming condition of the decision task did so to limit a drop of reward rate.

Humans use minimum cost movements in a whole-body task

Humans have elegant bodies that allow gymnastics, piano playing, and tool use, but understanding how they do this in detail is difficult because their musculoskeletal systems are extraordinarily complicated. Nonetheless, common movements like walking and reaching can be stereotypical, and a very large number of studies have shown their energetic cost to be a major factor. In contrast, one might think that general movements are very individuated and intractable, but our previous study has shown that in an arbitrary set of whole-body movements used to trace large-scale closed curves, near-identical posture sequences were chosen across different subjects, both in the average trajectories of the body's limbs and in the variance within trajectories. The commonalities in that result motivate explanations for its generality. One explanation could be that humans also choose trajectories that are economical in cost. To test this hypothesis, we situate the tracing data within a forty eight degree of freedom human dynamic model that allows the computation of movement cost. Using the model to compare movement cost data from nominal tracings against various perturbed tracings shows that the latter are more energetically expensive, inferring that the original traces were chosen on the basis of minimum cost.

Why Does Rubin's Vase Differ Radically From Optical Illusions? Framing Effects Contra Cognitive Illusions

Many researchers use the term "context" loosely to denote diverse kinds of reference points. The issue is not about terminology but rather about the common conflation of one kind of reference points, such as rules of perception, which is responsible for optical illusions, with another kind, known as "context" or "frame," as exemplified in Rubin's vase. Many researchers regard Rubin's vase as a special kind of optical illusions. This paper rather argues that the two phenomena are radically different. Optical illusions are occasional mistakes that people quickly recognize and eagerly correct, while the different figures of Rubin's vase are not mistakes but, rather, the outcomes of different perspectives that do not need correction. The competing figures in Rubin's vase can, at best, in light of more information, be more warranted or unwarranted. This paper discusses at length one ramification of the proposed distinction. The framing effects, such as loss/gain frame, are the products of contexts and, hence, resemble greatly the figures in Rubin's vase. In contrast, cognitive illusions generated occasionally by the rules of thumb (heuristics) are mistakes and, hence, resemble optical illusions. The proposed distinction carries other ramifications regarding, e.g., happiness studies, moral judgments, and the new philosophy of science.

The cost of correcting for error during sensorimotor adaptation

Learning from error is often a slow process. In machine learning, the learning rate depends on a loss function that specifies a cost for error. Here, we hypothesized that during motor learning, error carries an implicit cost for the brain because the act of correcting for error consumes time and energy. Thus, if this implicit cost could be increased, it may robustly alter how the brain learns from error. To vary the implicit cost of error, we designed a task that combined saccade adaptation with motion discrimination: movement errors resulted in corrective saccades, but those corrections took time away from acquiring information in the discrimination task. We then modulated error cost using coherence of the discrimination task and found that when error cost was large, pupil diameter increased and the brain learned more from error. However, when error cost was small, the pupil constricted and the brain learned less from the same error. Thus, during sensorimotor adaptation, the act of correcting for error carries an implicit cost for the brain. Modulating this cost affects how much the brain learns from error.

The Role of Information in Knowledge-How

Knowledge-how is the kind of knowledge implicated in skill employment and acquisition. Intellectualists claim that knowledge-how is a special type of propositional knowledge. Anti-intellectualists claim that knowledge-how is not propositional. We argue that both views face two open challenges. The first challenge pertains to the relationship between informational states and motor variability. The second pertains to the epistemic function of practice that leads to skill (and knowledge-how). The aim of this paper is to suggest a general conceptual framework based on functional information with both intellectualist and anti-intellectualist features. Our proposal, we argue, avoids the above challenges, and can further the debate on knowledge-how and skill.

Using metabolic energy to quantify the subjective value of physical effort

Economists have known for centuries that to understand an individual's decisions, we must consider not only the objective value of the goal at stake, but its subjective value as well. However, achieving that goal ultimately requires expenditure of effort. Surprisingly, despite the ubiquitous role of effort in decision-making and movement, we currently do not understand how effort is subjectively valued in daily movements. Part of the difficulty arises from the lack of an objective measure of effort. Here, we use a physiological approach to address this knowledge gap. We quantified objective effort costs by measuring metabolic cost via expired gas analysis as participants performed a reaching task against increasing resistance. We then used neuroeconomic methods to quantify each individual's subjective valuation of effort. Rather than the diminishing sensitivity observed in reward valuation, effort was valued objectively, on average. This is significantly less than the near-quadratic sensitivity to effort observed previously in force-based motor tasks. Moreover, there was significant inter-individual variability with many participants undervaluing or overvaluing effort. These findings demonstrate that in contrast with monetary decisions in which subjective value exhibits diminishing marginal returns, effort costs are valued more objectively in low-effort reaching movements common in daily life.

Stochastic optimal feedforward-feedback control determines timing and variability of arm movements with or without vision

Human movements with or without vision exhibit timing (i.e. speed and duration) and variability characteristics which are not well captured by existing computational models. Here, we introduce a stochastic optimal feedforward-feedback control (SFFC) model that can predict the nominal timing and trial-by-trial variability of self-paced arm reaching movements carried out with or without online visual feedback of the hand. In SFFC, movement timing results from the minimization of the intrinsic factors of effort and variance due to constant and signal-dependent motor noise, and movement variability depends on the integration of visual feedback. Reaching arm movements data are used to examine the effect of online vision on movement timing and variability, and test the model. This modelling suggests that the central nervous system predicts the effects of sensorimotor noise to generate an optimal feedforward motor command, and triggers optimal feedback corrections to task-related errors based on the available limb state estimate.

Collective music listening: Movement energy is enhanced by groove and visual social cues

The regularity of musical beat makes it a powerful stimulus promoting movement synchrony among people. Synchrony can increase interpersonal trust, affiliation, and cooperation. Musical pieces can be classified according to the quality of groove; the higher the groove, the more it induces the desire to move. We investigated questions related to collective music-listening among 33 participants in an experiment conducted in a naturalistic yet acoustically controlled setting of a research concert hall with motion tracking. First, does higher groove music induce (1) movement with more energy and (2) higher interpersonal movement coordination? Second, does visual social information manipulated by having eyes open or eyes closed also affect energy and coordination? Participants listened to pieces from four categories formed by crossing groove (high, low) with tempo (higher, lower). Their upper body movement was recorded via head markers. Self-reported ratings of grooviness, emotional valence, emotional intensity, and familiarity were collected after each song. A biomechanically motivated measure of movement energy increased with high-groove songs and was positively correlated with grooviness ratings, confirming the theoretically implied but less tested motor response to groove. Participants’ ratings of emotional valence and emotional intensity correlated positively with movement energy, suggesting that movement energy relates to emotional engagement with music. Movement energy was higher in eyes-open trials, suggesting that seeing each other enhanced participants’ responses, consistent with social facilitation or contagion. Furthermore, interpersonal coordination was higher both for the high-groove and eyes-open conditions, indicating that the social situation of collective music listening affects how music is experienced.

Perceived effort affects choice of limb and reaction time of movements

The decision regarding which arm to use to perform a task reflects a complex process that can be influenced by many factors, including effort requirements of acquiring the goal. In this study, we considered a virtual reality environment in which people reached to a visual target in three-dimensional space. To vary the cost of reaching, we altered the visual feedback associated with motion of one arm but not the other. This altered the extent of motion that was required to reach, thus changing the effort required to acquire the goal. We then measured how that change in effort affected the decision regarding which arm to use, as well as the preparation time for the movement that ensued. As expected, with increased visual amplification of one arm (reduced effort to reach the goal), subjects increased the probability of choosing that arm. Surprisingly, however, the reaction times to start these movements were also reduced: despite constancy of the visual representation of the target, reaction times were shorter for movements with less effort. Thus, as the perceived effort associated with accomplishing a goal was reduced for a given limb, the decision-making process was biased toward use of that limb. Furthermore, movements that were perceived to be less effortful were performed with shorter reaction times. These results suggest that visual amplification can alter the perceived effort associated with using a limb, thus increasing frequency of use. This may provide a useful method to increase use of a limb during rehabilitation.NEW & NOTEWORTHY We report that visual amplification may serve as an effective means to alter the perceived effort associated with use of a limb. This method may provide an effective tool with which use of the affected limb can be encouraged noninvasively after neurological injury.

Population coding in the cerebellum: a machine learning perspective

The cere resembles a feedforward, three-layer network of neurons in which the "hidden layer" consists of Purkinje cells (P-cells) and the output layer consists of deep cerebellar nucleus (DCN) neurons. In this analogy, the output of each DCN neuron is a prediction that is compared with the actual observation, resulting in an error signal that originates in the inferior olive. Efficient learning requires that the error signal reach the DCN neurons, as well as the P-cells that project onto them. However, this basic rule of learning is violated in the cerebellum: the olivary projections to the DCN are weak, particularly in adulthood. Instead, an extraordinarily strong signal is sent from the olive to the P-cells, producing complex spikes. Curiously, P-cells are grouped into small populations that converge onto single DCN neurons. Why are the P-cells organized in this way, and what is the membership criterion of each population? Here, I apply elementary mathematics from machine learning and consider the fact that P-cells that form a population exhibit a special property: they can synchronize their complex spikes, which in turn suppress activity of DCN neuron they project to. Thus complex spikes cannot only act as a teaching signal for a P-cell, but through complex spike synchrony, a P-cell population may act as a surrogate teacher for the DCN neuron that produced the erroneous output. It appears that grouping of P-cells into small populations that share a preference for error satisfies a critical requirement of efficient learning: providing error information to the output layer neuron (DCN) that was responsible for the error, as well as the hidden layer neurons (P-cells) that contributed to it. This population coding may account for several remarkable features of behavior during learning, including multiple timescales, protection from erasure, and spontaneous recovery of memory.