Decision making, movement planning and statistical decision theory

Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach

Learning to make adaptive decisions involves making choices, assessing their consequence, and leveraging this assessment to attain higher rewarding states. Despite vast literature on value-based decision-making, relatively little is known about the cognitive processes underlying decisions in highly uncertain contexts. Real world decisions are rarely accompanied by immediate feedback, explicit rewards, or complete knowledge of the environment. Being able to make informed decisions in such contexts requires significant knowledge about the environment, which can only be gained via exploration. Here we aim at understanding and formalizing the brain mechanisms underlying these processes. To this end, we first designed and performed an experimental task. Human participants had to learn to maximize reward while making sequences of decisions with only basic knowledge of the environment, and in the absence of explicit performance cues. Participants had to rely on their own internal assessment of performance to reveal a covert relationship between their choices and their subsequent consequences to find a strategy leading to the highest cumulative reward. Our results show that the participants' reaction times were longer whenever the decision involved a future consequence, suggesting greater introspection whenever a delayed value had to be considered. The learning time varied significantly across participants. Second, we formalized the neurocognitive processes underlying decision-making within this task, combining mean-field representations of competing neural populations with a reinforcement learning mechanism. This model provided a plausible characterization of the brain dynamics underlying these processes, and reproduced each aspect of the participants' behavior, from their reaction times and choices to their learning rates. In summary, both the experimental results and the model provide a principled explanation to how delayed value may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in these uncertain scenarios.

Sensorimotor confidence for tracking eye movements

For successful interactions with the world, we often have to evaluate our own performance. Although eye movements are one of the most frequent actions we perform, we are typically unaware of them. Here, we investigated whether there is any evidence for metacognitive sensitivity for the accuracy of eye movements. Participants tracked a dot cloud as it followed an unpredictable sinusoidal trajectory and then reported if they thought their performance was better or worse than their average tracking performance. Our results show above-chance identification of better tracking behavior across all trials and also for repeated attempts of the same target trajectories. Sensitivity in discriminating performance between better and worse trials was stable across sessions, but judgements within a trial relied more on performance in the final seconds. This behavior matched previous reports when judging the quality of hand movements, although overall metacognitive sensitivity for eye movements was significantly lower.

Getting stuck in a rut as an emergent feature of a dynamic decision-making system

Human sensorimotor decision making has a tendency to get 'stuck in a rut', being biased towards selecting a previously implemented action structure (hysteresis). Existing explanations propose this is the consequence of an agent efficiently modifying an existing plan, rather than creating a new plan from scratch. Instead, we propose that hysteresis is an emergent property of a system learning from the consequences of its actions. To examine this, 152 participants moved a cursor to a target on a tablet device while avoiding an obstacle. Hysteresis was observed when the obstacle moved sequentially across the screen between trials, whereby the participant continued moving around the same side of the obstacle despite it now requiring a larger movement than the alternative. Two further experiments (n = 20) showed an attenuation when time and resource constraints were eased. We created a simple computational model capturing probabilistic estimate updating that showed the same patterns of results. This provides, to our knowledge, the first computational demonstration of how sensorimotor decision making can get 'stuck in a rut' through the updating of the probability estimates associated with actions.

Systematic bias in representation of reaction time distribution

A correct perception of one's own abilities is essential for making appropriate decisions. A well-known bias in probability perception is that rare events are overestimated. Here, we examined whether such a bias also exists for action outcomes using a simple reaction task. In Experiment 1, after completing a set of 30 trials of the simple reaction task, participants were required to judge the probability that they would be able to respond before a given reference time when performing the task next. We assessed the difference between the actual reaction times and the probability judgement and found that the represented probability distribution was more widely distributed than the actual one, suggesting that low-probability events were overestimated and high-probability events were underestimated. Experiment 2 confirmed the presence of such a bias in the representation of both one's own and another's reaction times. In addition, Experiment 3 showed the presence of such a bias regardless of the difference between the representation of another's reaction times and the mere numerical representation. Furthermore, Experiment 4 found the presence of such a bias even when the information regarding actual reaction times was visually shown before the representation. The present results reveal the existence of a highly robust bias in the representation of motor performance, which reflects the ubiquitous bias in probability perception and is difficult to eliminate.

Metacognition and Confidence: A Review and Synthesis

Determining the psychological, computational, and neural bases of confidence and uncertainty holds promise for understanding foundational aspects of human metacognition. While a neuroscience of confidence has focused on the mechanisms underpinning subpersonal phenomena such as representations of uncertainty in the visual or motor system, metacognition research has been concerned with personal-level beliefs and knowledge about self-performance. I provide a road map for bridging this divide by focusing on a particular class of confidence computation: propositional confidence in one's own (hypothetical) decisions or actions. Propositional confidence is informed by the observer's models of the world and their cognitive system, which may be more or less accurate-thus explaining why metacognitive judgments are inferential and sometimes diverge from task performance. Disparate findings on the neural basis of uncertainty and performance monitoring are integrated into a common framework, and a new understanding of the locus of action of metacognitive interventions is developed.

Corrective feedback guides human perceptual decision-making by informing about the world state rather than rewarding its choice

Corrective feedback received on perceptual decisions is crucial for adjusting decision-making strategies to improve future choices. However, its complex interaction with other decision components, such as previous stimuli and choices, challenges a principled account of how it shapes subsequent decisions. One popular approach, based on animal behavior and extended to human perceptual decision-making, employs "reinforcement learning," a principle proven successful in reward-based decision-making. The core idea behind this approach is that decision-makers, although engaged in a perceptual task, treat corrective feedback as rewards from which they learn choice values. Here, we explore an alternative idea, which is that humans consider corrective feedback on perceptual decisions as evidence of the actual state of the world rather than as rewards for their choices. By implementing these "feedback-as-reward" and "feedback-as-evidence" hypotheses on a shared learning platform, we show that the latter outperforms the former in explaining how corrective feedback adjusts the decision-making strategy along with past stimuli and choices. Our work suggests that humans learn about what has happened in their environment rather than the values of their own choices through corrective feedback during perceptual decision-making.

Studying the neural representations of uncertainty

The study of the brain's representations of uncertainty is a central topic in neuroscience. Unlike most quantities of which the neural representation is studied, uncertainty is a property of an observer's beliefs about the world, which poses specific methodological challenges. We analyze how the literature on the neural representations of uncertainty addresses those challenges and distinguish between 'code-driven' and 'correlational' approaches. Code-driven approaches make assumptions about the neural code for representing world states and the associated uncertainty. By contrast, correlational approaches search for relationships between uncertainty and neural activity without constraints on the neural representation of the world state that this uncertainty accompanies. To compare these two approaches, we apply several criteria for neural representations: sensitivity, specificity, invariance and functionality. Our analysis reveals that the two approaches lead to different but complementary findings, shaping new research questions and guiding future experiments.

Detecting visual texture patterns in binary sequences through pattern features

We measured human ability to detect texture patterns in a signal detection task. Observers viewed sequences of 20 blue or yellow tokens placed horizontally in a row. They attempted to discriminate sequences generated by a random generator ("a fair coin") from sequences produced by a disrupted Markov sequence (DMS) generator. The DMSs were generated in two stages: first a sequence was generated using a Markov chain with probability, pr = 0.9, that a token would be the same color as the token to its left. The Markov sequence was then disrupted by flipping each token from blue to yellow or vice versa with probability, pd-the probability of disruption. Disruption played the role of noise in signal detection terms. We can frame what observers are asked to do as detecting Markov texture patterns disrupted by noise. The experiment included three conditions differing in pd (0.1, 0.2, 0.3). Ninety-two observers participated, each in only one condition. Overall, human observers' sensitivities to texture patterns (d' values) were markedly less than those of an optimal Bayesian observer. We considered the possibility that observers based their judgments not on the entire texture sequence but on specific features of the sequences such as the length of the longest repeating subsequence. We compared human performance with that of multiple optimal Bayesian classifiers based on such features. We identify the single- and multiple-feature models that best match the performance of observers across conditions and develop a pattern feature pool model for the signal detection task considered.

Motor Decision-Making as a Common Denominator in Motor Pathology and a Possible Rehabilitation Target

Despite the substantial progress in motor rehabilitation, patient involvement and motivation remain major challenges. They are typically addressed with communicational and environmental strategies, as well as with improved goal-setting procedures. Here we suggest a new research direction and framework involving Neuroeconomics principles to investigate the role of Motor Decision-Making (MDM) parameters in motivational component and motor performance in rehabilitation. We argue that investigating NE principles could bring new approaches aimed at increasing active patient engagement in the rehabilitation process by introducing more movement choice, and adapting existing goal-setting procedures. We discuss possible MDM implementation strategies and illustrate possible research directions using examples of stroke and psychiatric disorders.

Motor variability modulates calibration of precisely timed movements

Interacting with the environment often requires precisely timed movements, challenging the brain to minimize the detrimental impact of neural noise. Recent research demonstrates that the brain exploits the variability of its temporal estimates and recalibrates perception accordingly. Time-critical movements, however, contain a sensory measurement and a motor stage. The brain must have knowledge of both in order to avoid maladapted behavior. By manipulating sensory and motor variability, we show that the sensorimotor system recalibrates sensory and motor uncertainty separately. Serial dependencies between observed interval durations in the previous and motor reproductions in the current trial were weighted by the variability of movements. These serial dependencies generalized across different effectors, but not to a visual discrimination task. Our results suggest that the brain has accurate knowledge about contributions of motor uncertainty to errors in temporal movements. This knowledge about motor uncertainty seems to be processed separately from knowledge about sensory uncertainty.

Feeling lucky? prospective and retrospective cues for sensorimotor confidence

On a daily basis, humans interact with the outside world using judgments of sensorimotor confidence, constantly evaluating our actions for success. We ask, what sensory and motor-execution cues are used in making these judgements and when are they available? Two sources of temporally distinct information are prospective cues, available prior to the action (e.g., knowledge of motor noise and past performance), and retrospective cues specific to the action itself (e.g., proprioceptive measurements). We investigated the use of these two cues in two tasks, a secondary motor-awareness task and a main task in which participants reached toward a visual target with an unseen hand and then made a continuous judgment of confidence about the success of the reach. Confidence was reported by setting the size of a circle centered on the reach-target location, where a larger circle reflects lower confidence. Points were awarded if the confidence circle enclosed the true endpoint, with fewer points returned for larger circles. This incentivized accurate reaches and attentive reporting to maximize the score. We compared three Bayesian-inference models of sensorimotor confidence based on either prospective cues, retrospective cues, or both sources of information to maximize expected gain (i.e., an ideal-performance model). Our findings showed two distinct strategies: participants either performed as ideal observers, using both prospective and retrospective cues to make the confidence judgment, or relied solely on prospective information, ignoring retrospective cues. Thus, participants can make use of retrospective cues, evidenced by the behavior observed in our motor-awareness task, but these cues are not always included in the computation of sensorimotor confidence.

Sensory and multisensory reasoning: Is Bayesian updating modality-dependent?

Bayesianism assumes that probabilistic updating does not depend on the sensory modality by which information is processed. In this study, we investigate whether probability judgments based on visual and auditory information conform to this assumption. In a series of five experiments, we found that this is indeed the case when information is acquired through a single modality (i.e., only auditory or only visual) but not necessarily so when it comes from multiple modalities (i.e., audio-visual). In the latter case, judgments prove more accurate when both visual and auditory information individually support (i.e., increase the probability of) the hypothesis they also jointly support (synergy condition) than when either visual or auditory information support one hypothesis that is not the one they jointly support (contrast condition). In the extreme case in which both visual and auditory information individually support an alternative hypothesis to the one they jointly support (i.e., double-contrast condition), participants' accuracy is not only lower than in the synergy condition but near chance. This synergy-contrast effect represents a violation of the assumption that information modality is irrelevant for Bayesian updating and indicates an important limitation of multisensory integration, one which has not been previously documented.

Capacity Limits Lead to Information Bottlenecks in Ongoing Rapid Motor Behaviors

Studies of ongoing, rapid motor behaviors have often focused on the decision-making implicit in the task. Here, we instead study how decision-making integrates with the perceptual and motor systems and propose a framework of limited-capacity, pipelined processing with flexible resources to understand rapid motor behaviors. Results from three experiments show that human performance is consistent with our framework: participants perform objectively worse as task difficulty increases, and, surprisingly, this drop in performance is largest for the most skilled performers. As well, our analysis shows that the worst-performing participants can perform equally well under increased task demands, which is consistent with flexible neural resources being allocated to reduce bottleneck effects and improve overall performance. We conclude that capacity limits lead to information bottlenecks and that processes like attention help reduce the effects that these bottlenecks have on maximal performance.

Reward-Dependent Selection of Feedback Gains Impacts Rapid Motor Decisions

Target reward influences motor planning strategies through modulation of movement vigor. Considering current theories of sensorimotor control suggesting that movement planning consists in selecting a goal-directed control strategy, we sought to investigate the influence of reward on feedback control. Here, we explored this question in three human reaching experiments. First, we altered the explicit reward associated with the goal target and found an overall increase in feedback gains for higher target rewards, highlighted by larger velocities, feedback responses to external loads, and background muscle activity. Then, we investigated whether the differences in target rewards across multiple goals impacted rapid motor decisions during movement. We observed idiosyncratic switching strategies dependent on both target rewards and, surprisingly, the feedback gains at perturbation onset: the more vigorous movements were less likely to switch to a new goal following perturbations. To gain further insight into a causal influence of the feedback gains on rapid motor decisions, we demonstrated that biasing the baseline activity and reflex gains by means of a background load evoked a larger proportion of target switches in the direction opposite to the background load associated with lower muscle activity. Together, our results demonstrate an impact of target reward on feedback control and highlight the competition between movement vigor and flexibility.

Mechanisms of Human Motor Learning Do Not Function Independently

Human motor learning is governed by a suite of interacting mechanisms each one of which modifies behavior in distinct ways and rely on different neural circuits. In recent years, much attention has been given to one type of motor learning, called motor adaptation. Here, the field has generally focused on the interactions of three mechanisms: sensory prediction error SPE-driven, explicit (strategy-based), and reinforcement learning. Studies of these mechanisms have largely treated them as modular, aiming to model how the outputs of each are combined in the production of overt behavior. However, when examined closely the results of some studies also suggest the existence of additional interactions between the sub-components of each learning mechanism. In this perspective, we propose that these sub-component interactions represent a critical means through which different motor learning mechanisms are combined to produce movement; understanding such interactions is critical to advancing our knowledge of how humans learn new behaviors. We review current literature studying interactions between SPE-driven, explicit, and reinforcement mechanisms of motor learning. We then present evidence of sub-component interactions between SPE-driven and reinforcement learning as well as between SPE-driven and explicit learning from studies of people with cerebellar degeneration. Finally, we discuss the implications of interactions between learning mechanism sub-components for future research in human motor learning.

Bayesian decision theory and navigation

Spatial navigation is a complex cognitive activity that depends on perception, action, memory, reasoning, and problem-solving. Effective navigation depends on the ability to combine information from multiple spatial cues to estimate one's position and the locations of goals. Spatial cues include landmarks, and other visible features of the environment, and body-based cues generated by self-motion (vestibular, proprioceptive, and efferent information). A number of projects have investigated the extent to which visual cues and body-based cues are combined optimally according to statistical principles. Possible limitations of these investigations are that they have not accounted for navigators' prior experiences with or assumptions about the task environment and have not tested complete decision models. We examine cue combination in spatial navigation from a Bayesian perspective and present the fundamental principles of Bayesian decision theory. We show that a complete Bayesian decision model with an explicit loss function can explain a discrepancy between optimal cue weights and empirical cues weights observed by (Chen et al. Cognitive Psychology, 95, 105-144, 2017) and that the use of informative priors to represent cue bias can explain the incongruity between heading variability and heading direction observed by (Zhao and Warren 2015b, Psychological Science, 26[6], 915-924). We also discuss (Petzschner and Glasauer's , Journal of Neuroscience, 31(47), 17220-17229, 2011) use of priors to explain biases in estimates of linear displacements during visual path integration. We conclude that Bayesian decision theory offers a productive theoretical framework for investigating human spatial navigation and believe that it will lead to a deeper understanding of navigational behaviors.

The road towards understanding embodied decisions

Most current decision-making research focuses on classical economic scenarios, where choice offers are prespecified and where action dynamics play no role in the decision. However, our brains evolved to deal with different choice situations: "embodied decisions". As examples of embodied decisions, consider a lion that has to decide which gazelle to chase in the savannah or a person who has to select the next stone to jump on when crossing a river. Embodied decision settings raise novel questions, such as how people select from time-varying choice options and how they track the most relevant choice attributes; but they have long remained challenging to study empirically. Here, we summarize recent progress in the study of embodied decisions in sports analytics and experimental psychology. Furthermore, we introduce a formal methodology to identify the relevant dimensions of embodied choices (present and future affordances) and to map them into the attributes of classical economic decisions (probabilities and utilities), hence aligning them. Studying embodied decisions will greatly expand our understanding of what decision-making is.

Heuristics contribute to sensorimotor decision-making under risk

Research in psychophysics argues that incentivized sensorimotor decisions (such as deciding where to reach to get a reward) maximize expected gain, suggesting that these decisions may be impervious to cognitive biases and heuristics. We tested this hypothesis in two experiments, directly comparing the predictive accuracy of an optimal model and plausible suboptimal models. We obtained strong evidence that people deviated from the optimal strategy by excessively avoiding loss regions when the potential loss was zero and failing to shift far enough away from loss regions when potential losses outweighed the potential gains. Although allowing nonlinear distortions of value and probability information improved the fit of value-maximizing models, behavior was best described by a model encapsulating a simple heuristic strategy. This suggests that visuomotor decisions are likely influenced by biases and heuristics observed in more classical economic decision-making tasks.

A Collision Avoidance Algorithm for Human Motion Prediction Based on Perceived Risk of Collision: Part 1-Model Development

OCCUPATIONAL APPLICATIONSDigital human models have been widely used in occupational biomechanics assessments to prevent potential injury risks, such as automotive assembly lines, box lifting, patient repositioning, and the mining industry. Motion prediction is one of the important capabilities in digital human models, and collision avoidance is involved in human motion prediction. We propose an algorithm that will ensure human motions are predicted realistically, and finally, use of this algorithm could help enhance the accuracy of injury risk assessments using digital human models.

A Collision Avoidance Algorithm for Human Motion Prediction Based on Perceived Risk of Collision: Part 2-Application

Occupational ApplicationDigital human models have been widely used for occupational assessments to reduce potential injury risk, such as automotive assembly lines, box lifting, and in the mining industry. Human motion prediction is one of the important capabilities in digital human models, and collision avoidance is involved in human motion prediction. An algorithm proposed earlier was implemented for human motion prediction, and simulated results were found to have a good correlation with the experimental studies. Use of this algorithm can help ensure that human motion is predicted realistically, and thus can impact the accuracy of injury risk assessments.

Validating model-based Bayesian integration using prior-cost metamers

There are two competing views on how humans make decisions under uncertainty. Bayesian decision theory posits that humans optimize their behavior by establishing and integrating internal models of past sensory experiences (priors) and decision outcomes (cost functions). An alternative hypothesis posits that decisions are optimized through trial and error without explicit internal models for priors and cost functions. To distinguish between these possibilities, we introduce a paradigm that probes the sensitivity of humans to transitions between prior-cost pairs that demand the same optimal policy (metamers) but distinct internal models. We demonstrate the utility of our approach in two experiments that were classically explained by Bayesian theory. Our approach validates the Bayesian learning strategy in an interval timing task but not in a visuomotor rotation task. More generally, our work provides a domain-general approach for testing the circumstances under which humans explicitly implement model-based Bayesian computations.

Affective Bias Through the Lens of Signal Detection Theory

Affective bias – a propensity to focus on negative information at the expense of positive information – is a core feature of many mental health problems. However, it can be caused by wide range of possible underlying cognitive mechanisms. Here we illustrate this by focusing on one particular behavioural signature of affective bias – increased tendency of anxious/depressed individuals to predict lower rewards – in the context of the Signal Detection Theory (SDT) modelling framework. Specifically, we show how to apply this framework to measure affective bias and compare it to the behaviour of an optimal observer. We also show how to extend the framework to make predictions about bias when the individual holds incorrect assumptions about the decision context. Building on this theoretical foundation, we propose five experiments to test five hypothetical sources of this affective bias: beliefs about prior probabilities, beliefs about performance, subjective value of reward, learning differences, and need for accuracy differences. We argue that greater precision about the mechanisms driving affective bias may eventually enable us to better understand the mechanisms underlying mood and anxiety disorders.

Why are the batteries in the microwave?: Use of semantic information under uncertainty in a search task

A major problem in human cognition is to understand how newly acquired information and long-standing beliefs about the environment combine to make decisions and plan behaviors. Over-dependence on long-standing beliefs may be a significant source of suboptimal decision-making in unusual circumstances. While the contribution of long-standing beliefs about the environment to search in real-world scenes is well-studied, less is known about how new evidence informs search decisions, and it is unclear whether the two sources of information are used together optimally to guide search. The present study expanded on the literature on semantic guidance in visual search by modeling a Bayesian ideal observer's use of long-standing semantic beliefs and recent experience in an active search task. The ability to adjust expectations to the task environment was simulated using the Bayesian ideal observer, and subjects' performance was compared to ideal observers that depended on prior knowledge and recent experience to varying degrees. Target locations were either congruent with scene semantics, incongruent with what would be expected from scene semantics, or random. Half of the subjects were able to learn to search for the target in incongruent locations over repeated experimental sessions when it was optimal to do so. These results suggest that searchers can learn to prioritize recent experience over knowledge of scenes in a near-optimal fashion when it is beneficial to do so, as long as the evidence from recent experience was learnable.

Competition Rather Than Observation and Cooperation Facilitates Optimal Motor Planning

Humans tend to select motor planning with a high reward and low success compared with motor planning, which has a small reward and high success rate. Previous studies have shown such a risk-seeking property in motor decision tasks. However, it is unclear how to facilitate a shift from risk-seeking to optimal motor planning that maximizes the expected reward. Here, we investigate the effect of interacting with virtual partners/opponents on motor plans since interpersonal interaction has a powerful influence on human perception, action, and cognition. This study compared three types of interactions (competition, cooperation, and observation) and two types of virtual partners/opponents (those engaged in optimal motor planning and those engaged in risk-averse motor planning). As reported in previous studies, the participants took a risky aim point when they performed a motor decision task alone. However, we found that the participant's aim point was significantly modulated when they performed the same task while competing with a risk-averse opponent (p = 0.018) and that there was no significant difference from the optimal aim point (p = 0.63). No significant modulation in the aim points was observed during the cooperation and observation tasks. These results highlight the importance of competition for modulating suboptimal decision-making and optimizing motor performance.

A description-experience gap in statistical intuitions: Of smart babies, risk-savvy chimps, intuitive statisticians, and stupid grown-ups

Comparison of different lines of research on statistical intuitions and probabilistic reasoning reveals several puzzling contradictions. Whereas babies seem to be intuitive statisticians, surprisingly capable of statistical learning and inference, adults' statistical inferences have been found to be inconsistent with the rules of probability theory and statistics. Whereas researchers in the 1960s concluded that people's probability updating is "conservatively" proportional to normative predictions, probability updating research in the 1970s suggested that people are incapable of following Bayes's rule. And whereas animals appear to be strikingly risk savvy, humans often seem "irrational" when dealing with probabilistic information. Drawing on research on the description-experience gap in risky choice, we integrate and systematize these findings from disparate fields of inquiry that have, to date, operated largely in parallel. Our synthesis shows that a key factor in understanding inconsistencies in statistical intuitions research is whether probabilistic inferences are based on symbolic, abstract descriptions or on the direct experience of statistical information. We delineate this view from other conceptual accounts, consider potential mechanisms by which attributes of first-hand experience can facilitate appropriate statistical inference, and identify conditions under which they improve or impair probabilistic reasoning. To capture the full scope of human statistical intuition, we conclude, research on probabilistic reasoning across the lifespan, across species, and across research traditions must bear in mind that experience and symbolic description of the world may engage systematically distinct cognitive processes.

Reaching for known unknowns: Rapid reach decisions accurately reflect the future state of dynamic probabilistic information

Everyday tasks such as catching a ball appear effortless, but in fact require complex interactions and tight temporal coordination between the brain's visual and motor systems. What makes such interceptive actions particularly impressive is the capacity of the brain to account for temporal delays in the central nervous system-a limitation that can be mitigated by making predictions about the environment as well as one's own actions. Here, we wanted to assess how well human participants can plan an upcoming movement based on a dynamic, predictable stimulus that is not the target of action. A central stationary or rotating stimulus determined the probability that each of two potential targets would be the eventual target of a rapid reach-to-touch movement. We examined the extent to which reach movement trajectories convey internal predictions about the future state of dynamic probabilistic information conveyed by the rotating stimulus. We show that movement trajectories reflect the target probabilities determined at movement onset, suggesting that humans rapidly and accurately integrate visuospatial predictions and estimates of their own reaction times to effectively guide action.

Perceptual uncertainty and action consequences independently affect hand movements in a virtual environment

When we use virtual and augmented reality (VR/AR) environments to investigate behaviour or train motor skills, we expect that the insights or skills acquired in VR/AR transfer to real-world settings. Motor behaviour is strongly influenced by perceptual uncertainty and the expected consequences of actions. VR/AR differ in both of these aspects from natural environments. Perceptual information in VR/AR is less reliable than in natural environments, and the knowledge of acting in a virtual environment might modulate our expectations of action consequences. Using mirror reflections to create a virtual environment free of perceptual artefacts, we show that hand movements in an obstacle avoidance task systematically differed between real and virtual obstacles and that these behavioural differences occurred independent of the quality of the available perceptual information. This suggests that even when perceptual correspondence between natural and virtual environments is achieved, action correspondence does not necessarily follow due to the disparity in the expected consequences of actions in the two environments.

Performance monitoring for sensorimotor confidence: A visuomotor tracking study

To best interact with the external world, humans are often required to consider the quality of their actions. Sometimes the environment furnishes rewards or punishments to signal action efficacy. However, when such feedback is absent or only partial, we must rely on internally generated signals to evaluate our performance (i.e., metacognition). Yet, very little is known about how humans form such judgements of sensorimotor confidence. Do they monitor their actual performance or do they rely on cues to sensorimotor uncertainty? We investigated sensorimotor metacognition in two visuomotor tracking experiments, where participants followed an unpredictably moving dot cloud with a mouse cursor as it followed a random horizontal trajectory. Their goal was to infer the underlying target generating the dots, track it for several seconds, and then report their confidence in their tracking as better or worse than their average. In Experiment 1, we manipulated task difficulty with two methods: varying the size of the dot cloud and varying the stability of the target's velocity. In Experiment 2, the stimulus statistics were fixed and duration of the stimulus presentation was varied. We found similar levels of metacognitive sensitivity in all experiments, which was evidence against the cue-based strategy. The temporal analysis of metacognitive sensitivity revealed a recency effect, where error later in the trial had a greater influence on the sensorimotor confidence, consistent with a performance-monitoring strategy. From these results, we conclude that humans predominantly monitored their tracking performance, albeit inefficiently, to build a sense of sensorimotor confidence.

Visual-reward driven changes of movement during action execution

Motor decision-making is often described as a sequential process, beginning with the assessment of available options and leading to the execution of a selected movement. While this view is likely to be accurate for decisions requiring significant deliberation, it would seem unfit for choices between movements in dynamic environments. In this study, we examined whether and how non-selected motor options may be considered post-movement onset. We hypothesized that a change in reward at any point in time implies a dynamic reassessment of options, even after an initial decision has been made. To test this, we performed a decision-making task in which human participants were instructed to execute a reaching movement from an origin to a rectangular target to attain a reward. Reward depended on arrival precision and on the specific distribution of reward presented along the target. On a third of trials, we changed the initial reward distribution post-movement onset. Our results indicated that participants frequently change their initially selected movements when a change is associated with an increase in reward. This process occurs quicker than overall, average reaction times. Finally, changes in movement are not only dependent on reward but also on the current state of the motor apparatus.

The bounded rationality of probability distortion

In decision making under risk (DMR) participants' choices are based on probability values systematically different from those that are objectively correct. Similar systematic distortions are found in tasks involving relative frequency judgments (JRF). These distortions limit performance in a wide variety of tasks and an evident question is, Why do we systematically fail in our use of probability and relative frequency information? We propose a bounded log-odds model (BLO) of probability and relative frequency distortion based on three assumptions: 1) log-odds: probability and relative frequency are mapped to an internal log-odds scale, 2) boundedness: the range of representations of probability and relative frequency are bounded and the bounds change dynamically with task, and 3) variance compensation: the mapping compensates in part for uncertainty in probability and relative frequency values. We compared human performance in both DMR and JRF tasks to the predictions of the BLO model as well as 11 alternative models, each missing one or more of the underlying BLO assumptions (factorial model comparison). The BLO model and its assumptions proved to be superior to any of the alternatives. In a separate analysis, we found that BLO accounts for individual participants' data better than any previous model in the DMR literature. We also found that, subject to the boundedness limitation, participants' choice of distortion approximately maximized the mutual information between objective task-relevant values and internal values, a form of bounded rationality.

Statistical decision theory and multiscale analyses of human brain data

BACKGROUND

In the era of Big Data, large scale electrophysiological data from animal and human studies are abundant. These data contain information at multiple spatiotemporal scales. However, current approaches for the analysis of electrophysiological data often focus on a single spatiotemporal scale only.

NEW METHOD

We discuss a multiscale approach for the analysis of electrophysiological data. This is based on combining neural models that describe brain data at different scales. It allows us to make laminar-specific inferences about neurobiological properties of cortical sources using non invasive human electrophysiology data.

RESULTS

We provide a mathematical proof of this approach using statistical decision theory. We also consider its extensions to brain imaging studies including data from the same subjects performing different tasks. As an illustration, we show that changes in gamma oscillations between different people might originate from differences in recurrent connection strengths of inhibitory interneurons in layers 5/6.

COMPARISON WITH EXISTING METHODS

This is a new approach that follows up on our recent work. It is different from other approaches where the scale of spatiotemporal dynamics is fixed.

CONCLUSIONS

We discuss a multiscale approach for the analysis of human MEG data. This uses a neural mass model that includes constraints informed by a compartmental model. This has two advantages. First, it allows us to find differences in cortical laminar dynamics and understand neurobiological properties like neuromodulation, excitation to inhibition balance etc. using non invasive data. Second, it allows us to validate macroscale models by exploiting animal data.

Choices in a key press decision-making task are more optimal after gaining both aiming and reward experience

When presented with two different target-penalty configurations of similar maximum expected gain (MEG), participants prefer aiming to configurations with more advantageous spatial, rather than more advantageous gain parameters-perhaps due to the motor system's inherent prioritisation of spatial information during movements with high accuracy demands such as aiming. To test this hypothesis, participants in the present studies chose between target-penalty configurations via key presses to reduce the importance of spatial parameters of the response and performance-related feedback. Configurations varied in spatial (target-penalty region overlap) and gain parameters (negative penalty values) and could have similar or different MEG. Choices were made without prior aiming experience (Experiment 1), after aiming experience provided information of movement variability (Experiment 2), or after aiming experience provided information of movement variability and outcome feedback (Experiment 3). Overall, configurations with advantageous spatial or gain parameters were chosen equally (Both-Similar condition) in all experiments. However, average behaviour at the group level was not reflective of the behaviour of most individual participants with three subgroups emerging: those with a value preference, distance preference, or no preference. In Experiments 1 and 2, these individual differences cannot be explained by MEG differences between configurations or participants' movement variability, but these variables predicted choice behaviour in Experiment 3. Further in the Both-Different condition, participants only selected the larger MEG configuration at a level above chance when both variability and outcome information were given prior to the key press task (Experiment 3). In sum, the data indicate that prioritisation of spatial information did not emerge at the group level when performing key presses and more optimal behaviour emerged when information regarding movement variability and outcome feedback were given.

Competition between salience and informational value for saccade adaptation

What we see is influenced by where we look. When confronted with multiple relevant targets, inaccurate saccade target selection can impair perceptual performance. Here we ask whether endpoint selection can be optimized by the mechanism maintaining saccade accuracy: saccade adaptation. Therefore, we introduce a double-target adaptation task, where a presaccadic peripheral stimulus (plaid) splits vertically into its two components (Gabor patches) during horizontal saccades. While both targets were task-relevant, one of them provided more information for the perceptual task, because it could only be identified after the saccade with near-foveal vision. The other target was highly salient and could also be identified in the presaccadic plaid using peripheral vision. This double-target paradigm induced saccade adaptation: Without a perceptual task, participants adapted to the salient target. When both targets were judged sequentially, participants mostly adapted to the target they had to judge first. When targets were judged simultaneously, endpoints were biased toward the informative target but showed no gradual learning and fell short of optimality. We observed gradual adaptation when targets shifted randomly such that a strategic adjustment of endpoints was not possible. Overall, these findings show that when multiple targets compete, our oculomotor system can learn to adjust endpoints in order to maximize information for perception. Yet individual variability and other factors affecting target priority play a crucial role.

The Difficulty of Effectively Using Allocentric Prior Information in a Spatial Recall Task

Prior information represents the long-term statistical structure of an environment. For example, colds develop more often than throat cancer, making the former a more likely diagnosis for a sore throat. There is ample evidence for effective use of prior information during a variety of perceptual tasks, including the ability to recall locations using an egocentric (self-based) frame. However, it is not yet known if people can use prior information effectively when using an allocentric (world-based) frame. Forty-eight adults were shown sixty sets of three target locations in a sparse virtual environment with three beacons. The targets were drawn from one of four prior distributions. They were then asked to point to the targets after a delay and a change in perspective. While searches were biased towards the beacons, we did not find any evidence that participants successfully exploited the prior distributions of targets. These results suggest that allocentric reasoning does not conform to normative Bayesian models: we saw no evidence for use of priors in our cognitively-complex (allocentric) task, unlike in previous, simpler (egocentric) recall tasks. It is possible that this reflects the high biological cost of processing precise allocentric information.

Humans incorporate trial-to-trial working memory uncertainty into rewarded decisions

Information stored in working memory (WM) is incorporated into many daily decisions and actions, and many complex decisions involve WM; however, there has been little work on investigating what WM information is used in memory decisions. Here we try to draw connections between WM and decision making by manipulating prior beliefs in a standard WM task with rewards. We use this paradigm to show that WM contains a representation of the trial-by-trial uncertainty of visual stimuli. This uncertainty is incorporated into rewarded decisions along with other information, such as expectations about the environment. By studying WM in parallel with decision making, we can gain new insight into how these systems work together.

Working memory (WM) plays an important role in action planning and decision making; however, both the informational content of memory and how that information is used in decisions remain poorly understood. To investigate this, we used a color WM task in which subjects viewed colored stimuli and reported both an estimate of a stimulus color and a measure of memory uncertainty, obtained through a rewarded decision. Reported memory uncertainty is correlated with memory error, showing that people incorporate their trial-to-trial memory quality into rewarded decisions. Moreover, memory uncertainty can be combined with other sources of information; after inducing expectations (prior beliefs) about stimuli probabilities, we found that estimates became shifted toward expected colors, with the shift increasing with reported uncertainty. The data are best fit by models in which people incorporate their trial-to-trial memory uncertainty with potential rewards and prior beliefs. Our results suggest that WM represents uncertainty information, and that this can be combined with prior beliefs. This highlights the potential complexity of WM representations and shows that rewarded decision can be a powerful tool for examining WM and informing and constraining theoretical, computational, and neurobiological models of memory.

Human decision making anticipates future performance in motor learning

It is well-established that people can factor into account the distribution of their errors in motor performance so as to optimize reward. Here we asked whether, in the context of motor learning where errors decrease across trials, people take into account their future, improved performance so as to make optimal decisions to maximize reward. One group of participants performed a virtual throwing task in which, periodically, they were given the opportunity to select from a set of smaller targets of increasing value. A second group of participants performed a reaching task under a visuomotor rotation in which, after performing a initial set of trials, they selected a reward structure (ratio of points for target hits and misses) for different exploitation horizons (i.e., numbers of trials they might be asked to perform). Because movement errors decreased exponentially across trials in both learning tasks, optimal target selection (task 1) and optimal reward structure selection (task 2) required taking into account future performance. The results from both tasks indicate that people anticipate their future motor performance so as to make decisions that will improve their expected future reward.

Optimizing motor decision-making through competition with opponents

Although optimal decision-making is essential for sports performance and fine motor control, it has been repeatedly confirmed that humans show a strong risk-seeking bias, selecting a risky strategy over an optimal solution. Despite such evidence, the ideal method to promote optimal decision-making remains unclear. Here, we propose that interactions with other people can influence motor decision-making and improve risk-seeking bias. We developed a competitive reaching game (a variant of the “chicken game”) in which aiming for greater rewards increased the risk of no reward and subjects competed for the total reward with their opponent. The game resembles situations in sports, such as a penalty kick in soccer, service in tennis, the strike zone in baseball, or take-off in ski jumping. In five different experiments, we demonstrated that, at the beginning of the competitive game, the subjects robustly switched their risk-seeking strategy to a risk-averse strategy. Following the reversal of the strategy, the subjects achieved optimal decision-making when competing with risk-averse opponents. This optimality was achieved by a non-linear influence of an opponent’s decisions on a subject’s decisions. These results suggest that interactions with others can alter human motor decision strategies and that competition with a risk-averse opponent is key for optimizing motor decision-making.

Tuning of Standing Postural Responses to Instability and Cost Function

Whole-body movements are performed daily, and humans must constantly take into account the inherent instability of a standing posture. At times these movements may be performed in risky environments and when facing different costs of failure. The aim of the study was to test the hypothesis that in upright stance participants continuously estimate both probability of failure and cost of failure such that their postural responses will be based on these estimates. We designed a snowboard riding simulation experiment where participants were asked to control the position of a moving snowboard within a snow track in a risky environment. Cost functions were provided by modifying the penalty of riding in the area adjacent to the snow track. Uncertainty was modified by changing the gain of postural responses while participants were standing on a rocker board. We demonstrated that participants continually evaluated the environmental cost function and compensated for additional risk with feedback-based postural changes, even when probability of failure was negligible. Results showed also that the participants' estimates of the probability of failure accounted for their own inherent instability. Moreover, participants showed a tendency to overweight large probabilities of failure with more biomechanically constrained standing postures that results in suboptimal estimates of risky environments. Overall, our results suggest that participants tune their standing postural responses by empirically estimating the cost of failure and the uncertainty level in order to minimize the risk of falling when cost is high.

Transcranial Direct Current Stimulation Over Dorsolateral Prefrontal Cortex Modulates Risk-Attitude in Motor Decision-Making

Humans often face situations requiring a decision about where to throw an object or when to respond to a stimulus under risk. Several behavioral studies have shown that such motor decisions can be suboptimal, which results from a cognitive bias toward risk-seeking behavior. However, brain regions involved in risk-attitude of motor decision-making remain unclear. Here, we investigated the role of the dorsolateral prefrontal cortex (DLPFC) in risky motor decisions using transcranial direct current stimulation (tDCS). The experiment comprised a selective timing task requiring participants to make a continuous decision about the timing of their response under the risk of no rewards. The participants performed this task twice in a day: before and while receiving either anodal stimulation over the right DLPFC with cathodal stimulation over the left DLPFC (20 min, 2 mA), cathodal stimulation over the right DLPFC with anodal stimulation over the left DLPFC, or sham stimulation. In line with previous studies, their strategies before the stimulation were biased toward risk-seeking. During anodal stimulation over right DLPFC with cathodal stimulation over left DLPFC, participants showed a more conservative strategy to avoid the risk of no rewards. The additional experiment confirmed that tDCS did not affect the ability of timing control regarding the time intervals at which they aimed to respond. These results suggest a potential role for the DLPFC in modulating action selection in motor decision-making under risk.

Active Haptic Perception in Robots: A Review

In the past few years a new scenario for robot-based applications has emerged. Service and mobile robots have opened new market niches. Also, new frameworks for shop-floor robot applications have been developed. In all these contexts, robots are requested to perform tasks within open-ended conditions, possibly dynamically varying. These new requirements ask also for a change of paradigm in the design of robots: on-line and safe feedback motion control becomes the core of modern robot systems. Future robots will learn autonomously, interact safely and possess qualities like self-maintenance. Attaining these features would have been relatively easy if a complete model of the environment was available, and if the robot actuators could execute motion commands perfectly relative to this model. Unfortunately, a complete world model is not available and robots have to plan and execute the tasks in the presence of environmental uncertainties which makes sensing an important component of new generation robots. For this reason, today's new generation robots are equipped with more and more sensing components, and consequently they are ready to actively deal with the high complexity of the real world. Complex sensorimotor tasks such as exploration require coordination between the motor system and the sensory feedback. For robot control purposes, sensory feedback should be adequately organized in terms of relevant features and the associated data representation. In this paper, we propose an overall functional picture linking sensing to action in closed-loop sensorimotor control of robots for touch (hands, fingers). Basic qualities of haptic perception in humans inspire the models and categories comprising the proposed classification. The objective is to provide a reasoned, principled perspective on the connections between different taxonomies used in the Robotics and human haptic literature. The specific case of active exploration is chosen to ground interesting use cases. Two reasons motivate this choice. First, in the literature on haptics, exploration has been treated only to a limited extent compared to grasping and manipulation. Second, exploration involves specific robot behaviors that exploit distributed and heterogeneous sensory data.

Object interaction and walking: Integration of old and new skills in infant development

Manual skills such as reaching, grasping, and exploring objects appear months earlier in infancy than locomotor skills such as walking. To what extent do infants incorporate an old skill (manual actions on objects) into the development of a new skill (walking)? We video recorded 64 sessions of infants during free play in a laboratory playroom. Infants' age (12.7-19.5 months), walking experience (0.5-10.3 months), and walking proficiency (speed, step length, etc.) varied widely. We found that the earlier developing skills of holding and exploring objects are immediately incorporated into the later developing skill of walking. Although holding incurred a reliable cost to infants' gait patterns, holding and exploring objects in hand were relatively common activities, and did not change with development. Moreover, holding objects was equally common in standing and walking. However, infants did not interact with objects indiscriminately: Object exploration was more frequent while standing than walking, and infants selectively chose lighter objects to carry and explore. Findings suggest that the earlier appearance of some skills may serve to motivate and enrich later appearing skills.

A Model to Estimate the Optimal Layout for Assistive Communication Touchscreen Devices in Children With Dyskinetic Cerebral Palsy

Excess involuntary movements and slowness of movement in children with dyskinetic cerebral palsy often result in the inability to properly interact with augmentative and alternative communication (AAC) devices. This significantly limits communication. It is, therefore, essential to know how to adjust the device layout in order to maximize each child's rate of communication. The aim of this paper was to develop a mathematical model to estimate the information rate in children with dyskinetic cerebral palsy and to determine the optimal AAC layout for a touchscreen tablet that results in enhanced speed of communication. The model predicts information rate based on button size, number, spacing between buttons, and the probability of making an error or missing target buttons. Estimation of the information rate confirmed our hypothesis of lower channel capacity in children with dyskinetic cerebral palsy compared with age-matched healthy children. Information rate increased when the AAC layout was customized based on the optimal parameters predicted by the model. In conclusion, this paper quantifies the effect of motor impairments on communication with assistive communication devices and shows that communication performance can be improved by optimally matching the parameters of the AAC touchscreen device to the abilities of the child.

Rapid Visuomotor Responses Reflect Value-Based Decisions

Cognitive decision-making is known to be sensitive to the values of potential options, which are the probability and size of rewards associated with different choices. Here, we examine whether rapid motor responses to perturbations of visual feedback about movement, which mediate low-level and involuntary feedback control loops, reflect computations associated with high-level value-based decision-making. In three experiments involving human participants, we varied the value associated with different potential targets for reaching movements by controlling the distributions of rewards across the targets (Experiment 1), the probability with which each target could be specified (Experiment 2), or both (Experiment 3). We found that the size of rapid and involuntary feedback responses to movement perturbations was strongly influenced by the relative value between targets. A statistical model of relative value that includes a term for risk sensitivity provided the best fit to the visuomotor response data, illustrating that feedback control policies are biased to favor more frequent task success at the expense of the overall extrinsic reward accumulated through movement. Importantly however, the regulation of rapid feedback responses was associated with successful pursuit of high-value task outcomes. This implies that when we move, the brain specifies a set of feedback control gains that enable low-level motor areas not only to generate efficient and accurate movement, but also to rapidly and adaptively respond to evolving sensory information in a manner consistent with value-based decision-making.SIGNIFICANCE STATEMENT Current theories of sensorimotor control suggest that, rather than selecting and planning the details of movements in advance, the role of the brain is to set time-varying feedback gains that continuously transform sensory information into motor commands by feedback control. Here, we examine whether the fastest motor responses to perturbations of movement, which mediate low-level and involuntary feedback control loops (i.e., reflexes), reflect computations associated with high-level, value-based decision-making. We find that rapid feedback responses during reaching reflect the relative probabilities and rewards associated with target options. This suggests that low-order components of the sensorimotor control hierarchy, which generate rapid and automatic responses, can continuously evaluate evolving sensory evidence and initiate responses according to the prospect of reward.

Human representation of multimodal distributions as clusters of samples

Behavioral and neuroimaging evidence shows that human decisions are sensitive to the statistical regularities (mean, variance, skewness, etc.) of reward distributions. However, it is unclear what representations human observers form to approximate reward distributions, or probability distributions in general. When the possible values of a probability distribution are numerous, it is cognitively costly and perhaps unrealistic to maintain in mind the probability of each possible value. Here we propose a Clusters of Samples (CoS) representation model: The samples of the to-be-represented distribution are classified into a small number of clusters and only the centroids and relative weights of the clusters are retained for future use. We tested the behavioral relevance of CoS in four experiments. On each trial, human subjects reported the mean and mode of a sequentially presented multimodal distribution of spatial positions or orientations. By varying the global and local features of the distributions, we observed systematic errors in the reported mean and mode. We found that our CoS representation of probability distributions outperformed alternative models in accounting for subjects’ response patterns. The ostensible influence of positive/negative skewness on the over/under estimation of the reported mean, analogous to the “skewness preference” phenomenon in decisions, could be well explained by models based on CoS.

Life is full of uncertainties: An action may yield multiple possible consequences and a percept may imply multiple possible causes. To survive, humans and animals must compensate for the uncertainty in the environment and in their own perceptual and motor systems. However, how humans represent probability distributions to fulfill probabilistic computations for perception and action remains elusive. The number of possible values in a distribution is vast and grows exponentially with the dimension of the distribution. It would be costly, if not impossible, to maintain the probability of each possible value. Here we propose a sparse representation of probability distributions, which can reduce an arbitrary distribution to a small set of coefficients while still keeping important global and local features of the original distribution. Our experiments provide preliminary evidence for the use of such representations in human cognition.

Internal Models in Biological Control

Rationality principles such as optimal feedback control and Bayesian inference underpin a probabilistic framework that has accounted for a range of empirical phenomena in biological sensorimotor control. To facilitate the optimization of flexible and robust behaviors consistent with these theories, the ability to construct internal models of the motor system and environmental dynamics can be crucial. In the context of this theoretic formalism, we review the computational roles played by such internal models and the neural and behavioral evidence for their implementation in the brain.