The road towards understanding embodied decisions

Toward a neuroscience of natural behavior

One of the most exciting new developments in systems neuroscience is the progress being made toward neurophysiological experiments that move beyond simplified laboratory settings and address the richness of natural behavior. This is enabled by technological advances such as wireless recording in freely moving animals, automated quantification of behavior, and new methods for analyzing large data sets. Beyond new empirical methods and data, however, there is also a need for new theories and concepts to interpret that data. Such theories need to address the particular challenges of natural behavior, which often differ significantly from the scenarios studied in traditional laboratory settings. Here, we discuss some strategies for developing such novel theories and concepts and some example hypotheses being proposed.

Graded decisions in the human brain

Decision-makers objectively commit to a definitive choice, yet at the subjective level, human decisions appear to be associated with a degree of uncertainty. Whether decisions are definitive (i.e., concluding in all-or-none choices), or whether the underlying representations are graded, remains unclear. To answer this question, we recorded intracranial neural signals directly from the brain while human subjects made perceptual decisions. The recordings revealed that broadband gamma activity reflecting each individual's decision-making process, ramped up gradually while being graded by the accumulated decision evidence. Crucially, this grading effect persisted throughout the decision process without ever reaching a definite bound at the time of choice. This effect was most prominent in the parietal cortex, a brain region traditionally implicated in decision-making. These results provide neural evidence for a graded decision process in humans and an analog framework for flexible choice behavior.

Influence of age and cognitive demand on motor decision making under uncertainty: a study on goal directed reaching movements

In everyday life, we constantly make decisions about actions to be performed subsequently. Research on motor decision making has provided empirical evidence for an influence of decision uncertainty on movement execution in young adults. Further, decision uncertainty was suggested to be increased in older adults due to limited cognitive resources for the integration of information and the prediction of the decision outcomes. However, the influence of cognitive aging on decision uncertainty during motor decision making and movement execution has not been investigated, yet. Thus, in the current study, we presented young and older adults with a motor decision making task, in which participants had to decide on pointing towards one out of five potential targets under varying cognitive demands. Statistical analyses revealed stronger decreases in correctly deciding upon the pointing target, i.e. task performance, from low to higher cognitive demand in older as compared to young adults. Decision confidence also decreased more strongly in older adults with increasing cognitive demand, however, only when collapsing across correct and incorrect decision trials, but not when considering correct decision trials, only. Further, older adults executed reaching movements with longer reaction times and increased path length, though the latter, again, not when considering correct decision trials, only. Last, reaction time and variability in movement execution were both affected by cognitive demand. The outcomes of this study provide a differentiated picture of the distinct and joint effects of aging and cognitive demand during motor decision making.

Dual-tasking modulates movement speed but not value-based choices during walking

Value-based decision-making often occurs in multitasking scenarios relying on both cognitive and motor processes. Yet, laboratory experiments often isolate these processes, thereby neglecting potential interactions. This isolated approach reveals a dichotomy: the cognitive process by which reward influences decision-making is capacity-limited, whereas the influence of motor cost is free of such constraints. If true, dual-tasking should predominantly impair reward processing but not affect the impact of motor costs. To test this hypothesis, we designed a decision-making task in which participants made choices to walk toward targets for rewards while navigating past an obstacle. The motor cost to reach these rewards varied in real-time. Participants either solely performed the decision-making task, or additionally performed a secondary pitch-recall task. Results revealed that while both reward and motor costs influenced decision-making, the secondary task did not affect these factors. Instead, dual-tasking slowed down participants' walking, thereby reducing the overall reward rate. Hence, contrary to the prediction that the added cognitive demand would affect the weighing of reward or motor cost differentially, these processes seem to be maintained at the expense of slowing down the motor system. This slowdown may be indicative of interference at the locomotor level, thereby underpinning motor-cognitive interactions during decision-making.

How Healthy Older Adults Enact Lateral Maneuvers While Walking

BACKGROUND

Walking requires frequent maneuvers to navigate changing environments with shifting goals. Humans accomplish maneuvers and simultaneously maintain balance primarily by modulating their foot placement, but a direct trade-off between these two objectives has been proposed. As older adults may rely more on foot placement to maintain lateral balance, they may be less able to adequately adapt stepping to perform lateral maneuvers.

RESEARCH QUESTION

How do older adults adapt stepping to enact lateral lane-change maneuvers, and how do physical and perceived ability influence their task performance?

METHODS

Twenty young (21.7 ± 2.6 yrs) and 18 older (71.6 ± 6.0 yrs) adults walked on a motorized treadmill in a virtual environment. Following an audible and visual cue, participants switched between two parallel paths, centered 0.6 m apart, to continue walking on their new path. We quantified when participants initiated the maneuver following the cue, as well as their step width, lateral position, and stepping variability ellipses at each maneuver step.

RESULTS

Young and older adults did not differ in when they initiated the maneuver, but participants with lower perceived ability took longer to do so. Young and older adults also did not exhibit differences in step width or lateral positions at any maneuver step, but participants with greater physical ability reached their new path faster. While only older adults exhibited stepping adaptations prior to initiating the maneuver, both groups traded off stability for maneuverability to enact the lateral maneuver.

SIGNIFICANCE

Physical and perceived balance ability, rather than age per se, differentially influenced maneuver task performance. Humans must make decisions related to the task of walking itself and do so based on both physical and perceived factors. Understanding and targeting these interactions may help improve walking performance among older adults.

Rapid, systematic updating of movement by accumulated decision evidence

Acting in the natural world requires not only deciding among multiple options but also converting decisions into motor commands. How the dynamics of decision formation influence the fine kinematics of response movement remains, however, poorly understood. Here we investigate how the accumulation of decision evidence shapes the response orienting trajectories in a task where freely-moving rats combine prior expectations and auditory information to select between two possible options. Response trajectories and their motor vigor are initially determined by the prior. Rats movements then incorporate sensory information as early as 60 ms after stimulus onset by accelerating or slowing depending on how much the stimulus supports their initial choice. When the stimulus evidence is in strong contradiction, rats change their mind and reverse their initial trajectory. Human subjects performing an equivalent task display a remarkably similar behavior. We encapsulate these results in a computational model that, by mapping the decision variable onto the movement kinematics at discrete time points, captures subjects' choices, trajectories and changes of mind. Our results show that motor responses are not ballistic. Instead, they are systematically and rapidly updated, as they smoothly unfold over time, by the parallel dynamics of the underlying decision process.

Generalizing stepping concepts to non-straight walking

People rarely walk in straight lines. Instead, we make frequent turns or other maneuvers. Spatiotemporal parameters fundamentally characterize gait. For straight walking, these parameters are well-defined for the task of walking on a straight path. Generalizing these concepts to non-straight walking, however, is not straightforward. People follow non-straight paths imposed by their environment (sidewalk, windy hiking trail, etc.) or choose readily-predictable, stereotypical paths of their own. People actively maintain lateral position to stay on their path and readily adapt their stepping when their path changes. We therefore propose a conceptually coherent convention that defines step lengths and widths relative to predefined walking paths. Our convention simply re-aligns lab-based coordinates to be tangent to a walker's path at the mid-point between the two footsteps that define each step. We hypothesized this would yield results both more correct and more consistent with notions from straight walking. We defined several common non-straight walking tasks: single turns, lateral lane changes, walking on circular paths, and walking on arbitrary curvilinear paths. For each, we simulated idealized step sequences denoting "perfect" performance with known constant step lengths and widths. We compared results to path-independent alternatives. For each, we directly quantified accuracy relative to known true values. Results strongly confirmed our hypothesis. Our convention returned vastly smaller errors and introduced no artificial stepping asymmetries across all tasks. All results for our convention rationally generalized concepts from straight walking. Taking walking paths explicitly into account as important task goals themselves thus resolves conceptual ambiguities of prior approaches.

Sex differences in patch-leaving foraging decisions in rats

The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.

Sex differences in patch-leaving foraging decisions in rats

The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.

Generalizing Stepping Concepts To Non-Straight Walking

People rarely walk in straight lines. Instead, we make frequent turns or other maneuvers. Spatiotemporal parameters fundamentally characterize gait. For straight walking, these parameters are well-defined for that task of walking on a straight path. Generalizing these concepts to non-straight walking, however, is not straightforward. People also follow non-straight paths imposed by their environment (store aisle, sidewalk, etc.) or choose readily-predictable, stereotypical paths of their own. People actively maintain lateral position to stay on their path and readily adapt their stepping when their path changes. We therefore propose a conceptually coherent convention that defines step lengths and widths relative to known walking paths. Our convention simply re-aligns lab-based coordinates to be tangent to a walker's path at the mid-point between the two footsteps that define each step. We hypothesized this would yield results both more correct and more consistent with notions from straight walking. We defined several common non-straight walking tasks: single turns, lateral lane changes, walking on circular paths, and walking on arbitrary curvilinear paths. For each, we simulated idealized step sequences denoting "perfect" performance with known constant step lengths and widths. We compared results to path- independent alternatives. For each, we directly quantified accuracy relative to known true values. Results strongly confirmed our hypothesis. Our convention returned vastly smaller errors and introduced no artificial stepping asymmetries across all tasks. All results for our convention rationally generalized concepts from straight walking. Taking walking paths explicitly into account as important task goals themselves thus resolves conceptual ambiguities of prior approaches.

Beyond simple laboratory studies: Developing sophisticated models to study rich behavior

Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.

Kinematic priming of action predictions

The ability to anticipate what others will do next is crucial for navigating social, interactive environments. Here, we develop an experimental and analytical framework to measure the implicit readout of prospective intention information from movement kinematics. Using a primed action categorization task, we first demonstrate implicit access to intention information by establishing a novel form of priming, which we term kinematic priming: subtle differences in movement kinematics prime action prediction. Next, using data collected from the same participants in a forced-choice intention discrimination task 1 h later, we quantify single-trial intention readout-the amount of intention information read by individual perceivers in individual kinematic primes-and assess whether it can be used to predict the amount of kinematic priming. We demonstrate that the amount of kinematic priming, as indexed by both response times (RTs) and initial fixations to a given probe, is directly proportional to the amount of intention information read by the individual perceiver at the single-trial level. These results demonstrate that human perceivers have rapid, implicit access to intention information encoded in movement kinematics and highlight the potential of our approach to reveal the computations that permit the readout of this information with single-subject, single-trial resolution.

Evaluation of abstract rule-based associations in the human premotor cortex during passive observation

Decision-making often manifests in behavior, typically yielding overt motor actions. This complex process requires the registration of sensory information with one's internal representation of the current context, before a categorical judgment of the most appropriate motor behavior can be issued. The construct concept of embodied decision-making encapsulates this sequence of complex processes, whereby behaviorally salient information from the environment is represented in an abstracted space of potential motor actions rather than only in an abstract cognitive "decision" space. Theoretical foundations and some empirical evidence account for support the involvement of premotor cortical circuits in embodied cognitive functions. Animal models show that premotor circuits participate in the registration and evaluation of actions performed by peers in social situations, that is, prior to controlling one's voluntary movements guided by arbitrary stimulus-response rules. However, such evidence from human data is currently limited. Here we used time-resolved magnetoencephalography imaging to characterize activations of the premotor cortex as human participants observed arbitrary, non-biological visual stimuli that either respected or violated a simple stimulus-response association rule. The participants had learned this rule previously, either actively, by performing a motor task (active learning), or passively, by observing a computer perform the same task (passive learning). We discovered that the human premotor cortex is activated during the passive observation of the correct execution of a sequence of events according to a rule learned previously. Premotor activation also differs when the subjects observe incorrect stimulus sequences. These premotor effects are present even when the observed events are of a non-motor, abstract nature, and even when the stimulus-response association rule was learned via passive observations of a computer agent performing the task, without requiring overt motor actions from the human participant. We found evidence of these phenomena by tracking cortical beta-band signaling in temporal alignment with the observation of task events and behavior. We conclude that premotor cortical circuits that are typically engaged during voluntary motor behavior are also involved in the interpretation of events of a non-ecological, unfamiliar nature but related to a learned abstract rule. As such, the present study provides the first evidence of neurophysiological processes of embodied decision-making in human premotor circuits when the observed events do not involve motor actions of a third party.

Embodied choices bypass narratives under radical uncertainty

Johnson et al. suggest that we rely on narratives to make choices under radical uncertainty. We argue that in its current version Conviction Narrative Theory (CNT) does not account for embodied, direct sensorimotor influences on choices under radical uncertainty that may bypass narratives, particularly in highly time-constrained situations. We therefore suggest to extend CNT by an embodied choice perspective.

Decision heuristics in contexts integrating action selection and execution

Heuristics can inform human decision making in complex environments through a reduction of computational requirements (accuracy-resource trade-off) and a robustness to overparameterisation (less-is-more). However, tasks capturing the efficiency of heuristics typically ignore action proficiency in determining rewards. The requisite movement parameterisation in sensorimotor control questions whether heuristics preserve efficiency when actions are nontrivial. We developed a novel action selection-execution task requiring joint optimisation of action selection and spatio-temporal skillful execution. State-appropriate choices could be determined by a simple spatial heuristic, or by more complex planning. Computational models of action selection parsimoniously distinguished human participants who adopted the heuristic from those using a more complex planning strategy. Broader comparative analyses then revealed that participants using the heuristic showed combined decisional (selection) and skill (execution) advantages, consistent with a less-is-more framework. In addition, the skill advantage of the heuristic group was predominantly in the core spatial features that also shaped their decision policy, evidence that the dimensions of information guiding action selection might be yoked to salient features in skill learning.

Embodied decision biases: individually stable across different tasks?

In everyday life, action and decision-making often run in parallel. Action-based models argue that action and decision-making strongly interact and, more specifically, that action can bias decision-making. This embodied decision bias is thought to originate from changes in motor costs and/or cognitive crosstalk. Recent research confirmed embodied decision biases for different tasks including walking and manual movements. Yet, whether such biases generalize within individuals across different tasks remains to be determined. To test this, we used two different decision-making tasks that have independently been shown to reliably produce embodied decision biases. In a within-participant design, participants performed two tasks in a counterbalanced fashion: (i) a walking paradigm for which it is known that motor costs systematically influence reward decisions, and (ii) a manual movement task in which motor costs and cognitive crosstalk have been shown to impact reward decisions. In both tasks, we successfully replicated the predicted embodied decision biases. However, there was no evidence that the strength of the biases correlated between tasks. Hence, our findings do not confirm that embodied decision biases transfer between tasks. Future research is needed to examine whether this lack of transfer may be due to different causes underlying the impact of motor costs on decisions and the impact of cognitive crosstalk or task-specific differences.

How Healthy Older Adults Enact Lateral Maneuvers While Walking

Background

Walking requires frequent maneuvers to navigate changing environments with shifting goals. Humans accomplish maneuvers and simultaneously maintain balance primarily by modulating their foot placement, but a direct trade-off between these two objectives has been proposed. As older adults rely more on foot placement to maintain lateral balance, they may be less able to adequately adapt stepping to perform lateral maneuvers.

Research Question

How do older adults adapt stepping to enact lateral lane-change maneuvers, and how do physical and perceived ability influence their task performance?

Methods

Twenty young (21.7 ± 2.6 yrs) and 18 older (71.6 ± 6.0 yrs) adults walked on a motorized treadmill in a virtual environment. Following an audible and visual cue, participants switched between two parallel paths, centered 0.6m apart, to continue walking on their new path. We quantified when participants initiated the maneuver following the cue, as well as their step width, lateral position, and stepping variability ellipses at each maneuver step.

Results

Young and older adults did not differ in when they initiated the maneuver, but participants with lower perceived ability took longer to do so. Young and older adults also did not exhibit differences in step width or lateral positions at any maneuver step, but participants with greater physical ability reached their new path faster. While only older adults exhibited stepping adaptations prior to initiating the maneuver, both groups traded-off stability for maneuverability to enact the lateral maneuver.

Significance

Physical and perceived balance ability, rather than age per se, differentially influenced maneuver task performance. Humans must make decisions related to the task of walking itself and do so based on both physical and perceived factors. Understanding and targeting these interactions may help improve walking performance among older adults.

Learning from humans to build social cognition among robots

Self-organized groups of robots have generally coordinated their behaviors using quite simple social interactions. Although simple interactions are sufficient for some group behaviors, future research needs to investigate more elaborate forms of coordination, such as social cognition, to progress towards real deployments. In this perspective, we define social cognition among robots as the combination of social inference, social learning, social influence, and knowledge transfer, and propose that these abilities can be established in robots by building underlying mechanisms based on behaviors observed in humans. We review key social processes observed in humans that could inspire valuable capabilities in robots and propose that relevant insights from human social cognition can be obtained by studying human-controlled avatars in virtual environments that have the correct balance of embodiment and constraints. Such environments need to allow participants to engage in embodied social behaviors, for instance through situatedness and bodily involvement, but, at the same time, need to artificially constrain humans to the operational conditions of robots, for instance in terms of perception and communication. We illustrate our proposed experimental method with example setups in a multi-user virtual environment.

Adaptive multi-objective control explains how humans make lateral maneuvers while walking

To successfully traverse their environment, humans often perform maneuvers to achieve desired task goals while simultaneously maintaining balance. Humans accomplish these tasks primarily by modulating their foot placements. As humans are more unstable laterally, we must better understand how humans modulate lateral foot placement. We previously developed a theoretical framework and corresponding computational models to describe how humans regulate lateral stepping during straight-ahead continuous walking. We identified goal functions for step width and lateral body position that define the walking task and determine the set of all possible task solutions as Goal Equivalent Manifolds (GEMs). Here, we used this framework to determine if humans can regulate lateral stepping during non-steady-state lateral maneuvers by minimizing errors consistent with these goal functions. Twenty young healthy adults each performed four lateral lane-change maneuvers in a virtual reality environment. Extending our general lateral stepping regulation framework, we first re-examined the requirements of such transient walking tasks. Doing so yielded new theoretical predictions regarding how steps during any such maneuver should be regulated to minimize error costs, consistent with the goals required at each step and with how these costs are adapted at each step during the maneuver. Humans performed the experimental lateral maneuvers in a manner consistent with our theoretical predictions. Furthermore, their stepping behavior was well modeled by allowing the parameters of our previous lateral stepping models to adapt from step to step. To our knowledge, our results are the first to demonstrate humans might use evolving cost landscapes in real time to perform such an adaptive motor task and, furthermore, that such adaptation can occur quickly-over only one step. Thus, the predictive capabilities of our general stepping regulation framework extend to a much greater range of walking tasks beyond just normal, straight-ahead walking.

Effects of the Task Complexity on the Single Movement Response Time of Upper and Lower Limbs in Police Officers

Police officers occasionally encounter belligerents resisting or even physically assaulting them without or with objects. The self-defense or legal utilization of use of force to disable the offender from harming an officer or others may depend on a single movement speed of hands and legs. This study investigated the effects of task complexity on a single movement response time of the upper and lower limbs in police officers. The sample consisted of 32 male police officers aged between 23 and 50 years. They performed a single movement as fast as possible with their upper and lower limb in three incrementally more complex tasks. In the first task, participants acted on a light signal and with their dominant limb they had to turn off the signal as fast as possible. In the second task, on the light signal, participants could turn off the light with free choice of the upper limb in a hand task or lower limb in a leg task. In the third task, participants had to turn the light off with the right limb if the light turned red and with the left limb if the light turned blue. The BlazePod device was used to assess the movement response time. The results show that there was a significant effect of task complexity on the single movement response time of the hand (F = 24.5, p < 0.001) and leg (F = 46.2, p < 0.001). The training of police officers should utilize specific and situational tasks to improve movement response time by improving the redundancy in decision-making processes during work-specific tasks of different complexity.

The evolution of brain architectures for predictive coding and active inference

This article considers the evolution of brain architectures for predictive processing. We argue that brain mechanisms for predictive perception and action are not late evolutionary additions of advanced creatures like us. Rather, they emerged gradually from simpler predictive loops (e.g. autonomic and motor reflexes) that were a legacy from our earlier evolutionary ancestors-and were key to solving their fundamental problems of adaptive regulation. We characterize simpler-to-more-complex brains formally, in terms of generative models that include predictive loops of increasing hierarchical breadth and depth. These may start from a simple homeostatic motif and be elaborated during evolution in four main ways: these include the multimodal expansion of predictive control into an allostatic loop; its duplication to form multiple sensorimotor loops that expand an animal's behavioural repertoire; and the gradual endowment of generative models with hierarchical depth (to deal with aspects of the world that unfold at different spatial scales) and temporal depth (to select plans in a future-oriented manner). In turn, these elaborations underwrite the solution to biological regulation problems faced by increasingly sophisticated animals. Our proposal aligns neuroscientific theorising-about predictive processing-with evolutionary and comparative data on brain architectures in different animal species. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.