One to four, and nothing more: nonconscious parallel individuation of objects during action planning

Capacity Limits Lead to Information Bottlenecks in Ongoing Rapid Motor Behaviors

Studies of ongoing, rapid motor behaviors have often focused on the decision-making implicit in the task. Here, we instead study how decision-making integrates with the perceptual and motor systems and propose a framework of limited-capacity, pipelined processing with flexible resources to understand rapid motor behaviors. Results from three experiments show that human performance is consistent with our framework: participants perform objectively worse as task difficulty increases, and, surprisingly, this drop in performance is largest for the most skilled performers. As well, our analysis shows that the worst-performing participants can perform equally well under increased task demands, which is consistent with flexible neural resources being allocated to reduce bottleneck effects and improve overall performance. We conclude that capacity limits lead to information bottlenecks and that processes like attention help reduce the effects that these bottlenecks have on maximal performance.

Anticipatory action planning for stepping onto competing potential targets

The brain plans an anticipatory action for performing tasks successfully and effortlessly even if there are multiple possible options. There is increasing evidence that, when multiple actions are possible, the brain considers two factors when planning an anticipatory action—the probabilistic value and the action cost for each potential action. When the action involves maintaining upright balance, such as standing, stepping, or walking, the action cost for maintaining postural stability could be considered dominantly. We addressed this issue by using a “go-before-you-know” task to step onto a target on the floor. In this task, two potential targets were located on the medial or lateral side of the stepping foot, and the true target was cued only after participants shifted their loads to leave that foot. Participants initiated their stepping actions without knowing which of the potential targets would be the true one. The results showed that, for the majority of participants, lateral displacements of the center of pressure (COP) with two potential targets were similar to those when a single target exists on the individual’s medial side. Given that mediolateral postural stability became more destabilized with stepping onto the medial target than stepping onto the lateral target, they were likely to plan their mediolateral components of the postural adjustments for the worst-case scenario (i.e., falling). Additionally, posterior COP movements with two potential targets became smaller than those with a single target, suggesting an effort to create extra time to determine the true target and to adjust the swing foot. Based on these findings, we concluded that action costs for maintaining postural stability were considered dominantly for planning an anticipatory action to accomplish a stepping task successfully while ensuring upright balance.

Different planning policies for the initial movement velocity depending on whether the known uncertainty is in the cursor or in the target: Motor planning in situations where two potential movement distances exist

During goal-directed behaviors, individuals can be required to start a movement before deciding on the final goal. Previous studies have focused on the initial movement direction in situations involving multiple targets in different directions from the starting position and have shown that the movement is initiated in the average direction among the target directions. However, the previous studies only included situations with targets at equivalent distances, and the characteristics of motor planning in situations with multiple movement possibilities over different potential distances are unclear. In such situations, movement velocity is another important control variable. Furthermore, while previous studies examined situations with an uncertain motor target position, uncertainty can also exist in the effector position (e.g., body or tool locations). Therefore, we examined (1) whether the average output is confirmed in the initial movement velocity during execution in situations involving two potential movements with different distances. In addition, we examined (2) whether planning of the movement velocity can differ depending on the presence of uncertainty in the cursor or the target. In the main conditions, the participants were required to start a reaching movement with two potential movement distances; in the two-cursor condition, two cursors were presented before the start of the trial, and in the two-target condition, two targets were presented. As a control condition, a distance condition corresponding to each main condition was also performed. In the control condition, the initial movement velocity varied linearly with distance. Then, we tested whether the initial movement velocity in situations with two potential movement distances would follow the averaging output of the corresponding control condition. The results revealed that while the initial movement velocity in the two-target condition was slower than the averaging output, that in the two-cursor condition approached the averaging output. These results suggest that the velocity profile of the goal-directed movement is not simply averaged in a situation where two potential targets exist, and that there is a difference in the planning policy of the initial movement depending on whether the known uncertainty is for the movement goal or the effector.

The dynamics of decision-making and action during active sampling

Embodied Cognition Theories (ECTs) of decision-making propose that the decision process pervades the execution of choice actions and manifests itself in these actions. Decision-making scenarios where actions not only express the choice but also help sample information can provide a valuable, ecologically relevant model for this framework. We present a study to address this paradigmatic situation in humans. Subjects categorized (2AFC task) a central object image, blurred to different extents, by moving a cursor toward the left or right of the display. Upward cursor movements reduced the image blur and could be used to sample information. Thus, actions for decision and actions for sampling were orthogonal to each other. We analyzed response trajectories to test whether information-sampling movements co-occurred with the ongoing decision process. Trajectories were bimodally distributed, with one kind being direct towards one response option (non-sampling), and the other kind containing an initial upward component before veering off towards an option (sampling). This implies that there was an initial decision at the early stage of a trial, whether to sample information or not. Importantly, in sampling trials trajectories were not purely upward, but rather had a significant horizontal deviation early on. This result suggests that movements to sample information exhibit an online interaction with the decision process, therefore supporting the prediction of the ECTs under ecologically relevant constrains.

Modulation of initial movement for double potential targets with specific time constraints

In goal-directed behavior, individuals are often required to plan and execute a movement with multiple competing reach targets simultaneously. The time constraint assigned to the target is an important factor that affect the initial movement planning, but the adjustments made to the starting behavior considering the time constraints specific to each target have not yet been clarified. The current study examined how humans adjusted their motor planning for double potential targets with independent time constraints under a go-before-you-know situation. The results revealed that the initial movements were modulated depending on the time constraints for potential targets. However, under tight time constraints, the performance in the double-target condition was lower than the single-target condition, which was a control condition implemented to estimate performance when one target is ignored. These results indicate that the initial movement for multiple potential targets with independent time constraints can be modified, but the planning is suboptimal.

Sensorimotor strategy selection under time constraints in the presence of two motor targets with different values

Goal-directed movements often require choosing an option from multiple potential goals under time constraints. However, there are limited studies on how humans change their time spent on decision-making and movement patterns according to time constraints. Here, we examined how sensorimotor strategies are selected under time constraints when the target values are uncertain. In the double-target condition, the values were uncertain until the movement onset and presented immediately afterwards. The behavior in this condition was compared to the single-target condition, in relation to time constraints and target-separation-angles. The results showed that the participants frequently used the choice-reaction even under tight time constraints, and their performance was consistently lower than that in the single-target condition. Additionally, in the double-target condition, differences in the movement trajectory depending on the time constraint and target-separation angle were confirmed. Specifically, the longer the time constraint, the higher the frequency of the intermediate behavior (to initiate movement toward the intermediate direction of two targets) or the change-of-mind behavior (to change the aiming target during movement). Furthermore, the smaller the target-separation angle, the higher the frequency of intermediate behavior, but the frequency of change-of-mind was not affected by the target-separation angle. These results suggest that the participants initiated the movement at an incomplete value judgment stage in some trials. Furthermore, they seemed to select a strategy to utilize the information obtained during the movement, taking into account the time constraints and target-separation angle. Our results show a consistent cognitive bias in choosing a higher value when multiple alternatives have different values. Additionally, we also suggest flexibility and adaptability in the movement patterns in response to time constraints.

Action planning and control under uncertainty emerge through a desirability-driven competition between parallel encoding motor plans

Living in an uncertain world, nearly all of our decisions are made with some degree of uncertainty about the consequences of actions selected. Although a significant progress has been made in understanding how the sensorimotor system incorporates uncertainty into the decision-making process, the preponderance of studies focus on tasks in which selection and action are two separate processes. First people select among alternative options and then initiate an action to implement the choice. However, we often make decisions during ongoing actions in which the value and availability of the alternatives can change with time and previous actions. The current study aims to decipher how the brain deals with uncertainty in decisions that evolve while acting. To address this question, we trained individuals to perform rapid reaching movements towards two potential targets, where the true target location was revealed only after the movement initiation. We found that reaction time and initial approach direction are correlated, where initial movements towards intermediate locations have longer reaction times than movements that aim directly to the target locations. Interestingly, the association between reaction time and approach direction was independent of the target probability. By modeling the task within a recently proposed neurodynamical framework, we showed that action planning and control under uncertainty emerge through a desirability-driven competition between motor plans that are encoded in parallel.

Motor planning under uncertainty

Actions often require the selection of a specific goal amongst a range of possibilities, like when a softball player must precisely position her glove to field a fast-approaching ground ball. Previous studies have suggested that during goal uncertainty the brain prepares for all potential goals in parallel and averages the corresponding motor plans to command an intermediate movement that is progressively refined as additional information becomes available. Although intermediate movements are widely observed, they could instead reflect a neural decision about the single best action choice given the uncertainty present. Here we systematically dissociate these possibilities using novel experimental manipulations and find that when confronted with uncertainty, humans generate a motor plan that optimizes task performance rather than averaging potential motor plans. In addition to accurate predictions of population-averaged changes in motor output, a novel computational model based on this performance-optimization theory accounted for a majority of the variance in individual differences between participants. Our findings resolve a long-standing question about how the brain selects an action to execute during goal uncertainty, providing fundamental insight into motor planning in the nervous system.

Multiple-object tracking and visually guided touch

Multiple-object tracking (MOT) involves keeping track of the positions of multiple independent target items as they move among distractors. According to Pylyshyn (Cognition, 80, 127-158, 2001), the item individuation mechanism used in MOT is also used in visually guided touch. To test this, we compared single-task MOT (MOT alone) with dual-task MOT (MOT while touching items that changed colour), looking for interference: cases where single-task performance was worse than dual-task. Touching items that changed colour interfered with MOT, but effects varied depending on whether the item touched was a target or distractor in MOT. Touching distractors always reduced MOT performance more than touching targets. Touching targets during MOT did not interfere when there was only a single target to track but interfered more once there were two or more targets. We also measured interference based on latencies to touch items that changed colour, comparing single and dual-task conditions (touch alone, touch + track). MOT interfered with touch, increasing RT to touch items that changed colour, with latencies significantly higher when those items were distractors rather than targets. Overall, there was general interference (differences between single and dual-task performance), as might be expected if coordinating the two tasks required a common limited resource such as general attention or working memory. However, there was also differential interference that varied based on whether the touched item was a target or distractor in MOT. This differential interference suggests the specific mechanisms used in MOT may also play a role in visually guided touch.

Visual working memory and action: Functional links and bi-directional influences

Working memory bridges perception to action over extended delays, enabling flexible goal-directed behaviour. To date, studies of visual working memory – concerned with detailed visual representations such as shape and colour – have considered visual memory predominantly in the context of visual task demands, such as visual identification and search. Another key purpose of visual working memory is to directly inform and guide upcoming actions. Taking this as a starting point, I review emerging evidence for the pervasive bi-directional links between visual working memory and (planned) action, and discuss these links from the perspective of their common goal of enabling flexible and precise behaviour.

Reaching decisions during ongoing movements

Neurophysiological studies suggest that when decisions are made between concrete actions, the selection process involves a competition between potential action representations in the same sensorimotor structures involved in executing those actions. However, it is unclear how such models can explain situations, often encountered during natural behavior, in which we make decisions while were are already engaged in performing an action. Does the process of deliberation characterized in classical studies of decision-making proceed the same way when subjects are deciding while already acting? In the present study, human subjects continuously tracked a target moving in the horizontal plane and were occasionally presented with a new target to which they could freely choose to switch at any time, whereupon it became the new tracked target. We found that the probability of choosing to switch increased with decreasing distance to the new target and increasing size of the new target relative to the tracked target, as well as when the direction to the new target was aligned (either toward or opposite) to the current tracking direction. However, contrary to our expectations, subjects did not choose targets that minimized the energetic costs of execution, as calculated by a biomechanical model of the arm. When the constraints of continuous tracking were removed in variants of the task involving point-to-point movements, the expected preference for lower cost choices was seen. These results are discussed in the context of current theories of nested feedback control, internal models of forward dynamics, and high-dimensional neural spaces.NEW & NOTEWORTHY Current theories of decision-making primarily address how subjects make decisions before executing selected actions. However, in our daily lives we often make decisions while already performing some action (e.g., while playing a sport or navigating through a crowd). To gain insight into how current theories can be extended to such "decide-while-acting" scenarios, we examined human decisions during continuous manual tracking and found some intriguing departures from how decisions are made in classical "decide-then-act" paradigms.

Decision-making in sensorimotor control

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

Concurrent visual and motor selection during visual working memory guided action

Visual working memory enables us to hold onto past sensations in anticipation that these may become relevant for guiding future actions. Yet laboratory tasks have treated visual working memories in isolation from their prospective actions and have focused on the mechanisms of memory retention rather than utilization. To understand how visual memories become used for action, we linked individual memory items to particular actions and independently tracked the neural dynamics of visual and motor selection when memories became used for action. This revealed concurrent visual-motor selection, engaging appropriate visual and motor brain areas at the same time. Thus we show that items in visual working memory can invoke multiple, item-specific, action plans that can be accessed together with the visual representations that guide them, affording fast and precise memory-guided behavior.

Target Uncertainty During Motor Decision-Making: The Time Course of Movement Variability Reveals the Effect of Different Sources of Uncertainty on the Control of Reaching Movements

The processes underlying motor decision-making have recently caught considerable amount of scientific attention, focusing on the integration of empirical evidence from sensorimotor control research with psychological theories and computational models on decision-making. Empirical studies on motor decision-making suggest that the kinematics of goal-directed reaching movements are sensitive to the level of target uncertainty during movement planning. However, the source of uncertainty as a relevant factor influencing the process of motor decision-making has not been sufficiently considered, yet. In this study, we test the assumption that the source of target uncertainty has an effect on motor decision-making, which can be proven by analyzing movement variability during the time course of movement execution. Ten healthy young adults performed three blocks with 66 trials of goal-directed reaching movements in each block, across which the source and level of reach target uncertainty at movement onset were manipulated (“no uncertainty”, “extrinsic uncertainty”, and “intrinsic uncertainty”). Fingertip position of the right index finger was recorded using an optical motion tracking system. Standard kinematic measures (i.e., path length and movement duration) as well as variability of fingertip position across the time course of movement execution and at movement end were analyzed. In line with previous studies, we found that a high level of extrinsic target uncertainty leads to increased overall movement duration, which could be attributed to increased path length in this condition, as compared to intrinsic and no target uncertainty (all p < 0.001). Movement duration and path length did not show any differences between the latter two conditions. However, the time course analysis of movement variability revealed significant differences between these two conditions, with increased variability of fingertip position in the presence of intrinsic target uncertainty (Condition × Sampling point: p = 0.01), though considerably less than under high extrinsic target uncertainty (p ≤ 0.001). These findings suggest that both the level and source of uncertainty have a significant effect on the processing of potential action plans during motor decision-making, which can be revealed through the analysis of the time course of movement variability at the end-effector level.

Visual Selection of the Future Reach Path in Obstacle Avoidance

In two EEG experiments, we studied the role of visual attention during the preparation of manual movements around an obstacle. Participants performed rapid hand movements to a goal position avoiding a central obstacle either on the left or right side, depending on the pitch of the acoustical go signal. We used a dot probe paradigm to analyze the deployment of spatial attention in the visual field during the motor preparation. Briefly after the go signal but still before the hand movement actually started, a visual transient was flashed either on the planned pathway of the hand (congruent trials) or on the opposite, movement-irrelevant side (incongruent trials). The P1/N1 components that were evoked by the onset of the dot probe were enhanced in congruent trials where the visual transient was presented on the planned path of the hand. The results indicate that, during movement preparation, attention is allocated selectively to the planned trajectory the hand is going to take around the obstacle.

Spontaneous in-flight accommodation of hand orientation to unseen grasp targets: A case of action blindsight

The division of labour between the dorsal and ventral visual pathways is well established. The ventral stream supports object identification, while the dorsal stream supports online processing of visual information in the service of visually guided actions. Here, we report a case of an individual with a right inferior quadrantanopia who exhibited accurate spontaneous rotation of his wrist when grasping a target object in his blind visual field. His accurate wrist orientation was observed despite the fact that he exhibited no sensitivity to the orientation of the handle in a perceptual matching task. These findings indicate that non-geniculostriate visual pathways process basic volumetric information relevant to grasping, and reinforce the observation that phenomenal awareness is not necessary for an object's volumetric properties to influence visuomotor performance.

Confidence judgments during ratio comparisons reveal a Bayesian bias

Rational numbers are essential in mathematics and decision-making but humans often and erroneously rely on the magnitude of the numerator or denominator to determine the relative size of a quotient. The source of this flawed whole number strategy is poorly understood. Here we test the Bayesian hypothesis that the human bias toward large values in the numerator or denominator of a ratio estimate is the result of higher confidence in large samples. Larger values are considered a better (more certain) instance of that ratio than the same ratio composed of smaller values. We collected confidence measures explicitly (Experiment 1) and implicitly (Experiment 2) during subjects' comparisons of non-symbolic proportions (images with arrays of orange and blue dots). We manipulated the discernibility of the fractions to control difficulty and varied the cardinality and congruency of the numerators, denominators, and ratio values (e.g. 8/20 vs. 5/10 and 16/40 vs. 10/20). The results revealed that subjects' confidence during ratio comparisons was modulated by the numerical magnitude of the fraction's components, consistent with a Bayesian perception of relative ratios. The results suggest that the large number bias could arise from greater confidence in large samples.

Extinction as a deficit of the decision-making circuitry in the posterior parietal cortex

Extinction is a common neurologic deficit that often occurs as one of a constellation of symptoms seen with lesions of the posterior parietal cortex (PPC). Although extinction has typically been considered a deficit in the allocation of attention, new findings, particularly from nonhuman primate studies, point to one potential and important source of extinction as damage to decision-making circuits for actions within the PPC. This new understanding provides clues to potential therapies for extinction. Also the finding that the PPC is important for action decisions and action planning has led to new neuroprosthetic applications using PPC recordings as control signals to assist paralyzed patients.

Grip force when reaching with target uncertainty provides evidence for motor optimization over averaging

When presented with competing potential reach targets and required to launch a movement before knowing which one will be cued as the target, people initially reach in the average target direction. Although this spatial averaging could arise from executing a weighted average of motor plans for the potential targets, it could also arise from planning a single, optimal movement. To test between these alternatives we used a task in which participants were required to reach to either a single target or towards two potential targets while grasping an object. A robotic device applied a lateral elastic load to the object requiring large grip forces for reaches to targets either side of midline and a minimal grip force for midline movements. As expected, in trials with two targets located either side of midline, participants initially reached straight ahead. Critically, on these trials the initial grip force was minimal, appropriate for the midline movement, and not the average of the large grip forces required for movements to the individual targets. These results indicate that under conditions of target uncertainty, people do not execute an average of planned actions but rather a single movement that optimizes motor costs.

Reaching reveals that best-versus-rest processing contributes to biased decision making

The study of human decision making has revealed many contexts in which decisions are systematically biased. These biases are particularly evident in risky decisions, characterized by choice outcomes that are probabilistic. One recently explored bias is the extreme-outcome rule: the tendency for participants to overvalue both the best and worst outcome when they learn about choice probabilities through trial and error (aka experience). Here we aimed to test whether the extreme-outcome rule arises in part from a disproportionate subjective weight on extreme values. Participants reached to choose between two options in a riskless task where each choice option always produced the same result. In contrast to the idea that the overvaluing of extreme outcomes results from participants overestimating the underlying choice probabilities (e.g. treating a 50% "worst" outcome as though it occurred 60% of the time), we find overvaluation of extreme outcomes even when they are not probabilistic. Particularly, we find strong evidence for overvaluation of the best outcome relative to all other outcomes in how participants enact their decision (reaction times and reaching movements), but no evidence for such overvaluation in participants' choice accuracy. Compared to the extreme-outcome rule, these results are more simply characterized in a framework where the "best" option is given a boost in processing relative to the "rest" of other available options.

Rapid Automatic Motor Encoding of Competing Reach Options

Mounting neural evidence suggests that, in situations in which there are multiple potential targets for action, the brain prepares, in parallel, competing movements associated with these targets, prior to implementing one of them. Central to this interpretation is the idea that competing viewed targets, prior to selection, are rapidly and automatically transformed into corresponding motor representations. Here, by applying target-specific, gradual visuomotor rotations and dissociating, unbeknownst to participants, the visual direction of potential targets from the direction of the movements required to reach the same targets, we provide direct evidence for this provocative idea. Our results offer strong empirical support for theories suggesting that competing action options are automatically represented in terms of the movements required to attain them. The rapid motor encoding of potential targets may support the fast optimization of motor costs under conditions of target uncertainty and allow the motor system to inform decisions about target selection.

The sequential encoding of competing action goals involves dynamic restructuring of motor plans in working memory

Recent neural and behavioral findings provide support for the influential idea that in situations in which multiple action options are presented simultaneously, we prepare action plans for each competing option before deciding between and executing one of those plans. However, in natural, everyday environments, our available action options frequently change from one moment to the next, and there is often uncertainty as to whether additional options will become available before having to select a particular course of action. Here, with the use of a target-directed reaching task, we show that in this situation, the brain specifies a competing action for each new, sequentially presented potential target and that recently formed action plans can be revisited and updated so as to conform with separate, more newly developed, plans. These findings indicate that the brain forms labile motor plans for sequentially arising target options that can be flexibly restructured to accommodate new motor plans.

Oscillatory Activity in the Infant Brain and the Representation of Small Numbers

Gamma-band oscillatory activity (GBA) is an established neural signature of sustained occluded object representation in infants and adults. However, it is not yet known whether the magnitude of GBA in the infant brain reflects the quantity of occluded items held in memory. To examine this, we compared GBA of 6–8 month-old infants during occlusion periods after the representation of two objects vs. that of one object. We found that maintaining a representation of two objects during occlusion resulted in significantly greater GBA relative to maintaining a single object. Further, this enhancement was located in the right occipital region, which is consistent with previous object representation research in adults and infants. We conclude that enhanced GBA reflects neural processes underlying infants’ representation of small numbers.

Parallel specification of competing sensorimotor control policies for alternative action options

Recent theory proposes that the brain, when confronted with several action possibilities, prepares multiple competing movements before deciding among them. Psychophysical supporting evidence for this idea comes from the observation that when reaching towards multiple potential targets, the initial movement is directed towards the average location of the targets, consistent with multiple prepared reaches being executed simultaneously. However, reach planning involves far more than specifying movement direction; it requires the specification of a sensorimotor control policy that sets feedback gains shaping how the motor system responds to errors induced by noise or external perturbations. Here we found that, when a subject is reaching towards multiple potential targets, the feedback gain corresponds to an average of the gains specified when reaching to each target presented alone. Our findings provide evidence that the brain, when presented with multiple action options, computes multiple competing sensorimotor control policies in parallel before implementing one of them.

Estimating group size from human speech: Three's a conversation, but four's a crowd

Much previous research has examined various aspects of auditory processing, including the localization of sounds, and the influence of lexical and indexical information on language processing. In the present set of experiments we explored the ability of listeners to estimate the number of speakers in a group solely from the information in an auditory signal. The bound on accurately estimating the number of simultaneous speakers is 3. We suggest that subitization-the ability to estimate numerosity of visual and auditory elements without explicitly counting these elements-rather than the capacity of short-term memory, may underlie this limitation. The cognitive constraint on estimating the number of simultaneous speakers may have implications for a wide variety of seemingly unrelated psychological phenomena.

Dynamic Integration of Value Information into a Common Probability Currency as a Theory for Flexible Decision Making

Decisions involve two fundamental problems, selecting goals and generating actions to pursue those goals. While simple decisions involve choosing a goal and pursuing it, humans evolved to survive in hostile dynamic environments where goal availability and value can change with time and previous actions, entangling goal decisions with action selection. Recent studies suggest the brain generates concurrent action-plans for competing goals, using online information to bias the competition until a single goal is pursued. This creates a challenging problem of integrating information across diverse types, including both the dynamic value of the goal and the costs of action. We model the computations underlying dynamic decision-making with disparate value types, using the probability of getting the highest pay-off with the least effort as a common currency that supports goal competition. This framework predicts many aspects of decision behavior that have eluded a common explanation.

Action plan co-optimization reveals the parallel encoding of competing reach movements

Several influential cognitive theories propose that in situations affording more than one possible target of action, we prepare multiple competing movements before selecting one. Here we provide direct evidence for this provocative but largely untested idea and demonstrate why preparing multiple movements is computationally advantageous. Using a reaching task in which movements are initiated after one of two potential targets is cued, we show that the movement generated for the cued target borrows components of the movement that would have been required for the other, competing target. This interaction can only arise if multiple potential movements are fully specified in advance and we demonstrate that it reduces the time required to launch a given action plan. Our findings suggest that this co-optimization of motor plans is highly automatic and largely occurs outside conscious awareness.

Several prominent cognitive theories propose that in situations affording more than one action strategy, the brain prepares multiple competing movements prior to selecting one. Here the authors provide direct experimental evidence for this provocative but largely untested notion.

A biologically plausible computational theory for value integration and action selection in decisions with competing alternatives

Decision making is a vital component of human and animal behavior that involves selecting between alternative options and generating actions to implement the choices. Although decisions can be as simple as choosing a goal and then pursuing it, humans and animals usually have to make decisions in dynamic environments where the value and the availability of an option change unpredictably with time and previous actions. A predator chasing multiple prey exemplifies how goals can dynamically change and compete during ongoing actions. Classical psychological theories posit that decision making takes place within frontal areas and is a separate process from perception and action. However, recent findings argue for additional mechanisms and suggest the decisions between actions often emerge through a continuous competition within the same brain regions that plan and guide action execution. According to these findings, the sensorimotor system generates concurrent action-plans for competing goals and uses online information to bias the competition until a single goal is pursued. This information is diverse, relating to both the dynamic value of the goal and the cost of acting, creating a challenging problem in integrating information across these diverse variables in real time. We introduce a computational framework for dynamically integrating value information from disparate sources in decision tasks with competing actions. We evaluated the framework in a series of oculomotor and reaching decision tasks and found that it captures many features of choice/motor behavior, as well as its neural underpinnings that previously have eluded a common explanation.

In high-pressure situations, such as driving on a highway or flying a plane, people have limited time to select between competing options while acting. Each option is usually accompanied with reward benefits (e.g., avoid traffic) and action costs (e.g., fuel consumption) that characterize the value of the option. The value and the availability of an option can change dynamically even during ongoing actions which compounds the decision-making challenge. How the brain dynamically integrates value information from disparate sources and selects between competing options is still poorly understood. In the current study, we present a neurodynamical framework to show how a distributed brain network can solve the problem of value integration and action selection in decisions with competing alternatives. It combines dynamic neural field theory with stochastic optimal control theory, and includes circuitry for perception, expected reward, effort cost and decision-making. It provides a principled way to explain both the neural and the behavioral findings from a series of visuomotor decision tasks in human and animal studies. For instance, the model shows how the competitive interactions between populations of neurons within and between sensorimotor regions can result in “spatial-averaging” movements, and how decision-variables influence neural activity and choice behavior.

Small number preference in guiding attention

Healthy individuals are usually biased toward small numbers when they are asked to mentally bisect number intervals or generate number sequences. Number magnitude may be represented spatially along a left-to-right mental number line. The preference for small numbers is believed to reflect the leftward spatial bias of this numerical representation. This study examined whether small numbers captured visual attention more than larger numbers. Participants were asked to detect a target pre-cued by a small or a large number. We found that the response was faster when the target was pre-cued by a small number than when pre-cued by a large number, suggesting that visual attention is preferentially allocated to small numbers. In addition, this attentional preference for small numbers was distinct for participants of different educational backgrounds. For science or engineering participants, this small number preference was enhanced by left-hand responding and was positively correlated with the small number preference in a random number generation task, suggesting that the small number preference was attributable to a leftward bias of the spatial representation. For liberal arts participants, however, left-hand responding did not enhance the small number preference and no correlations were found between the attention task and the random number generation task, suggesting that non-spatial processing mediated the small number preference. Our findings show that the small number preference occurs as early as the perceptual processing stage and distinct mechanisms underlie the preference for small numbers for participants with different educational backgrounds.

Three-dimensional reach trajectories as a probe of real-time decision-making between multiple competing targets

Though several features of cognitive processing can be inferred from the discrete measurement [e.g., reaction time (RT), accuracy, etc.] of participants' conscious reports (e.g., verbal or key-press responses), it is becoming increasingly clear that a much richer understanding of these features can be captured from continuous measures of rapid, largely non-conscious behaviors like hand or eye movements. Here, using new experimental data, we describe in detail both the approach and analyses implemented in some of our previous studies that have used rapid reaching movements under cases of target uncertainty in order to probe the features, constraints and dynamics of stimulus-related processing in the brain. This work, as well as that of others, shows that when individuals are simultaneously presented with multiple potential targets—only one of which will be cued after reach onset—they produce initial reach trajectories that are spatially biased in accordance with the probabilistic distribution of targets. Such “spatial averaging” effects are consistent with observations from neurophysiological studies showing that neuronal populations in sensorimotor brain structures represent multiple target choices in parallel and they compete for selection. These effects also confirm and help extend computational models aimed at understanding the underlying mechanisms that support action-target selection. We suggest that the use of this simple, yet powerful behavioral paradigm for providing a “real-time” visualization of ongoing cognitive processes occurring at the neural level offers great promise for studying processes related to a wide range of psychological phenomena, such as decision-making and the representation of objects.

Connecting the Dots

The perceptual system parses complex scenes into discrete objects. Parsing is also required for planning visually guided movements when more than one potential target is present. To examine whether visual perception and motor planning use the same or different parsing strategies, we used the connectedness illusion, in which observers typically report seeing fewer targets if pairs of targets are connected by short lines. We found that despite this illusion, when observers are asked to make speeded reaches toward targets in such displays, their reaches are unaffected by the presence of the connecting lines. Instead, their movement plans, as revealed by their movement trajectories, are influenced by the number of potential targets irrespective of whether connecting lines are present or not. This suggests that scene parsing for perception depends on mechanisms that are distinct from those that allow observers to plan rapid and efficient target-directed movements in situations with multiple potential targets.

Simultaneous encoding of the direction and orientation of potential targets during reach planning: evidence of multiple competing reach plans

Reaches performed in many natural situations involve selecting a specific target from a number of alternatives. Recent studies show that before reaching, multiple potential reach targets are encoded in brain regions involved in action control and that, when people are required to initiate the reach before the target is specified, initial hand direction is biased by the spatial distribution of potential targets. These findings have led to the suggestion that the brain, during planning, simultaneously prepares multiple reaches to potential targets. In addition to hand direction, reach planning often involves specifying other parameters such as wrist orientation. For example, when posting a letter in a mail slot, both the location and orientation of the slot must be encoded to control hand direction and orientation. Therefore, if the brain prepares multiple reaches to potential targets and if these targets require the specification of hand direction and orientation, then both of these variables should be biased by the spatial distribution of potential targets. To test this prediction, we examined a task in which participants moved a hand-held rectangular tool toward multiple rectangular targets of varying location and orientation, one of which was selected, with equal probability as the actual target after movement initiation. We found that initial hand direction and orientation were biased by the spatial distributions of potential target locations and orientations, respectively. This result is consistent with the idea that the brain, in cases of target uncertainty, simultaneously plans fully specified reaching movements to all potential targets.

Predicting targets of human reaching motions using different sensing technologies

Rapid recognition of voluntary motions is crucial in human-computer interaction, but few studies compare the predictive abilities of different sensing technologies. This paper thus compares performances of different technologies when predicting targets of human reaching motions: electroencephalography (EEG), electrooculography, camera-based eye tracking, electromyography (EMG), hand position, and the user's preferences. Supervised machine learning is used to make predictions at different points in time (before and during limb motion) with each individual sensing modality. Different modalities are then combined using an algorithm that takes into account the different times at which modalities provide useful information. Results show that EEG can make predictions before limb motion onset, but requires subject-specific training and exhibits decreased performance as the number of possible targets increases. EMG and hand position give high accuracy, but only once the motion has begun. Eye tracking is robust and exhibits high accuracy at the very onset of limb motion. Several advantages of combining different modalities are also shown, including advantages of combining measurements with contextual data. Finally, some recommendations are given for sensing modalities with regard to different criteria and applications. The information could aid human-computer interaction designers in selecting and evaluating appropriate equipment for their applications.

Two systems of non-symbolic numerical cognition

Studies of human adults, infants, and non-human animals demonstrate that non-symbolic numerical cognition is supported by at least two distinct cognitive systems: a “parallel individuation system” that encodes the numerical identity of individual items and an “approximate number system” that encodes the approximate numerical magnitude, or numerosity, of a set. The exact nature and role of these systems, however, have been debated for over a 100-years. Some argue that the non-symbolic representation of small numbers (<4) is carried out solely by the parallel individuation system and the non-symbolic representation of large numbers (>4) is carried out solely by the approximate number system. Others argue that all numbers are represented by the approximate number system. This debate has been fueled largely by some studies showing dissociations between small and large number processing and other studies showing similar processing of small and large numbers. Recent work has addressed this debate by showing that the two systems are present and distinct from early infancy, persist despite the acquisition of a symbolic number system, activate distinct cortical networks, and engage differentially based attentional constraints. Based on the recent discoveries, I provide a hypothesis that may explain the puzzling findings and makes testable predictions as to when each system will be engaged. In particular, when items are presented under conditions that allow selection of individuals, they will be represented as distinct mental items through parallel individuation and not as a numerical magnitude. In contrast, when items are presented outside attentional limits (e.g., too many, too close together, under high attentional load), they will be represented as a single mental numerical magnitude and not as distinct mental items. These predictions provide a basis on which researchers can further investigate the role of each system in the development of uniquely human numerical thought.