The continuous end-state comfort effect: weighted integration of multiple biases

Where scrollbars are clicked, and why

Scrolling is a widely used mean to interact with visual displays, usually to move content to a certain target location on the display. Understanding how user scroll might identify potentially suboptimal use and allows to infer users' intentions. In the present study, we examined where users click on a scrollbar depending on the intended scrolling action. In two online experiments, click positions were systematically adapted to the intended scrolling action. Click position selection could not be explained as strict optimization of the distance traveled with the cursor, memory load, or motor-cognitive factors. By contrast, for identical scrolling actions click positions strongly depended on the context and on previous scrolls. The behavior of our participants closely resembled behavior observed for manipulation of other physical devices and suggested a simple heuristic of movement planning. The results have implications for modeling human-computer interaction and may contribute to predicting user behavior.

Coordinating Initial and Final Action Goals in Planning Grasp-to-Rotate Movements: An ERP Study

Action goals have often been investigated in previous studies within a single action. However, most of the manual actions (such as prehension) are not restricted to a single action towards the object but can involve multiple follow-up actions to achieve a further purpose. The coordination of the initial (grip posture) and final (task purpose) action goals within such complex actions is still not fully understood. In the present experiment, the neural mechanisms underlying the goal coordination were investigated with the help of event-related potentials (ERP). With the "first cue - second cue - imperative signal" design, the action goals were presented separately in different sequences (either "final-initial" or "initial-final"), and participants were instructed to plan and execute a grasp-to-rotate movement with either free-choice or specified grasping. Results revealed that shorter reaction times were needed for the final-initial than for the initial-final trials only when the movement requires a free-choice grasping. At the moment when the goal information was incomplete (at the first cue), final goals evoked a larger anterior P2 than initial goals, whereas initial goals elicited a larger anterior N2 and a more robust frontal negativity (400--550 ms) than final goals. When the goal information was complete (at the second cue), we only found a larger P2 for final goals than for initial goals in free-choice grasping. Moreover, a larger N2 was also found for the specified than for the free-choice grasping in initial-final trials. These neurophysiological results indicate that final goals are more critical than initial grip postures in planning prehensile movements. The initial and final action goals seem to be preferably coordinated in a hierarchical manner, that is, the final task purpose is processed with precedence, whereas the initial grip posture is selected depending on the final task purpose.

Predictable real-time constraints reveal anticipatory strategies of coupled planning in a sequential pick and place task

Planning a sequence of two motor elements is much more than concatenating two independent movements. However, very little is known about the cognitive strategies that are used to perform fluent sequences for intentional object manipulation. In this series of studies, the participants’ task was to reach for and pick to place a wooden cylinder to set it on a place pad of three different diameters, which served to modify terminal accuracy constraints. Participants were required to perform the sequences (1) at their preferred speed or (2) as fast as possible. Action kinematics were recorded with the Qualisys motion-capture system in order to implement a real-time protocol to get participants to engage in a true interactive relation. Results revealed that with low internal constraints (at preferred speed), low coupling between the two elements of the motor sequence was observed, suggesting a step-by-step planning strategy. Under high constraints (at fastest speed), an important terminal accuracy effect back propagated to modify early kinematic parameters of the first element, suggesting strong coupling of the parameters in an encapsulated planning strategy. In Studies 2 and 3, we further manipulated instructions and timing constraints to confirm the importance of time and predictability of external information for coupled planning. These findings overall sustain the hypothesis that coupled planning can take place in a pick and place task when anticipatory strategies are possible. This mode of action planning may be the key reason why motor intention can be read through the observation of micro variations in body kinematics.

End-state comfort effect in manipulative motor actions of typical and atypical children: a systematic review

ABSTRACT This study aimed to make a systematic review of scientific articles on the planning of manual motor actions of typical and atypical children. To do so, a search of articles published between 1996 and 2017 was done on PubMed, LILACS, Science Direct, and SciElo databases. Original articles in English and Portuguese evaluating the planning of motor actions in typical and atypical children performing manual tasks were selected. For the analysis, the population age, number of children, type of task, main results, site of study and impact of the journal were considered. From the eighteen articles found, twelve were about typical children (from nine months old to twenty years old) and six about atypical children (from three to fourteen years old) diagnosed with autism, hemiplegic cerebral palsy, and developmental coordination disorders. In nine- to ten-year-old typical children, the planning ability of manual motor actions develops over the time and is similar to that of an adult. The atypical children showed motor planning ability lower than that of the typical children, and determining the age when this planning ability is acquired was not possible.

Hands Ahead in Mind and Motion: Active Inference in Peripersonal Hand Space

According to theories of anticipatory behavior control, actions are initiated by predicting their sensory outcomes. From the perspective of event-predictive cognition and active inference, predictive processes activate currently desired events and event boundaries, as well as the expected sensorimotor mappings necessary to realize them, dependent on the involved predicted uncertainties before actual motor control unfolds. Accordingly, we asked whether peripersonal hand space is remapped in an uncertainty anticipating manner while grasping and placing bottles in a virtual reality (VR) setup. To investigate, we combined the crossmodal congruency paradigm with virtual object interactions in two experiments. As expected, an anticipatory crossmodal congruency effect (aCCE) at the future finger position on the bottle was detected. Moreover, a manipulation of the visuo-motor mapping of the participants’ virtual hand while approaching the bottle selectively reduced the aCCE at movement onset. Our results support theories of event-predictive, anticipatory behavior control and active inference, showing that expected uncertainties in movement control indeed influence anticipatory stimulus processing.

Emergence of anticipatory actions in a novel task

Humans normally adapt earlier segments of multistep motor actions in such a way that the execution of later segments is facilitated. For example, the kinematics of grasping movements are adapted to the requirements of the intended subsequent object manipulations. Here we studied which factors foster adaptation of earlier action segments to later ones in a novel task for which no prior experience existed. Participants executed a two-step isometric force production task, in which the force produced in the first segment determined the difficulty of the second segment. Adaptation of the first segment to the second one benefited from explicit knowledge of the dependency between both segments but not from extensive prior experience with the second segment. These observations show that adaptation of motor actions to subsequent actions demands the construction of a task representation that allows to plan the first action segment with respect to its successor. How specifically the first segment is tailored to the second one does not depend on prior experience with the second segment but depends on experience from performing the interdependent two-step action sequence.

Preschool children adapt grasping movements to upcoming object manipulations: Evidence from a dial rotation task

In adults, the motor plans for object-directed grasping movements reflects the anticipated requirements of intended future object manipulations. This prospective mode of planning has been termed second-order planning. Surprisingly, second-order planning is thought to be fully developed only by 10 years of age, when children master seemingly more complex motor skills. In this study, we tested the hypothesis that already 5- and 6-year-old children consistently use second-order planning but that this ability does not become apparent in tasks that are traditionally used to probe it. We asked 5- and 6-year-olds and adults to grasp and rotate a circular dial in a clockwise or counterclockwise direction. Although children's grasp selections were less consistent on an intra- and inter-individual level than adults' grasp selections, all children adjusted their grasps to the upcoming dial rotations. By contrast, in an also administered bar rotation task, only a subset of children adjusted their grasps to different bar rotations, thereby replicating previous results. The results indicate that 5- and 6-year-olds consistently use second-order planning in a dial rotation task, although this ability does not become apparent in bar rotation tasks.

Inverting the planning gradient: adjustment of grasps to late segments of multi-step object manipulations

When someone grasps an object, the grasp depends on the intended object manipulation and usually facilitates it. If several object manipulation steps are planned, the first step has been reported to primarily determine the grasp selection. We address whether the grasp can be aligned to the second step, if the second step's requirements exceed those of the first step. Participants grasped and rotated a dial first by a small extent and then by various extents in the opposite direction, without releasing the dial. On average, when the requirements of the first and the second step were similar, participants mostly aligned the grasp to the first step. When the requirements of the second step were considerably higher, participants aligned the grasp to the second step, even though the first step still had a considerable impact. Participants employed two different strategies. One subgroup initially aligned the grasp to the first step and then ceased adjusting the grasp to either step. Another group also initially aligned the grasp to the first step and then switched to aligning it primarily to the second step. The data suggest that participants are more likely to switch to the latter strategy when they experienced more awkward arm postures. In summary, grasp selections for multi-step object manipulations can be aligned to the second object manipulation step, if the requirements of this step clearly exceed those of the first step and if participants have some experience with the task.

Habit outweighs planning in grasp selection for object manipulation

Object-directed grasping movements are adapted to intended interactions with an object. We address whether adjusting the grasp for object manipulation is controlled habitually, based on past experiences, or by goal-directed planning, based on an evaluation of the expected action outcomes. Therefore, we asked participants to grasp and rotate a dial. In such tasks, participants typically grasp the dial with an excursed, uncomfortable arm posture, which then allows to complete the dial rotation in a comfortable end-state. We extended this task by manipulating the contingency between the orientation of the grasp and the resulting end-state of the arm. A one-step (control) group rotated the dial to a single target. A two-step group rotated the dial to an initial target and then in the opposite direction. A three-step group rotated the dial to the initial target, then in the opposite direction, and then back to the initial target. During practice, the two-step and three-step groups reduced the excursion of their grasps, thus avoiding overly excursed arm postures after the second rotation. When the two-step and three-step groups were asked to execute one-step rotations, their grasps resembled those that were acquired during the two-step and three-step rotations, respectively. However, the carry-over was not complete. This suggests that adjusting grasps for forthcoming object manipulations is controlled by a mixture of habitual and goal-directed processes. In the present experiment, the former contributed approximately twice as much to grasp selection than the latter.

No Interrelation of Motor Planning and Executive Functions across Young Ages

The present study examined the developmental trajectories of motor planning and executive functioning in children. To this end, we tested 217 participants with three motor tasks, measuring anticipatory planning abilities (i.e., the bar-transport-task, the sword-rotation-task and the grasp-height-task), and three cognitive tasks, measuring executive functions (i.e., the Tower-of-Hanoi-task, the Mosaic-task, and the D2-attention-endurance-task). Children were aged between 3 and 10 years and were separated into age groups by 1-year bins, resulting in a total of eight groups of children and an additional group of adults. Results suggested (1) a positive developmental trajectory for each of the sub-tests, with better task performance as children get older; (2) that the performance in the separate tasks was not correlated across participants in the different age groups; and (3) that there was no relationship between performance in the motor tasks and in the cognitive tasks used in the present study when controlling for age. These results suggest that both, motor planning and executive functions are rather heterogeneous domains of cognitive functioning with fewer interdependencies than often suggested.

Toward a Unified Sub-symbolic Computational Theory of Cognition

This paper proposes how various disciplinary theories of cognition may be combined into a unifying, sub-symbolic, computational theory of cognition. The following theories are considered for integration: psychological theories, including the theory of event coding, event segmentation theory, the theory of anticipatory behavioral control, and concept development; artificial intelligence and machine learning theories, including reinforcement learning and generative artificial neural networks; and theories from theoretical and computational neuroscience, including predictive coding and free energy-based inference. In the light of such a potential unification, it is discussed how abstract cognitive, conceptualized knowledge and understanding may be learned from actively gathered sensorimotor experiences. The unification rests on the free energy-based inference principle, which essentially implies that the brain builds a predictive, generative model of its environment. Neural activity-oriented inference causes the continuous adaptation of the currently active predictive encodings. Neural structure-oriented inference causes the longer term adaptation of the developing generative model as a whole. Finally, active inference strives for maintaining internal homeostasis, causing goal-directed motor behavior. To learn abstract, hierarchical encodings, however, it is proposed that free energy-based inference needs to be enhanced with structural priors, which bias cognitive development toward the formation of particular, behaviorally suitable encoding structures. As a result, it is hypothesized how abstract concepts can develop from, and thus how they are structured by and grounded in, sensorimotor experiences. Moreover, it is sketched-out how symbol-like thought can be generated by a temporarily active set of predictive encodings, which constitute a distributed neural attractor in the form of an interactive free-energy minimum. The activated, interactive network attractor essentially characterizes the semantics of a concept or a concept composition, such as an actual or imagined situation in our environment. Temporal successions of attractors then encode unfolding semantics, which may be generated by a behavioral or mental interaction with an actual or imagined situation in our environment. Implications, further predictions, possible verification, and falsifications, as well as potential enhancements into a fully spelled-out unified theory of cognition are discussed at the end of the paper.

Too much anticipation? Large anticipatory adjustments of grasping movements to minimal object manipulations

When humans grasp objects, the grasps foreshadow the intended object manipulation. It has been suggested that grasps are selected that lead to medial arm postures, which facilitate movement speed and precision, during critical phases of the object manipulation. In Experiment 1, it has been tested whether grasp selections lead to medial postures during rotations of a dial. Participants twisted their arms considerably before grasping the dial, even when the upcoming dial rotation was minimal (5°). Participants neither assumed a medial posture at any point during a short rotation, nor did they assume any of the postures involved in short rotations in the opposite direction. Thus, grasp selections did not necessarily lead to specific postures at any point of the object manipulation. Experiment 2 examined the effect of various grasps on the speed of dial rotations. A medial initial grasp resulted in the fastest dial rotations for most rotation angles. Spontaneously selected grasps were more excursed than necessary to maximize dial rotation speed. This apparent overshot might be explained by participants' sensitive to the variability of their grasps and is in line with the assumption that grasps facilitate control over the grasped object.

Social categorization and cooperation in motor joint action: evidence for a joint end-state comfort

The present study investigated to what extent group membership affects an actor's representation of their partner's task in cooperative joint action. Participants performed a joint pick-and-place task in a naturalistic, breakfast-table-like paradigm which allowed the demonstration of varying degrees of cooperation. Participants transported a wooden cup from one end of a table to the other, with one actor moving it to an intermediate position from where their partner transported it to a goal position. Hand and finger movements were recorded via 3D motion tracking to assess actors' cooperative behavior. Before the joint action task was performed, participants were categorized as belonging to the same or to different groups, supposedly based on an assessment of their cognitive processing styles. Results showed that the orientation of the actors' fingers when picking up the cup was affected by its required angle at the goal position. When placing the cup at the intermediate position, most actors adapted the rotation of the cup's handle to the joint action goal, thereby facilitating the partner's subsequent movement. Male actors demonstrated such cooperative behavior only when performing the task together with an ingroup partner, while female actors demonstrated cooperative behavior irrespective of social categorization. These results suggest that actors tend to represent a partner's end-state comfort and integrate it into their own movement planning in cooperative joint action. However, social factors like group membership may modulate this tendency.

Manual (a)symmetries in grasp posture planning: a short review

Many activities of daily living require that we physically interact with one or more objects. Object manipulation provides an intriguing domain in which the presence and extent of manual asymmetries can be studied on a motor planning and a motor execution level. In this literature review we present a state of the art for manual asymmetries at the level of motor planning during object manipulation. First, we introduce pioneering work on grasp posture planning. We then sketch the studies investigating the impact of future task demands during unimanual and bimanual object manipulation tasks in healthy adult populations. In sum, in contrast to motor execution, there is little evidence for hand-based performance differences in grasp posture planning. We discuss potential reasons for the lack of manual asymmetries in motor planning and outline potential avenues of future research.

The influence of reducing intermediate target constraints on grasp posture planning during a three-segment object manipulation task

The present experiment examined the influence of final target position on grasp posture planning during a three-segment object manipulation task in which the required object orientation at the first target position was unconstrained. Participants grasped a cylindrical object from a home position, placed it at an intermediate position in a freely chosen orientation, and subsequently placed it at one of four final target positions. Considerable inter-individual differences in initial grasp selection were observed which also led to differences in final grasp postures. Whereas some participants strongly adjusted their initial grasp postures to the final target orientation, and thus showed a preference for end-state comfort, other participants showed virtually no adjustment in initial grasp postures, hence satisfying initial-state comfort. Interestingly, as intermediate grasp postures were similar regardless of initial grasp adjustment, intermediate-state comfort was prioritized by all participants. These results provide further evidence for the interaction of multiple action selection constraints in grasp posture planning during multi-segment object manipulation tasks. Whereas some constraints may take strict precedence in a given task, other constraints may be more flexible and weighted differently among participants. This differentiated weighting leads to task- and subject-specific constraint hierarchies and is reflected in inter-individual differences in grasp selection.

The contribution of cognitive, kinematic, and dynamic factors to anticipatory grasp selection

Object-directed grasping movements are usually adjusted in anticipation of the direction and extent of a subsequent object rotation. Such anticipatory grasp selections have been mostly explained in terms of the kinematics of the arm movement. However, object rotations of different directions and extents also differ in their dynamics and in how the tasks are represented. Here, we examined how the dynamics, the kinematics, and the cognitive representation of an object manipulation affect anticipatory grasp selections. We asked participants to grasp an object and rotate it by different angles and in different directions. To examine the influence of dynamic factors, we varied the object's weight. To examine the influence of the cognitive task representation, we instructed identical object rotations as either toward-top or away-from-top rotations. While instructed object rotation and cognitive task representation did affect grasp selection over the entire course of the experiment, a rather small effect of object weight only appeared late in the experiment. We suggest that grasp selections are determined on different levels. The representation of the kinematics of the object movement determines grasp selection on a trial-by-trial basis. The effect of object weight affects grasp selection by a slower adaptation process. This result implies that even simple motor acts, such as grasping, can only be understood when cognitive factors, such as the task representation, are taken into account.

Grasp posture planning during multi-segment object manipulation tasks - interaction between cognitive and biomechanical factors

The present study examined adaptations in the planning of initial grasp postures during a multi-segment object manipulation task. Participants performed a grasping and placing task that consisted of one, two, or three movement segments. The position of the targets was manipulated such that the degree of object rotation between the home and temporally proximal positions, and between the temporally proximal and distal target positions, varied. Participants selected initial grasp postures based on the specific requirements of the temporally proximal and temporally distal action segments, and adjustments in initial grasp posture depended on the temporal order of target location. In addition, during the initial stages of the experimental session initial grasp postures were influenced to a larger extent by the demands of the temporally proximal segment. However, over time, participants overcame these cognitive limitations and adjusted their initial grasp postures more strongly to the requirements of the temporally distal segment. Taken together, these results indicate that grasp posture planning is influenced by cognitive and biomechanical factors, and that participants learn to anticipate the task demands of temporally distal task demands, which we hypothesize, reduce the burden on the central nervous system.

Individual differences in motor planning during a multi-segment object manipulation task

Research has demonstrated that people will adopt initially awkward grasps if they afford more comfortable postures at the end of the movement. This end-state comfort effect provides evidence that humans represent future posture states and select appropriate grasps in anticipation of these postures. The purpose of the study was to examine to what extent the final action goal of a task influences motor planning of preceding segments, and whether grasp postures are planned to optimize end-state comfort during a three-segment action sequence in which two objects are manipulated, and participants can select from a continuous range of possible grasp postures. In the current experiment, participants opened a drawer, grasped an object from inside the drawer, and placed it on a table in one of the three target orientations (0°, 90°, or 180° object rotation required). Grasp postures during the initial movement segment (drawer opening) were not influenced by the final action goal (i.e., required target orientation). In contrast, both the intermediate (i.e., object grasping) and the final movement segment (i.e., object placing) were influenced by target orientation. In addition, participants adopted different strategies to achieve the action goal when the object required 180° rotation, with 42 % of participants prioritizing intermediate-state comfort and 58 % prioritizing end-state comfort. The results indicate that individuals optimize task performance by selecting lower level constraints that allow for successful completion of the action goal and that the selection of these constraints is dependent upon contextual, environmental, and internal influences.

Where to Grasp a Tool?

Biomechanical and environmental constraints limit body movements and tool use actions. However, in the case of tool use, such constraints can often be overcome by adjusting a tool’s tool transformation to the requirements of the intended tool use action. The research presented here examined whether participants grasped a lever at different positions, thus modifying the lever’s tool transformation, to accommodate speed and accuracy requirements of different tasks. Participants were asked to quickly track a sequence of targets with the lever. If accuracy requirements were high, participants compensated for limits in the accuracy of hand movements by grasping the lever at a position that enabled precise control of the lever. If accuracy requirements were low, participants compensated for limits in hand speed by grasping the lever at a position that enabled fast lever movements with comparatively slow hand movements. This task-dependent grasp selection was only present after participants had practiced the tasks. The data show that in addition to adapting to fixed tool transformations, participants also actively controlled tool transformations to facilitate tool use actions.