Habitual and goal-directed factors in (everyday) object handling

How to construct liquid-crystal spectacles to control vision of real-world objects and environments

A major challenge in studying naturalistic vision lies in controlling stimulus and scene viewing time. This is especially the case for studies using real-world objects as stimuli (rather than computerized images) because real objects cannot be "onset" and "offset" in the same way that images can be. Since the late 1980s, one solution to this problem has been to have the observer wear electro-optic spectacles with computer-controlled liquid-crystal lenses that switch between transparent ("open") and translucent ("closed") states. Unfortunately, the commercially available glasses (PLATO Visual Occlusion Spectacles) command a high price tag, the hardware is fragile, and the glasses cannot be customized. This led us to explore how to manufacture liquid-crystal occlusion glasses in our own laboratory. Here, we share the products of our work by providing step-by-step instructions for researchers to design, build, operate, and test liquid-crystal glasses for use in experimental contexts. The glasses can be assembled with minimal technical knowledge using readily available components, and they can be customized for different populations and applications. The glasses are robust, and they can be produced at a fraction of the cost of commercial alternatives. Tests of reliability and temporal accuracy show that the performance of our laboratory prototype was comparable to that of the PLATO glasses. We discuss the results of our work with respect to implications for promoting rigor and reproducibility, potential use cases, comparisons with other liquid-crystal shutter glasses, and how users can find information regarding future updates and developments.

Grip selection without tool knowledge: end-state comfort effect in familiar and novel tool use

A well-known phenomenon for the study of movement planning is the end-state comfort (ESC) effect: When they reach and grasp tools, individuals tend to adopt uncomfortable initial hand postures if that allows a subsequent comfortable final posture. In the context of tool use, this effect is modulated by tool orientation, task goal, and cooperation. However, the cognitive bases of the ESC effect remain unclear. The goal of this study was to determine the contribution of semantic tool knowledge and technical reasoning to movement planning, by testing whether the ESC effect typically observed with familiar tools would also be observed with novel tools. Twenty-six participants were asked to reach and grasp familiar and novel tools under varying conditions (i.e., tool's handle downward vs. upward; tool transport vs. use; solo vs. cooperation). In our findings, the effects of tool orientation, task goal and cooperation were replicated with novel tools. It follows that semantic tool knowledge is not critical for the ESC effect to occur. In fact, we found an "habitual" effect: Participant adopted uncomfortable grips with familiar tools even when it was not necessary (i.e., to transport them), probably because of the interference of habitual movement programming with actual movement programming. A cognitive view of movement planning is proposed, according to which goal comprehension (1) may rely on semantic tool knowledge, technical reasoning, and/or social skills, (2) defines end-state configuration, which in turn (3) calibrates beginning-state comfort and hence the occurrence of the ESC effect.

Choice of end-state comfort is dependent upon the time spent at the beginning-state and the precision requirement of the end-state

Choice of posture while grasping an object typically depends upon several factors including the time spent in that posture, what postures were held prior to choosing that posture, and the precision required by the posture. The purpose of this study was to test choice of end-state thumb-up posture based on time spent at the beginning-state and the precision requirement of the end-state. To determine choice of thumb-up based on time or precision, we varied how long a subject had to hold the beginning state before moving an object to an end location. We made end-state precision either small or large and eliminated the precision needed to stand the object up at the end of the movement. A choice between "comfort" at the beginning or precision at the end-state would be demanded by the conditions with long beginning-state hold times and high precision demands. We aimed to determine which aspect of movement was of greater importance to individuals, overall "comfort" or precision. When the requirement was to hold the initial grasp longer, and the end-target was large, we predicted that we would see more thumb-up postures adopted at the beginning state. When the final placement was small and the initial posture was not constrained, we predicted we would see thumb-up postures adopted at the end state. On average, we found that, as beginning-state grasp time increased, more individuals chose beginning-state thumb-up postures. Perhaps, not surprisingly, we found distinct individual differences within our sample. Some individuals seemed to choose beginning-state thumb-up postures nearly 100% of the time, while other individuals chose end-state thumb-up postures nearly 100% of the time. Both the time spent in a posture and its precision requirements influenced planning, but not necessarily in a systematic way.

The ontogeny of efficient second-order action planning: The developing interplay of controlled and habitual processes in goal-directed actions

The ability to act efficiently plays an important role in everyday human life. The current study investigated efficient motor planning in 2- to 14-year-old children and adults (N = 246) in two different object manipulation tasks that involved everyday objects (a cup and a spoon). Importantly, we manipulated whether or not the efficient controlled grasp was incongruent with the habitual use of the object. We assessed to what extent participants planned their grasping action in an anticipatory controlled manner or relied on the habitual use of an object. We found the ability of efficient movement planning to be correlated between the two conceptually different tasks. Furthermore, the interplay of controlled and habitual processes of action control showed different developmental patterns for the two tasks and does not indicate a simple linear developmental trend. Thus, this study expands our knowledge on the ontogeny of efficient motor planning and highlights the developmental dynamics of the interplay of controlled and habitual processes in goal-directed action control.

Children's planning of efficient tool use in a social context

Efficient joint action requires that we anticipate situational demands both regarding our own and another person's perspective, and adapt our actions accordingly. Accordingly, when handing over a tool somebody else, it is advantageous to anticipate our future hand orientation (motor imagery), as well as the future orientation of the tool (mental rotation) relative to the other person, in order to make the transfer as smooth and efficient as possible. Furthermore, familiarity with specific tools might facilitate planning. We tested thirty-two 5.5- to 7-year-old children on a tool transfer task, asking if they consider another person's comfort when handing over different tools, and whether tool familiarity, motor imagery, and mental rotation are related to their grip choices. We compared the children's performance to that of an adult control group. Besides a rather low performance on the transfer task, we found differences in children's consideration of another person's comfort related to the specific tools they interacted with. Specifically, the unfamiliar tool (a bar) was transferred more efficiently than the familiar tools (hammer/brush). In addition, the results suggest a relation between children's consideration of another person's comfort and their mental rotation score, but no relation with their motor imagery score.

Hand preference for unimanual and bimanual tasks: Evidence from questionnaires and preferential reaching

The current research compared hand selection in a preferential reaching paradigm with unimanual (i.e., pick-up cup) and bimanual (pick-up cup and pour from pitcher) tasks. In addition, relationships between self-report, questionnaire-based hand preference (unimanual and bimanual) and patterns of hand selection were assessed. Data offer support for a division of labour between the hands in at the midline; however, bimanual selection otherwise reflects consideration of object proximity (i.e., location) and comfort (i.e., biomechanical constraints). When grasping cups in right space, the right-hand was used to stabilize the cup and left-hand to mobilize the pitcher, whereas the opposite pattern was observed in left-space. Unimanual hand selection was also driven by object location. Subsequent analyses revealed a relationship between unimanual measures, but not bimanual measures of hand preference. Overall, findings support the notion that questionnaire data are associated with hand preference for grasping to a certain extent; however, use of a comprehensive battery of assessments is recommended when assessing and/or predicting handedness.

Neural Correlates of Hand-Object Congruency Effects during Action Planning

Selecting hand actions to manipulate an object is affected both by perceptual factors and by action goals. Affordances may contribute to "stimulus-response" congruency effects driven by habitual actions to an object. In previous studies, we have demonstrated an influence of the congruency between hand and object orientations on response times when reaching to turn an object, such as a cup. In this study, we investigated how the representation of hand postures triggered by planning to turn a cup was influenced by this congruency effect, in an fMRI scanning environment. Healthy participants were asked to reach and turn a real cup that was placed in front of them either in an upright orientation or upside-down. They were instructed to use a hand orientation that was either congruent or incongruent with the cup orientation. As expected, the motor responses were faster when the hand and cup orientations were congruent. There was increased activity in a network of brain regions involving object-directed actions during action planning, which included bilateral primary and extrastriate visual, medial, and superior temporal areas, as well as superior parietal, primary motor, and premotor areas in the left hemisphere. Specific activation of the dorsal premotor cortex was associated with hand-object orientation congruency during planning and prior to any action taking place. Activity in that area and its connectivity with the lateral occipito-temporal cortex increased when planning incongruent (goal-directed) actions. The increased activity in premotor areas in trials where the orientation of the hand was incongruent to that of the object suggests a role in eliciting competing representations specified by hand postures in lateral occipito-temporal cortex.

The influence of object size on second-order planning in an overturned cup task

For children, second-order planning in object manipulation, as demonstrated by the end-state comfort (ESC) effect, is constrained by numerous factors. As a step towards delineating the influence of object size, children (ages 6-10, n = 113) and adults (ages 18-25, n = 17) performed an overturned cup task with 8 cup sizes (4.7-8.4 cm diameter). Contrary to the hypothesis, cup size did not predict the overall proportion of trials with ESC. An increase in age and hand length did result in more ESC. Subsequent analyses with children, which focused on each cup size, revealed an increase in the ratio of hand size to cup size resulted in an increase in ESC with the 4.7, 5.2, 6.2, and 6.7 cm cups, age was a significant predictor with 4.7, 5.2, and 6.2 cm cups, and the interaction was significant for the 6.2 cm cup. No other significant results emerged. Together, findings offer preliminary support for the notion that object size does constrain children's motor planning skills to a certain degree (i.e., with large cups); however, further inquiry is warranted to elucidate the link between object size and hand size in children's performance of tasks requiring second-order planning.

Effect of object and human-factor characteristics on the preference of thumb-index finger grasp type

This work is to investigate the factors affecting the preference of human thumb-index finger grasping type. A multinomial logistic regression analysis shown that the object characteristics (equivalent diameter and shape) and human-factor characteristics (hand-used, finger-length sum and finger-length ratio) had significant contributions on the preference of thumb-index finger grasp type (p < 0.05) but the gender had not (p > 0.05). Subsequently, two mathematical equations were proposed for predicting the probability at which the precision-pinch and power-grasp were chosen for grasping an object. The probability at which the precision-pinch was chosen gradually decreased with the increase in the equivalent diameter of objects, but it is opposite for the power-grasp case. The shorter the finger-length sum, the more likely the participant was to select the power-grasp for grasping an object compared to the precision-pinch. The power-grasp was the most frequently chosen for the finger-length ratios of 1.0-1.25 and 1.75-2.0. Practitioner summary: This fruitful study gave explanation of the relationship between the object and human-factor characteristics and the preference of human thumb-index finger grasp type, which would be helpful to make intelligent grasping planning strategies for two-finger bionic mechanical hands.

The influence of time constraints on posture choices during an end-state comfort task

People adopt comfortable postures for the end states of motor actions (end-state comfort; Rosenbaum & Jorgensen, 1992). The choice to end comfortably often elicits adoption of uncomfortable beginning states, demonstrating that a sequence of movement is planned in advance of movement onset. Many factors influence the choice of comfortable end-state postures including the greater precision and speed afforded by postures at joint angle mid-ranges (Short & Cauraugh, 1999). To date, there has been little evaluation of the hypothesis that postures are chosen based on minimizing the time spent in uncomfortable postures. The aim of this experiment was to examine how the relative time required to hold beginning and end-state postures influenced the choice of posture. Participants moved a two-toned wooden dowel from one location to another with the requirement to grasp the object and place a specified color down. Participants completed four conditions where no postures were held, only one posture was held, or both postures were held. We predicted more thumb-up postures for positions held longer regardless of whether these postures were at the end or beginning state. Results verified that the constraint of holding the initial posture led to decreased end-state comfort supporting the hypothesis that estimation of time spent in postures is an important constraint in planning. We also note marked individual differences in posture choices, particularly when the object was moved to the left.

Factors Influencing Manipulation of a Familiar Object in Patients With Limb Apraxia After Stroke

Previous studies have shown that hand actions to visual objects are affected both by perceptual factors and by action goals. Our aim was to study how these processes affected hand actions in chronic stroke patients, based on whether they had limb apraxia. Twenty-two left hemisphere, chronic stroke patients were measured on neuropsychological tasks of limb apraxia, which was identified in a subgroup of 10 patients. All patients underwent testing on a separate task of making simple reach and grasp actions to a cup. Their performance was compared to a group of 18 healthy age-matched volunteers. Participants were instructed to grasp the top or bottom of a cup to either lift or turn it over so as to end with a hand position that was either comfortable or uncomfortable. This task tested the influence of the compatibility of hand–cup orientation, as well as goals driven by the end-state comfort of the hand, on action selection for object manipulation. Participants’ performance was measured in terms of error rates, and speed of initiation and reaching (movement time) to the object. The patients’ performance was significantly delayed, and error rates increased when reaching to grasp a cup under conditions of poor compatibility and end-state comfort. The subgroup of patients with apraxia showed a decreased influence of compatibility of hand interaction with the cup, with increased error rates and delayed response times, compared to patients with no apraxia and healthy volunteers. This is despite the fact they did not display significant deficits on neuropsychological tasks of real object use. The study shows that patients with apraxia have difficulties in selecting elements of object-directed actions, pertaining to both habitual and goal-directed factors.

Hemispheric lateralization does not affect the cognitive and mechanical cost of a sequential motor task

In sequential, repetitive tasks, we often partially reuse former motor plans. This causes a persistence of an earlier adopted posture (termed motor hysteresis). The cost-optimization hypothesis states that a partial reuse reduces the cognitive cost of a movement, while the persistence in a former posture increases its mechanical cost. An optimal fraction of reuse, which depends on the relative cognitive and mechanical cost, minimizes the total movement cost. Several studies postulate differences in mechanical or cognitive cost as a result of hemispheric lateralization. In the current study, we asked whether these differences would result in different fractions of motor plan reuse. To this end, left- and right-handed dominant participants executed a sequential motor task (opening a column of drawers) with their dominant and non-dominant hand. The size of the motor hysteresis effect was measured as a proxy for the fraction of plan reuse. Participants used similar postures and exhibited a similar hysteresis effect, irrespective of hand and handedness. This finding indicates that either the cognitive and mechanical costs of a motor task are unaffected by hemispheric differences or that their effect on motor planning is negligible.

Investigating motor planning in children with DCD: Evidence from simple and complex grip-selection tasks

Several studies suggest that children with Developmental Coordination Disorder (DCD) may be able to plan simple movements as well as their peers, but experience increasing difficulties as the movements become complex. The present study aimed to clarify the nature of motor planning in DCD, including a putative deficit, by being the first to investigate motor planning using converging measures of simple and complex motor planning in a single sample of children with DCD. Boys aged between 8 and 12 years with (n = 10) and without DCD (n = 17) completed three commonly used 'simple' (bar grasping, sword, and bar transport tasks) measures and one 'complex' (octagon task) measure of end-state-comfort (ESC), a classic measurement of motor planning ability. To achieve ESC when manipulating an object, a person may choose to start with an uncomfortable grip in order to end the movement in a comfortable position. Results indicate that the participants with DCD planned for ESC as efficiently as their peers when performing the 'simple' measures of ESC but were significantly less likely to end their performances in ESC than those without DCD for the more 'complex' octagon task. Taken together, our data suggest that school-aged children with DCD may be able to plan simple movements as efficiently as their peers, but have more difficulty doing so for multi-movement or complex sequences. Based on the assumption that the efficiency of such motor planning is dependent on the integrity of internal modelling systems, we argue that our study provides indirect support for the internal modelling deficit hypothesis.

Switching between lift and use grasp actions

Switching between competing grasp postures incurs costs on speeded performance. We examined switch costs between lift versus use actions under task conditions that required subjects to identify familiar objects. There were no asymmetrical interference effects, though reliable costs occurred when the same object required a different action on consecutive trials. In addition, lift actions were faster to objects targeted for a prospective use action than objects irrelevant to this intended goal. The benefit of a lift-then-use action sequence was not merely due to the production of two different actions in short order on the same object; use actions to an object marked for the distal goal of a lift action were not faster than use actions applied to another object. We propose that the intention to use an object facilitates the prior action of lifting it because the motor sequence lift-then-use is habitually conscripted to enact the proper function of an object.

A direct effect of perception on action when grasping a cup

Affordances represent features of an object that trigger specific actions. Here we tested whether the presence and orientation of a handle on a cup could bias grasping movements towards it in conditions where subjects were explicitly told to ignore the handle. We quantified the grip aperture profile of twelve healthy participants instructed to grasp a cup from its body while it either had no handle, a handle pointing towards, or away from the grasping hand (3 ‘move’ conditions, with large grip aperture). To ensure the smaller grip aperture afforded by the handle was implicitly processed, we interspersed trials in which participants had to grasp the cup from its handle or a handle not attached to a cup with a small grip aperture. We found that grip aperture was smaller in the presence of a handle in the ‘move’ conditions, independently of its orientation. Our finding, of an effect of the handle during the execution of a grasp action, extends previous evidence of such an influence measured during motor preparation using simple reaction times. It suggests that the specific action elicited by an object’s attribute can affect movement performance in a sustained manner throughout movement execution.

End-State Comfort Across the Lifespan: A Cross-Sectional Investigation of How Movement Context Influences Motor Planning in an Overturned Glass Task

Young adults plan actions in advance to minimize the cost of movement. This is exemplified by the end-state comfort (ESC) effect. A pattern of improvement in ESC in children is linked to the development of cognitive control processes, and decline in older adults is attributed to cognitive decline. This study used a cross-sectional design to examine how movement context (pantomime, demonstration with image/glass as a guide, actual grasping) influences between-hand differences in ESC planning. Children (5- to 12-year-olds), young adults, and two groups of older adults (aged 60-70, and aged 71 and older) were assessed. Findings provide evidence for adult-like patterns of ESC in 8-year-olds. Results are attributed to improvements in proprioceptive acuity and proficiency in generating and implementing internal representations of action. For older adults early in the aging process, sensitivity to ESC did not differ from young adults. However, with increasing age, differences reflect challenges in motor planning with increases in cognitive demand, similar to previous work. Findings have implications for understanding lifespan motor behavior.

Movement Interferes with Visuospatial Working Memory during the Encoding: An ERP Study

The present study focuses on the functional interactions of cognition and manual action control. Particularly, we investigated the neurophysiological correlates of the dual-task costs of a manual-motor task (requiring grasping an object, holding it, and subsequently placing it on a target) for working memory (WM) domains (verbal and visuospatial) and processes (encoding and retrieval). Thirty participants were tested in a cognitive-motor dual-task paradigm, in which a single block (a verbal or visuospatial WM task) was compared with a dual block (concurrent performance of a WM task and a motor task). Event-related potentials (ERPs) were analyzed separately for the encoding and retrieval processes of verbal and visuospatial WM domains both in single and dual blocks. The behavioral analyses show that the motor task interfered with WM and decreased the memory performance. The performance decrease was larger for the visuospatial task compared with the verbal task, i.e., domain-specific memory costs were obtained. The ERP analyses show the domain-specific interference also at the neurophysiological level, which is further process-specific to encoding. That is, comparing the patterns of WM-related ERPs in the single block and dual block, we showed that visuospatial ERPs changed only for the encoding process when a motor task was performed at the same time. Generally, the present study provides evidence for domain- and process-specific interactions of a prepared manual-motor movement with WM (visuospatial domain during the encoding process). This study, therefore, provides an initial neurophysiological characterization of functional interactions of WM and manual actions in a cognitive-motor dual-task setting, and contributes to a better understanding of the neuro-cognitive mechanisms of motor action control.

Neurophysiology of Grasping Actions: Evidence from ERPs

We use our hands very frequently to interact with our environment. Neuropsychology together with lesion models and intracranial recordings and imaging work yielded important insights into the functional neuroanatomical correlates of grasping, one important function of our hands, pointing toward a functional parietofrontal brain network. Event-related potentials (ERPs) register directly electrical brain activity and are endowed with high temporal resolution but have long been assumed to be susceptible to movement artifacts. Recent work has shown that reliable ERPs can be obtained during movement execution. Here, we review the available ERP work on (uni) manual grasping actions. We discuss various ERP components and how they may be related to functional components of grasping according to traditional distinctions of manual actions such as planning and control phases. The ERP results are largely in line with the assumption of a parietofrontal network. But other questions remain, in particular regarding the temporal succession of frontal and parietal ERP effects. With the low number of ERP studies on grasping, not all ERP effects appear to be coherent with one another. Understanding the control of our hands may help to develop further neurocognitive theories of grasping and to make progress in prosthetics, rehabilitation or development of technical systems for support of human actions.

Habit outweighs planning in grasp selection for object manipulation

Object-directed grasping movements are adapted to intended interactions with an object. We address whether adjusting the grasp for object manipulation is controlled habitually, based on past experiences, or by goal-directed planning, based on an evaluation of the expected action outcomes. Therefore, we asked participants to grasp and rotate a dial. In such tasks, participants typically grasp the dial with an excursed, uncomfortable arm posture, which then allows to complete the dial rotation in a comfortable end-state. We extended this task by manipulating the contingency between the orientation of the grasp and the resulting end-state of the arm. A one-step (control) group rotated the dial to a single target. A two-step group rotated the dial to an initial target and then in the opposite direction. A three-step group rotated the dial to the initial target, then in the opposite direction, and then back to the initial target. During practice, the two-step and three-step groups reduced the excursion of their grasps, thus avoiding overly excursed arm postures after the second rotation. When the two-step and three-step groups were asked to execute one-step rotations, their grasps resembled those that were acquired during the two-step and three-step rotations, respectively. However, the carry-over was not complete. This suggests that adjusting grasps for forthcoming object manipulations is controlled by a mixture of habitual and goal-directed processes. In the present experiment, the former contributed approximately twice as much to grasp selection than the latter.

The development of tool use: Planning for end-state comfort

Some grips on the handle of a tool can be planned on the basis of information directly available in the scene. Other grips, however, must be planned on the basis of the final position of the hand. "End-state comfort" grips require an awkward or uncomfortable initial grip so as to later implement the action comfortably and efficiently. From a cognitive perspective, planning for end-state comfort requires a consistent representation of the entire action sequence, including the latter part, which is not based on information directly available in the scene. Many investigators have found that young children fail to demonstrate planning for end-state comfort and that adultlike performance does not appear until about 12 years of age. In 2 experiments, we used a hammering task that engaged children in a goal-directed action with multiple steps. We assessed end-state-comfort planning in novel ways by measuring children's hand choice, grip choice, and tool implementation over multiple trials. The hammering task also uniquely allowed us to assess the efficiency of implementation. We replicated the previous developmental trend in 4-, 8-, and 12-year-old children with our novel task. Most important, our data revealed that 4-year-olds are in a transitional stage during which several competing strategies were exhibited during a single session. Preschoolers changed their grip within trials and across trials, indicating awareness of errors and a willingness to sacrifice speed for more efficient implementation. The end-state-comfort grip initially competes as one grip type among many but gradually displaces all others. Children's sensitivity to costs and drive for efficiency may motivate this change. (PsycINFO Database Record

Ownership Status Influences the Degree of Joint Facilitatory Behavior

When engaging in joint activities, humans tend to sacrifice some of their own sensorimotor comfort and efficiency to facilitate a partner’s performance. In the two experiments reported here, we investigated whether ownership—a socioculturally based nonphysical feature ascribed to objects—influenced facilitatory motor behavior in joint action. Participants passed mugs that differed in ownership status across a table to a partner. We found that participants oriented handles less toward their partners when passing their own mugs than when passing mugs owned by their partners (Experiment 1) and mugs owned by the experimenter (Experiment 2). These findings indicate that individuals plan and execute actions that assist their partners but do so to a smaller degree if it is the individuals’ own property that the partners intend to manipulate. We discuss these findings in terms of underlying variables associated with ownership and conclude that a self-other distinction can be found in the human sensorimotor system.

Anticipatory Planning in Children with Autism Spectrum Disorder: An Assessment of Independent and Joint Action Tasks

Autism Spectrum Disorder (ASD) is one of the most common neurodevelopmental disorders. Although not a diagnostic feature, motor impairments have been recently acknowledged as prevalent and significant, such that these children have difficulties planning, organizing and coordinating movements. This study aimed to further investigate anticipatory motor planning in children with ASD by means of assessing end- and beginning-state comfort, considering inconsistent reports of end-state comfort in independent action, and the study of beginning-state comfort being limited to one study with young adults. Five- to eleven-year-old children with ASD, and chronologically age- and sex-matched typically-developing children picked-up a glass and: (1) poured a cup of water; and (2) passed it to the researcher to pour a cup of water. End-state comfort was deemed evident if participants grasped the glass thumb-down followed by a 180° rotation; therefore ending with a thumb-up posture. Beginning-state comfort was deemed evident if participants passed the glass to the researcher oriented upright. Findings revealed less end-state comfort in children with ASD, attributed to motor planning deficits. Beginning-state comfort did not differ, ascribed to the habitual nature of the task; therefore reflecting a stimulus-driven response as opposed to an action which reflects anticipatory planning. The findings support difficulties with motor planning and control for children with ASD in an independent task. However, when acting with a familiar object in joint action, behavior does not differ, likely indicative of a habitual, stimulus-driven response.

Toward a Unified Sub-symbolic Computational Theory of Cognition

This paper proposes how various disciplinary theories of cognition may be combined into a unifying, sub-symbolic, computational theory of cognition. The following theories are considered for integration: psychological theories, including the theory of event coding, event segmentation theory, the theory of anticipatory behavioral control, and concept development; artificial intelligence and machine learning theories, including reinforcement learning and generative artificial neural networks; and theories from theoretical and computational neuroscience, including predictive coding and free energy-based inference. In the light of such a potential unification, it is discussed how abstract cognitive, conceptualized knowledge and understanding may be learned from actively gathered sensorimotor experiences. The unification rests on the free energy-based inference principle, which essentially implies that the brain builds a predictive, generative model of its environment. Neural activity-oriented inference causes the continuous adaptation of the currently active predictive encodings. Neural structure-oriented inference causes the longer term adaptation of the developing generative model as a whole. Finally, active inference strives for maintaining internal homeostasis, causing goal-directed motor behavior. To learn abstract, hierarchical encodings, however, it is proposed that free energy-based inference needs to be enhanced with structural priors, which bias cognitive development toward the formation of particular, behaviorally suitable encoding structures. As a result, it is hypothesized how abstract concepts can develop from, and thus how they are structured by and grounded in, sensorimotor experiences. Moreover, it is sketched-out how symbol-like thought can be generated by a temporarily active set of predictive encodings, which constitute a distributed neural attractor in the form of an interactive free-energy minimum. The activated, interactive network attractor essentially characterizes the semantics of a concept or a concept composition, such as an actual or imagined situation in our environment. Temporal successions of attractors then encode unfolding semantics, which may be generated by a behavioral or mental interaction with an actual or imagined situation in our environment. Implications, further predictions, possible verification, and falsifications, as well as potential enhancements into a fully spelled-out unified theory of cognition are discussed at the end of the paper.

The influence of action execution on end-state comfort and underlying movement kinematics: An examination of right and left handed participants

People typically move in an anticipatory manner, planning the intended action in advance to minimize the energy costs associated with producing the action (e.g., Rosenbaum et al., 2009). This is exemplified behaviorally in the end-state comfort effect, which is characterized by the selection of an uncomfortable initial posture to enable a comfortable posture upon completion of the movement (Rosenbaum et al., 1990). The main objective of this study was to further investigate the end-state comfort effect in left- and right-handers (N=20). More specifically, to: (a) understand the influence of mode of action execution; and (b) delineate the role of handedness. The overturned glass task (Fischman, 1997) was used as means of assessment, where participants were asked to demonstrate picking up a glass to pour water in four modes of execution: (1) pantomime without a stimulus; (2) pantomime with image of the glass as a guide; (3) pantomime with glass as a guide; and (4) grasping the glass. End-state comfort was displayed regardless of mode of execution, hand used to complete the task or handedness group. However, kinematic analysis revealed distinct differences, highlighting how movement parameters are altered as a result the mode of action execution.

Too much anticipation? Large anticipatory adjustments of grasping movements to minimal object manipulations

When humans grasp objects, the grasps foreshadow the intended object manipulation. It has been suggested that grasps are selected that lead to medial arm postures, which facilitate movement speed and precision, during critical phases of the object manipulation. In Experiment 1, it has been tested whether grasp selections lead to medial postures during rotations of a dial. Participants twisted their arms considerably before grasping the dial, even when the upcoming dial rotation was minimal (5°). Participants neither assumed a medial posture at any point during a short rotation, nor did they assume any of the postures involved in short rotations in the opposite direction. Thus, grasp selections did not necessarily lead to specific postures at any point of the object manipulation. Experiment 2 examined the effect of various grasps on the speed of dial rotations. A medial initial grasp resulted in the fastest dial rotations for most rotation angles. Spontaneously selected grasps were more excursed than necessary to maximize dial rotation speed. This apparent overshot might be explained by participants' sensitive to the variability of their grasps and is in line with the assumption that grasps facilitate control over the grasped object.

Manual (a)symmetries in grasp posture planning: a short review

Many activities of daily living require that we physically interact with one or more objects. Object manipulation provides an intriguing domain in which the presence and extent of manual asymmetries can be studied on a motor planning and a motor execution level. In this literature review we present a state of the art for manual asymmetries at the level of motor planning during object manipulation. First, we introduce pioneering work on grasp posture planning. We then sketch the studies investigating the impact of future task demands during unimanual and bimanual object manipulation tasks in healthy adult populations. In sum, in contrast to motor execution, there is little evidence for hand-based performance differences in grasp posture planning. We discuss potential reasons for the lack of manual asymmetries in motor planning and outline potential avenues of future research.

Goal-oriented gaze strategies afforded by object interaction

Task influence has long been known to play a major role in the way our eyes scan a scene. Yet most studies focus either on visual search or on sequences of active tasks in complex real world scenarios. Few studies have contrasted the distribution of eye fixations during viewing and grasping objects. Here we address how attention is deployed when different actions are planned on objects, in contrast to when the same objects are categorized. In this respect, we are particularly interested in the role every fixation plays in the unfolding dynamics of action control. We conducted an eye-tracking experiment in which participants were shown images of real-world objects. Subjects were either to assign the displayed objects to one of two classes (categorization task), to mimic lifting (lifting task), or to mimic opening the object (opening task). Results suggest that even on simplified, two dimensional displays the eyes reveal the participant's intentions in an anticipatory fashion. For the active tasks, already the second saccade after stimulus onset was directed towards the central region between the two locations where the thumb and the rest of the fingers would be placed. An analysis of saliency at fixation locations showed that fixations in active tasks have higher correspondence with salient features than fixations in the passive task. We suggest that attention flexibly coordinates visual selection for information retrieval and motor planning, working as a gateway between three components, linking the task (action), the object (target), and the effector (hand) in an effective way.

Habitual vs non-habitual manual actions: an ERP study on overt movement execution

This study explored the neurophysiological mechanisms underlying the planning and execution of an overt goal-related handle rotation task. More specifically, we studied the neural basis of motor actions concerning the influence of the grasp choice. The aim of the present study was to differentiate cerebral activity between grips executed in a habitual and a non-habitual mode, and between specified and free grip choices. To our knowledge, this is the first study to differentiate cerebral activity underlying overt goal-related actions executed with a focus on the habitual mode. In a handle rotation task, participants had to use thumb-toward (habitual) or thumb-away (non-habitual) grips to rotate a handle to a given target position. Reaction and reach times were shorter for the habitual compared to the non-habitual mode indicating that the habitual mode requires less cognitive processing effort than the non-habitual mode. Neural processes for action execution (measured by event-related potentials (ERPs)) differed between habitual and non-habitual conditions. We found differential activity between habitual and non-habitual conditions in left and right frontal areas from −600 to 200 ms time-locked to reaching the target position. No differential neural activity could be traced for the specification of the grip. The results suggested that the frontal negativity reflected increased difficulty in movement precision control in the non-habitual mode compared to the habitual mode during the homing in phase of grasp and rotation actions.

The contribution of cognitive, kinematic, and dynamic factors to anticipatory grasp selection

Object-directed grasping movements are usually adjusted in anticipation of the direction and extent of a subsequent object rotation. Such anticipatory grasp selections have been mostly explained in terms of the kinematics of the arm movement. However, object rotations of different directions and extents also differ in their dynamics and in how the tasks are represented. Here, we examined how the dynamics, the kinematics, and the cognitive representation of an object manipulation affect anticipatory grasp selections. We asked participants to grasp an object and rotate it by different angles and in different directions. To examine the influence of dynamic factors, we varied the object's weight. To examine the influence of the cognitive task representation, we instructed identical object rotations as either toward-top or away-from-top rotations. While instructed object rotation and cognitive task representation did affect grasp selection over the entire course of the experiment, a rather small effect of object weight only appeared late in the experiment. We suggest that grasp selections are determined on different levels. The representation of the kinematics of the object movement determines grasp selection on a trial-by-trial basis. The effect of object weight affects grasp selection by a slower adaptation process. This result implies that even simple motor acts, such as grasping, can only be understood when cognitive factors, such as the task representation, are taken into account.

Perturbations in action goal influence bimanual grasp posture planning

The authors examined the effects of perturbations in action goal on bimanual grasp posture planning. Sixteen participants simultaneously reached for 2 cylinders and placed either the left or the right end of the cylinders into targets. As soon as the participants began their reaching movements, a secondary stimulus was triggered, which indicated whether the intended action goal for the left or right hand had changed. Overall, the tendency for a single hand to select end-state comfort compliant grasp postures was higher for the nonperturbed condition compared to both the perturbed left and perturbed right conditions. Furthermore, participants were more likely to plan their movements to ensure end-state comfort for both hands during nonperturbed trials, than perturbed trials, especially object end-orientation conditions that required the adoption of at least one underhand grasp posture to satisfy bimanual end-state comfort. Results indicated that when the action goal of a single object was perturbed, participants attempted to reduce the cognitive costs associated with grasp posture replanning by maintaining the original grasp posture plan, and tolerating grasp postures that result in less controllable final postures.

A Systematic Review of the End-State Comfort Effect in Normally Developing Children and in Children With Developmental Disorders

The end-state comfort (ESC) effect signifies the tendency to avoid uncomfortable postures at the end of goal-directed movements and can be reliably observed during object manipulation in adults, but only little is known about its development in children. The aim of the present paper is to provide a review of research on the ESC effect in normally developing children and in children with various developmental disorders, and to identify the factors constraining anticipatory planning skills. Three databases (Medline, Scopus, and PubMed) and relevant journals were scrutinized and a step-wise analysis procedure was employed to identify the relevant studies. Thirteen studies assessed the ESC effect in children, ranging from 1.5–14 years of age. Nine out of these thirteen studies reported the ESC effect to be present in normally developing children, but the results are inconsistent with regard to children’s age and the kind of ESC task used. Some evidence even suggests that these planning skills are intact in children with developmental disorders. Inconsistencies between findings are discussed in the light of moderating factors like the number of action steps, precision requirements, familiarity with the task, the task procedure, motivation, sample size, and age, as well as the cognitive and motor development of the participants. Further research is needed to investigate the onset and the developmental course of ESC planning, as well as the interdependencies with other cognitive abilities and sensory-motor skills.

Optimal control in the critical phase of movement: a functional approach to motor planning processes

Grasping movements are often planned in a way that they end in a position where joints are in an anatomically medial position. This behaviour is termed the "end-state comfort" (ESC) effect (Rosenbaum et al., 1990). We suggest that the anatomically medial position is favoured to control the most difficult part of the movement. In most experiments investigating ESC, objects have to be placed onto a target location, and the highest precision demand occurs at the end of the movement. Thus, ESC is confounded with movement difficulty. In this study, we dissociate movement difficulty and ESC. In our experiments, participants had to execute a task where the critical part of the movement was either at the end or at the beginning of the movement. Participants' grasping behaviour confirmed the hypothesis that movement planning is constrained by a goal for optimal control during the part of the movement that demands the highest precision, rather than by a goal to end in a comfortable state (Rosenbaum, Chapman, Weigelt, Weiss, & van der Wel, 2012). We identified recall and movement plan generating processes of motor planning (Cohen & Rosenbaum, 2004), that ensure the optimal control in the critical part of movement. Our results indicate that recall processes depend on motor experience which is acquired in different time scales. We suggest that motor planning processes are triggered only if the costs for executing movements controlled by recall processes exceed the costs for generating a motor plan.

Too Good to be True? Ideomotor Theory from a Computational Perspective

In recent years, Ideomotor Theory has regained widespread attention and sparked the development of a number of theories on goal-directed behavior and learning. However, there are two issues with previous studies' use of Ideomotor Theory. Although Ideomotor Theory is seen as very general, it is often studied in settings that are considerably more simplistic than most natural situations. Moreover, Ideomotor Theory's claim that effect anticipations directly trigger actions and that action-effect learning is based on the formation of direct action-effect associations is hard to address empirically. We address these points from a computational perspective. A simple computational model of Ideomotor Theory was tested in tasks with different degrees of complexity. The model evaluation showed that Ideomotor Theory is a computationally feasible approach for understanding efficient action-effect learning for goal-directed behavior if the following preconditions are met: (1) The range of potential actions and effects has to be restricted. (2) Effects have to follow actions within a short time window. (3) Actions have to be simple and may not require sequencing. The first two preconditions also limit human performance and thus support Ideomotor Theory. The last precondition can be circumvented by extending the model with more complex, indirect action generation processes. In conclusion, we suggest that Ideomotor Theory offers a comprehensive framework to understand action-effect learning. However, we also suggest that additional processes may mediate the conversion of effect anticipations into actions in many situations.

The end-state comfort effect in 3- to 8-year-old children in two object manipulation tasks

The aim of the study was to compare 3- to 8-year-old children's propensity to anticipate a comfortable hand posture at the end of a grasping movement (end-state comfort effect) between two different object manipulation tasks, the bar-transport task, and the overturned-glass task. In the bar-transport task, participants were asked to insert a vertically positioned bar into a small opening of a box. In the overturned-glass task, participants were asked to put an overturned-glass right-side-up on a coaster. Half of the participants experienced action effects (lights) as a consequence of their movements (AE groups), while the other half of the participants did not (No-AE groups). While there was no difference between the AE and No-AE groups, end-state comfort performance differed across age as well as between tasks. Results revealed a significant increase in end-state comfort performance in the bar-transport task from 13% in the 3-year-olds to 94% in the 8-year-olds. Interestingly, the number of children grasping the bar according to end-state comfort doubled from 3 to 4 years and from 4 to 5 years of age. In the overturned-glass task an increase in end-state comfort performance from already 63% in the 3-year-olds to 100% in the 8-year-olds was significant as well. When comparing end-state comfort performance across tasks, results showed that 3- and 4-year-old children were better at manipulating the glass as compared to manipulating the bar, most probably, because children are more familiar with manipulating glasses. Together, these results suggest that preschool years are an important period for the development of motor planning in which the familiarity with the object involved in the task plays a significant role in children's ability to plan their movements according to end-state comfort.

Dissociable brain systems mediate vicarious learning of stimulus-response and action-outcome contingencies

Two distinct strategies have been suggested to support action selection in humans and other animals on the basis of experiential learning: a goal-directed strategy that generates decisions based on the value and causal antecedents of action outcomes, and a habitual strategy that relies on the automatic elicitation of actions by environmental stimuli. In the present study, we investigated whether a similar dichotomy exists for actions that are acquired vicariously, through observation of other individuals rather than through direct experience, and assessed whether these strategies are mediated by distinct brain regions. We scanned participants with functional magnetic resonance imaging while they performed an observational learning task designed to encourage either goal-directed encoding of the consequences of observed actions, or a mapping of observed actions to conditional discriminative cues. Activity in different parts of the action observation network discriminated between the two conditions during observational learning and correlated with the degree of insensitivity to outcome devaluation in subsequent performance. Our findings suggest that, in striking parallel to experiential learning, neural systems mediating the observational acquisition of actions may be dissociated into distinct components: a goal-directed, outcome-sensitive component and a less flexible stimulus-response component.

The end-state comfort effect facilitates joint action

Motor experts can accurately predict the future actions of others by observing their movements. This report describes three experiments that investigate such predictions in everyday object manipulations and test whether these predictions facilitate responses to the actions of others. Observing video excerpts showing an actor reaching for a vertically mounted dial, participants in Experiment 1 needed to predict how the actor would rotate it. Their predictions were specific to the direction and extent of the dial rotation and improved proportionate to the length of the video clip shown. Testing whether such predictions facilitate responses, in the subsequent experiments responders had to undo an actor's actions, back-rotating a dial (Exp 2) and a bar (Exp 3). The responders' actions were initiated faster when the actors' movements obeyed the so-called end-state comfort principle than when they did not. Our experiments show that humans exploit the end-state comfort effect to tweak their predictions of the future actions of others. The results moreover suggest that the precision of these predictions is mediated by perceptual learning rather than by motor simulation.

Where to Grasp a Tool?

Biomechanical and environmental constraints limit body movements and tool use actions. However, in the case of tool use, such constraints can often be overcome by adjusting a tool’s tool transformation to the requirements of the intended tool use action. The research presented here examined whether participants grasped a lever at different positions, thus modifying the lever’s tool transformation, to accommodate speed and accuracy requirements of different tasks. Participants were asked to quickly track a sequence of targets with the lever. If accuracy requirements were high, participants compensated for limits in the accuracy of hand movements by grasping the lever at a position that enabled precise control of the lever. If accuracy requirements were low, participants compensated for limits in hand speed by grasping the lever at a position that enabled fast lever movements with comparatively slow hand movements. This task-dependent grasp selection was only present after participants had practiced the tasks. The data show that in addition to adapting to fixed tool transformations, participants also actively controlled tool transformations to facilitate tool use actions.