Choosing between alternative wrist postures: Action planning needs perception

Grasping with a Twist: Dissociating Action Goals from Motor Actions in Human Frontoparietal Circuits

In daily life, prehension is typically not the end goal of hand-object interactions but a precursor for manipulation. Nevertheless, functional MRI (fMRI) studies investigating manual manipulation have primarily relied on prehension as the end goal of an action. Here, we used slow event-related fMRI to investigate differences in neural activation patterns between prehension in isolation and prehension for object manipulation. Sixteen (seven males and nine females) participants were instructed either to simply grasp the handle of a rotatable dial (isolated prehension) or to grasp and turn it (prehension for object manipulation). We used representational similarity analysis (RSA) to investigate whether the experimental conditions could be discriminated from each other based on differences in task-related brain activation patterns. We also used temporal multivoxel pattern analysis (tMVPA) to examine the evolution of regional activation patterns over time. Importantly, we were able to differentiate isolated prehension and prehension for manipulation from activation patterns in the early visual cortex, the caudal intraparietal sulcus (cIPS), and the superior parietal lobule (SPL). Our findings indicate that object manipulation extends beyond the putative cortical grasping network (anterior intraparietal sulcus, premotor and motor cortices) to include the superior parietal lobule and early visual cortex.SIGNIFICANCE STATEMENT A simple act such as turning an oven dial requires not only that the CNS encode the initial state (starting dial orientation) of the object but also the appropriate posture to grasp it to achieve the desired end state (final dial orientation) and the motor commands to achieve that state. Using advanced temporal neuroimaging analysis techniques, we reveal how such actions unfold over time and how they differ between object manipulation (turning a dial) versus grasping alone. We find that a combination of brain areas implicated in visual processing and sensorimotor integration can distinguish between the complex and simple tasks during planning, with neural patterns that approximate those during the actual execution of the action.

Dual-task interference in action programming and action planning - Evidence from the end-state comfort effect

In the present study, we examined the extent of interference between a cognitive task (auditory n-back task) and different aspects of motor performance. Specifically, we wanted to find out whether such interference is more pronounced for aspects of planning as compared to programming. Here, motor planning is represented by a phenomenon called the "end-state comfort effect", the fact that we tolerate uncomfortable initial postures in favour of a more comfortable final posture. We asked participants to grasp differently sized cylindrical objects and to place them on target platforms of varying height (grasp-and-place task), So, participants were required to (1) adjust their hand opening to the object width (action programming) and (2) to plan whether to grasp the object higher or lower in order to be able to place it comfortably onto the low or high target platform. We found that participants demonstrated the end-state comfort effect by anticipating the final posture und planning the movement accordingly with a higher object-grasp for low end-target position and lower object-grasp height for high end-target position, respectively. The auditory task was negatively affected by having to perform a visuomotor task in parallel, suggesting that the two tasks share cognitive and attentional resources. No significant impact from the auditory task on the motor tasks was found. Accordingly, it was not possible to determine which of the two motor aspects (programming or planning) contributed more towards the interference observed in the auditory task. To address this question, we carried out a second experiment. For this second experiment we focussed on the interference effects found in the auditory task and contrasted two versions of the grasp-and-place task. In the first version of the task, the height of the target-shelf varied from trial-to-trial but the width of the target object remained the same. We assumed that this version had high planning demands and low programming demands. In the second version the width of the target object varied and the target-shelf height remained constant. Presumably this increased programming demands but reduced planning demands. Significant interference with the auditory task was only found for the first version, supporting the hypothesis that motor planning requires more cognitive resources and thus creates higher multitasking costs.

Perceptuo-motor planning during functional reaching after stroke

In healthy young adults, reaching movements are planned such that the initial grasp position on the object is modulated based on the final task goal. This perceptuo-motor coupling has been described as the end-state comfort effect. This study aimed to determine the extent to which visuo-perceptual and motor deficits, but not neglect, due to stroke impact end-state comfort measured as the grasp-height effect. Thirty-four older adults (17 controls, 17 chronic stroke) performed a functional goal-directed two-sequence task with each arm, consisting of reaching and moving a cylindrical object (drain plunger) from an initial to four target platform heights, standardized to body height, in a block randomized sequence. Arm motor impairment (Fugl-Meyer Assessment) and visual–perceptual deficits (Motor-Free Visual Perception Test) were assessed in stroke subjects, and arm and trunk kinematics were assessed in all subjects. The primary outcome measure of the grasp-height effect was the relationship between the grasp heights used at the home position and the final target platform heights. Mixed model analysis was used for data analysis. The grasp-height effect was present in all participants, but decreased in stroke subjects with visuo-perceptual impairments compared to controls. In stroke subjects with sensorimotor impairments alone, indicated by altered kinematics, the grasp-height effect was comparable to controls. This first study examining the grasp-height effect in individuals with stroke provides new knowledge of the impact of visuo-perceptual deficits on movement planning and execution, which may assist clinicians in selecting more effective treatment strategies to improve perceptuo-motor skills and enhance motor recovery.

Independent Causal Contributions of Alpha- and Beta-Band Oscillations during Movement Selection

To select a movement, specific neuronal populations controlling particular features of that movement need to be activated, whereas other populations are downregulated. The selective (dis)inhibition of cortical sensorimotor populations is governed by rhythmic neural activity in the alpha (8-12 Hz) and beta (15-25 Hz) frequency range. However, it is unclear whether and how these rhythms contribute independently to motor behavior. Building on a recent dissociation of the sensorimotor alpha- and beta-band rhythms, we test the hypothesis that the beta-band rhythm governs the disinhibition of task-relevant neuronal populations, whereas the alpha-band rhythm suppresses neurons that may interfere with task performance. Cortical alpha- and beta-band rhythms were manipulated with transcranial alternating current stimulation (tACS) while human participants selected how to grasp an object. Stimulation was applied at either 10 or 20 Hz and was imposed on the sensorimotor cortex contralaterally or ipsilaterally to the grasping hand. In line with task-induced changes in endogenous spectral power, the effect of the tACS intervention depended on the frequency and site of stimulation. Whereas tACS stimulation generally increased movement selection times, 10 Hz stimulation led to relatively faster selection times when applied to the hemisphere ipsilateral to the grasping hand, compared with other stimulation conditions. These effects occurred selectively when multiple movements were considered. These observations functionally differentiate the causal contribution of alpha- and beta-band oscillations to movement selection. The findings suggest that sensorimotor beta-band rhythms disinhibit task-relevant populations, whereas alpha-band rhythms inhibit neuronal populations that could interfere with movement selection.

SIGNIFICANCE STATEMENT

This study shows dissociable effects of 10 Hz and 20 Hz tACS on the duration of movement selection. These observations have two elements of general relevance. First, the finding that alpha- and beta-band oscillations contribute independently to movement selection provides insight in how oscillations orchestrate motor behavior, which is key to understand movement selection deficits in neurodegenerative disorders. Second, the findings highlight the potential of 10 Hz stimulation as a neurophysiologically grounded intervention to enhance human performance. In particular, this intervention can potentially be exploited to boost rehabilitation after neural damage by targeting the unaffected hemisphere.

Through the forest of motor representations

Following neuroscience, and using different labels, several philosophers have addressed the idea of the presence of a single representational mechanism lying in between (visual) perceptual processes and motor processes involved in different functions and useful for shaping suitable action performances: a motor representation (MR). MRs are the naturalized mental antecedents of action. This paper presents a new, non-monolithic view of MRs, according to which, contrarily to the received view, when looking at in between (visual) perceptual processes and motor processes, we find not only a single representational mechanism with different functions, but an ensemble of different sub-representational phenomena, each of which with a different function. This new view is able to avoid several issues emerging from the literature and to address something the literature is silent about, which however turns out to be crucial for a theory of MRs.

The Extrastriate Body Area Computes Desired Goal States during Action Planning

How do object perception and action interact at a neural level? Here we test the hypothesis that perceptual features, processed by the ventral visuoperceptual stream, are used as priors by the dorsal visuomotor stream to specify goal-directed grasping actions. We present three main findings, which were obtained by combining time-resolved transcranial magnetic stimulation and kinematic tracking of grasp-and-rotate object manipulations, in a group of healthy human participants (N = 22). First, the extrastriate body area (EBA), in the ventral stream, provides an initial structure to motor plans, based on current and desired states of a grasped object and of the grasping hand. Second, the contributions of EBA are earlier in time than those of a caudal intraparietal region known to specify the action plan. Third, the contributions of EBA are particularly important when desired and current object configurations differ, and multiple courses of actions are possible. These findings specify the temporal and functional characteristics for a mechanism that integrates perceptual processing with motor planning.

Interactions between dorsal and ventral streams for controlling skilled grasp

The two visual systems hypothesis suggests processing of visual information into two distinct routes in the brain: a dorsal stream for the control of actions and a ventral stream for the identification of objects. Recently, increasing evidence has shown that the dorsal and ventral streams are not strictly independent, but do interact with each other. In this paper, we argue that the interactions between dorsal and ventral streams are important for controlling complex object-oriented hand movements, especially skilled grasp. Anatomical studies have reported the existence of direct connections between dorsal and ventral stream areas. These physiological interconnections appear to be gradually more active as the precision demands of the grasp become higher. It is hypothesised that the dorsal stream needs to retrieve detailed information about object identity, stored in ventral stream areas, when the object properties require complex fine-tuning of the grasp. In turn, the ventral stream might receive up to date grasp-related information from dorsal stream areas to refine the object internal representation. Future research will provide direct evidence for which specific areas of the two streams interact, the timing of their interactions and in which behavioural context they occur.

Catching moving objects: Differential effects of background motion on action mode selection and movement control in 6- to 10-month-old infants

In human adults the use of visual information for selecting appropriate modes for action appears to be separate from the use of visual information for the control of movements of which the action is composed (Milner & Goodale, [1995] The visual brain in action; [2008] Neuropsychologia 46:774-785). More specifically, action mode selection primarily relies upon allocentric information, whereas movement control mainly exploits egocentric information. In the present study, we investigated to what degree this division is already present in 6- to 10-month-old infants when reaching for moving objects; that is, whether allocentric information is uniquely exploited for action mode selection (i.e., reaching with one or the other hand) or whether it is also used for movement control (i.e., reaching kinematics). Infants were presented with laterally approaching objects at two speeds (i.e., 20 and 40 cm/s) against a stationary or moving background. Background motion affects allocentric information about the object's velocity relative to its background. Results indicated that object speed constrained both infants' action mode selection and movement control. Importantly, however, the influence of background motion was limited to action mode selection and did not extend to movement control. The findings provide further support for the contention that during early development information usage is--at least to some degree--separated for action mode selection and movement control.

Interactions between dorsal and ventral streams for controlling skilled grasp

The two visual systems hypothesis suggests processing of visual information into two distinct routes in the brain: a dorsal stream for the control of actions and a ventral stream for the identification of objects. Recently, increasing evidence has shown that the dorsal and ventral streams are not strictly independent, but do interact with each other. In this paper, we argue that the interactions between dorsal and ventral streams are important for controlling complex object-oriented hand movements, especially skilled grasp. Anatomical studies have reported the existence of direct connections between dorsal and ventral stream areas. These physiological interconnections appear to be gradually more active as the precision demands of the grasp become higher. It is hypothesised that the dorsal stream needs to retrieve detailed information about object identity, stored in ventral stream areas, when the object properties require complex fine-tuning of the grasp. In turn, the ventral stream might receive up to date grasp-related information from dorsal stream areas to refine the object internal representation. Future research will provide direct evidence for which specific areas of the two streams interact, the timing of their interactions and in which behavioural context they occur.

•

The dorsal and ventral streams are both involved in skilled grasping movements.

•

Ventral areas feed dorsal areas with information about object identity.

•

Grasps of increased complexity require gradually higher recruitment of ventral areas.

•

Dorsal stream inputs could fine tune object representations stored in ventral areas.

Choosing actions

Actions that are chosen have properties that distinguish them from actions that are not. Of the nearly infinite possible actions that can achieve any given task, many of the unchosen actions are irrelevant, incorrect, or inappropriate. Others are relevant, correct, or appropriate but are disfavored for other reasons. Our research focuses on the question of what distinguishes actions that are chosen from actions that are possible but are not. We review studies that use simple preference methods to identify factors that contribute to action choices, especially for object-manipulation tasks. We can determine which factors are especially important through simple behavioral experiments.

Cognition, action, and object manipulation

Although psychology is the science of mental life and behavior, little attention has been paid to the means by which mental life is translated into behavior. One domain in which links between cognition and action have been explored is the manipulation of objects. This article reviews psychological research on this topic, with special emphasis on the tendency to grasp objects differently depending on what one plans to do with the objects. Such differential grasping has been demonstrated in a wide range of object manipulation tasks, including grasping an object in a way that reveals anticipation of the object's future orientation, height, and required placement precision. Differential grasping has also been demonstrated in a wide range of behaviors, including 1-hand grasps, 2-hand grasps, walking, and transferring objects from place to place as well as from person to person. The populations in which the tendency has been shown are also diverse, including nonhuman primates as well as human adults, children, and babies. The tendency is compromised in a variety of clinical populations and in children of a surprisingly advanced age. Verbal working memory is compromised as well if words are memorized while object manipulation tasks are performed; the recency portion of the serial position curve is reduced in this circumstance. In general, the research reviewed here points to rich connections between cognition and action as revealed through the study of object manipulation. Other implications concern affordances, Donders' law, naturalistic observation, and the teaching of psychology.

Distinct cortical networks support the planning and online control of reaching-to-grasp in humans

A number of brain imaging studies have identified regions involved in the planning and control of complex actions. Here we attempt to contrast activity related to planning and online control in the human brain during simple reaching and grasping movements. In four conditions, participants did one of the following: passively observed a grasp target; planned a grasping movement without executing; planned and then executed a grasp; or immediately executed a grasp. Neural activity was measured using functional magnetic resonance imaging and activity in the various conditions compared. Two large, independent networks of brain activity were identified: (i) a planning network including the premotor cortex, basal ganglia, anterior cingulate, posterior medial parietal area, superior parietal occipital cortex and middle intraparietal sulcus; and (ii) a control network including sensorimotor cortex, the cerebellum, the supramarginal gyrus and the superior parietal lobule. These findings provide evidence that the planning and control of even simple reaching and grasping actions use different brain regions, including different parts of the frontal and parietal lobes.

Imagined tool-use in near and far space modulates the extra-striate body area

Active tool-use can result in the incorporation of the tool into the body schema, e.g., the representation of the arm is enlarged according to tool length. This modification even influences the processing of space: using a long tool leads to a remapping of far space as near space. We here further investigate the interaction of the neural representations of the human body, tool use, and spatial domain. Functional magnetic resonance imaging (fMRI) was performed in twelve right-handed healthy individuals while they imagined moving a cylinder towards a target position in far or near space by mentally using either pliers or a joystick. The fMRI data revealed that already the imagined use of preferred tools in near and far space (i.e., pliers in far space, joystick in near space) modulated the neural activity in the extra-striate body area (EBA) located in the occipito-temporal cortex. Moreover, psycho-physical interaction analysis showed that during imagined tool-use the functional connectivity of left EBA to a network representing the near-personal space around the hand was strengthened. This increased functional connectivity is likely to reflect the neural processes underlying the incorporation of the tool into the body schema. Thus, the current data suggest that simulating tool-use modulates the representation of the human body in extra-striate cortex.

No dissociation between perception and action in patient DF when haptic feedback is withdrawn

Goodale et al. (1991) reported a striking dissociation between vision for perception and action. They examined DF, a human patient who had damage to her ventral visual stream and suffered from visual form agnosia. She was unable to perceive an object's size but could match the opening of her hand to the object's size during grasping. It was concluded that grasping relied on a separate representation of visual size in the dorsal stream and required no visual input from the ventral stream. This observation inspired the influential perception-action model, which claimed separate visual streams for perception and action. However, in grasping (but not in corresponding perceptual tasks), participants receive haptic feedback after each trial. Using this feedback, DF might compensate for her impaired size-vision. I reexamined DF's grasping behavior using a mirror apparatus to dissociate the image of an object from its physical presence. DF's grasping was only normal when she received haptic feedback. Thus, in grasping, DF can rely on haptic feedback to compensate for her deficit in size-perception. This can explain why her grasping is significantly better than her perceptual performance. The findings emphasize the extent of early interstream interactions and highlight the multimodal nature of sensory processing in the dorsal stream.

The extrastriate body area (EBA): One structure, multiple functions?

Abstract Downing and Peelen argue that the extrastriate body area (EBA) creates an unelaborated visual repre- sentation of the human body, but is not implicated in any higher-order computational process. We believe that this reflects an outdated view of brain function, in which neural regions are informationally encapsulated modules with fixed computational properties. In contrast, there is mounting evidence that functional properties may not be fixed but may be dependent on the context in which the region is recruited. We will illustrate this by taking a closer look at a visual area of which the properties are potentially even simpler: the primary visual cortex.

Motor planning is facilitated by adopting an action's goal posture: an fMRI study

Motor planning is a hierarchical process that is typically organized around an action's goal (e.g., drinking from a cup). However, the motor plan depends not only on the goal but also on the current body state. Here, we investigated how one's own body posture interacts with planning of goal-directed actions. Participants engaged in a grasp selection (GS) task while we manipulated their arm posture. They had to indicate how they would grasp a bar when transporting it from a start to goal position and orientation. We compared situations in which one's body posture was in-congruent with the start posture and/or goal posture of the planned movement. Behavioral results show that GS took longer when one's own body state was incongruent with the goal posture of the planned movement. Correspondingly, neural activity in the intraparietal sulcus (IPS) and extrastriate body area (EBA) was modulated by congruency between the body state and the action plan. IPS was sensitive to overall congruency between body posture and action plan, while the EBA was sensitive specifically to goal posture congruency. Together, our results suggest that IPS maintains an internal state of one's own body posture, while EBA contains a representation of the goal posture of the action plan.

On the relation between action selection and movement control in 5- to 9-month-old infants

Although 5-month-old infants select action modes that are adaptive to the size of the object (i.e., one- or two-handed reaching), it has largely remained unclear whether infants of this age control the ensuing movement to the size of the object (i.e., scaling of the aperture between hands). We examined 5-, 7-, and 9-month-olds' reaching behaviors to gain more insight into the developmental changes occurring in the visual guidance of action mode selection and movement control, and the relationship between these processes. Infants were presented with a small set of objects (i.e., 2, 3, 7, and 8 cm) and a large set of objects (i.e., 6, 9, 12, and 15 cm). For the first set of objects, it was found that the infants more often performed two-handed reaches for the larger objects based on visual information alone (i.e., before making contact with the object), thus showing adaptive action mode selection relative to object size. Kinematical analyses of the two-handed reaches for the second set of objects revealed that inter-trial variance in aperture between the hands decreased with the approach toward the object, indicating that infants' reaching is constrained by the object. Subsequent analysis showed that between hand aperture scaled to object size, indicating that visual control of the movement is adjusted to object size in infants as young as 5 months. Individual analyses indicated that the two processes were not dependent and followed distinct developmental trajectories. That is, adaptive selection of an action mode was not a prerequisite for appropriate aperture scaling, and vice versa. These findings are consistent with the idea of two separate and independent visual systems (Milner and Goodale in Neuropsychologia 46:774-785, 2008) during early infancy.

Selection of wrist posture in conditions of motor ambiguity

In our everyday motor interactions with objects, we often encounter situations where the features of an object are determinate (i.e., not perceptually ambiguous), but the mapping between those features and appropriate movement patterns is indeterminate, resulting in a lack of any clear preference for one posture over another. We call this indeterminacy in stimulus-response mapping 'motor ambiguity'. Here, we use a grasping task to investigate the decision mechanisms that mediate the basic behavior of selecting one wrist posture over another in conditions of motor ambiguity. Using one of two possible wrist postures, participants grasped a dowel that was presented at various orientations. At most orientations, there was a clear preference for one wrist posture over the other. Within a small range of orientations, however, participants were variable in their posture selection due to the fact that the dowel was ambiguous with respect to the hand posture it afforded. We observed longer reaction times (RT) during 'ambiguous' trials than during the 'unambiguous' trials. In two subsequent experiments, we explored the effects of foreknowledge and trial history on the selection of wrist posture. We found that foreknowledge led to shorter RT unless the previous trial involved selecting a posture in the ambiguous region, in which case foreknowledge gave no RT advantage. These results are discussed within the context of existing models of sensorimotor decision making.

The cognitive neuroscience of prehension: recent developments

Prehension, the capacity to reach and grasp, is the key behavior that allows humans to change their environment. It continues to serve as a remarkable experimental test case for probing the cognitive architecture of goal-oriented action. This review focuses on recent experimental evidence that enhances or modifies how we might conceptualize the neural substrates of prehension. Emphasis is placed on studies that consider how precision grasps are selected and transformed into motor commands. Then, the mechanisms that extract action relevant information from vision and touch are considered. These include consideration of how parallel perceptual networks within parietal cortex, along with the ventral stream, are connected and share information to achieve common motor goals. On-line control of grasping action is discussed within a state estimation framework. The review ends with a consideration about how prehension fits within larger action repertoires that solve more complex goals and the possible cortical architectures needed to organize these actions.

Do we have independent visual streams for perception and action?

The perception-action model proposes that vision-for-perception and vision-for-action are based on anatomically distinct and functionally independent streams within the visual cortex. This idea can account for diverse experimental findings, and has been hugely influential over the past two decades. The model itself comprises a set of core contrasts between the functional properties of the two visual streams. We critically review the evidence for these contrasts, arguing that each of them has either been refuted or found limited empirical support. We suggest that the perception-action model captures some broad patterns of functional localization, but that the specializations of the two streams are relative, not absolute. The ubiquity and extent of inter-stream interactions suggest that we should reject the idea that the ventral and dorsal streams are functionally independent processing pathways.