TMS of the anterior intraparietal area selectively modulates orientation change detection during action preparation

Action-specific feature processing in the human cortex: An fMRI study

Sensorimotor integration involves feedforward and reentrant processing of sensory input. Grasp-related motor activity precedes and is thought to influence visual object processing. Yet, while the importance of reentrant feedback is well established in perception, the top-down modulations for action and the neural circuits involved in this process have received less attention. Do action-specific intentions influence the processing of visual information in the human cortex? Using a cue-separation fMRI paradigm, we found that action-specific instruction processing (manual alignment vs. grasp) became apparent only after the visual presentation of oriented stimuli, and occurred as early as in the primary visual cortex and extended to the dorsal visual stream, motor and premotor areas. Further, dorsal stream area aIPS, known to be involved in object manipulation, and the primary visual cortex showed task-related functional connectivity with frontal, parietal and temporal areas, consistent with the idea that reentrant feedback from dorsal and ventral visual stream areas modifies visual inputs to prepare for action. Importantly, both the task-dependent modulations and connections were linked specifically to the object presentation phase of the task, suggesting a role in processing the action goal. Our results show that intended manual actions have an early, pervasive, and differential influence on the cortical processing of vision.

Visual working memory representations bias attention more when they are the target of an action plan

Attention has frequently been regarded as an emergent property of linking sensory representations to action plans. It has recently been proposed that similar mechanisms may operate within visual working memory (VWM), such that linking an object in VWM to an action plan strengthens its sensory memory representation, which then expresses as an attentional bias. Here we directly tested this hypothesis by comparing attentional biases induced by VWM representations which were the target of a future action, to those induced by VWM representations that were equally task-relevant, but not the direct target of action. We predicted that the first condition would result in a more prioritized memory state and hence stronger attentional biases. Specifically, participants memorized a geometric shape for a subsequent memory test. At test, in case of a match, participants either had to perform a grip movement on the matching object (action condition), or perform the same movement, but on an unrelated object (control condition). To assess any attentional biases, during the delay period between memorandum and test, participants performed a visual selection task in which either the target was surrounded by the memorized shape (congruent trials) or a distractor (incongruent trials). Eye movements were measured as a proxy for attentional priority. We found a significant interaction for saccade latencies between action condition and shape congruency, reflecting more pronounced VWM-based attentional biases in the action condition. Our results are consistent with the idea that action plans prioritize sensory representations in VWM.

Action planning modulates the representation of object features in human fronto-parietal and occipital cortex

The visual cortex has been extensively studied to investigate its role in object recognition but to a lesser degree to determine how action planning influences the representation of objects' features. We used functional MRI and pattern classification methods to determine if during action planning, object features (orientation and location) could be decoded in an action‐dependent way. Sixteen human participants used their right dominant hand to perform movements (Align or Open reach) towards one of two 3D‐real oriented objects that were simultaneously presented and placed on either side of a fixation cross. While both movements required aiming towards target location, Align but not Open reach movements required participants to precisely adjust hand orientation. Therefore, we hypothesized that if the representation of object features is modulated by the upcoming action, pre‐movement activity pattern would allow more accurate dissociation between object features in Align than Open reach tasks. We found such dissociation in the anterior and posterior parietal cortex, as well as in the dorsal premotor cortex, suggesting that visuomotor processing is modulated by the upcoming task. The early visual cortex showed significant decoding accuracy for the dissociation between object features in the Align but not Open reach task. However, there was no significant difference between the decoding accuracy in the two tasks. These results demonstrate that movement‐specific preparatory signals modulate object representation in the frontal and parietal cortex, and to a lesser extent in the early visual cortex, likely through feedback functional connections.

Horizontal target size perturbations during grasping movements are described by subsequent size perception and saccade amplitude

Perception and action are essential in our day-to-day interactions with the environment. Despite the dual-stream theory of action and perception, it is now accepted that action and perception processes interact with each other. However, little is known about the impact of unpredicted changes of target size during grasping actions on perception. We assessed whether size perception and saccade amplitude were affected before and after grasping a target that changed its horizontal size during the action execution under the presence or absence of tactile feedback. We have tested twenty-one participants in 4 blocks of 30 trials. Blocks were divided into two experimental tactile feedback paradigms: tactile and non-tactile. Trials consisted of 3 sequential phases: pre-grasping size perception, grasping, and post-grasping size perception. During pre- and post-phases, participants executed a saccade towards a horizontal bar and performed a manual size estimation of the bar size. During grasping phase, participants were asked to execute a saccade towards the bar and to make a grasping action towards the screen. While grasping, 3 horizontal size perturbation conditions were applied: non-perturbation, shortening, and lengthening. 30% of the trials presented perturbation, meaning a symmetrically shortened or lengthened by 33% of the original size. Participants’ hand and eye positions were assessed by a motion capture system and a mobile eye-tracker, respectively. After grasping, in both tactile and non-tactile feedback paradigms, size estimation was significantly reduced in lengthening (p = 0.002) and non-perturbation (p<0.001), whereas shortening did not induce significant adjustments (p = 0.86). After grasping, saccade amplitude became significantly longer in shortening (p<0.001) and significantly shorter in lengthening (p<0.001). Non-perturbation condition did not display adjustments (p = 0.95). Tactile feedback did not generate changes in the collected perceptual responses, but horizontal size perturbations did so, suggesting that all relevant target information used in the movement can be extracted from the post-action target perception.

Cerebellar contribution to emotional body language perception: a TMS study

Consistent evidence suggests that the cerebellum contributes to the processing of emotional facial expressions. However, it is not yet known whether the cerebellum is recruited when emotions are expressed by body postures or movements, or whether it is recruited differently for positive and negative emotions. In this study, we asked healthy participants to discriminate between body postures (with masked face) expressing emotions of opposite valence (happiness vs anger, Experiment 1), or of the same valence (negative: anger vs sadness; positive: happiness vs surprise, Experiment 2). While performing the task, participants received online transcranial magnetic stimulation (TMS) over a region of the posterior left cerebellum and over two control sites (early visual cortex and vertex). We found that TMS over the cerebellum affected participants' ability to discriminate emotional body postures, but only when one of the emotions was negatively valenced (i.e. anger). These findings suggest that the cerebellar region we stimulated is involved in processing the emotional content conveyed by body postures and gestures. Our findings complement prior evidence on the role of the cerebellum in emotional face processing and have important implications from a clinical perspective, where non-invasive cerebellar stimulation is a promising tool for the treatment of motor, cognitive and affective deficits.

Selective Modulation of Early Visual Cortical Activity by Movement Intention

The primate visual system contains myriad feedback projections from higher- to lower-order cortical areas, an architecture that has been implicated in the top-down modulation of early visual areas during working memory and attention. Here we tested the hypothesis that these feedback projections also modulate early visual cortical activity during the planning of visually guided actions. We show, across three separate human functional magnetic resonance imaging (fMRI) studies involving object-directed movements, that information related to the motor effector to be used (i.e., limb, eye) and action goal to be performed (i.e., grasp, reach) can be selectively decoded-prior to movement-from the retinotopic representation of the target object(s) in early visual cortex. We also find that during the planning of sequential actions involving objects in two different spatial locations, that motor-related information can be decoded from both locations in retinotopic cortex. Together, these findings indicate that movement planning selectively modulates early visual cortical activity patterns in an effector-specific, target-centric, and task-dependent manner. These findings offer a neural account of how motor-relevant target features are enhanced during action planning and suggest a possible role for early visual cortex in instituting a sensorimotor estimate of the visual consequences of movement.

Sense of external agency is sustained by multisensory functional integration in the somatosensory cortex

"Sense of agency" (SoA), the feeling of control for events caused by one's own actions, is deceived by visuomotor incongruence. Sensorimotor networks are implicated in SoA, however little evidence exists on brain functionality during agency processing. Concurrently, it has been suggested that the brain's intrinsic resting-state (rs) activity has a preliminary influence on processing of agency cues. Here, we investigated the relation between performance in an agency attribution task and functional interactions among brain regions as derived by network analysis of rs functional magnetic resonance imaging. The action-effect delay was adaptively increased (range 90-1,620 ms) and behavioral measures correlated to indices of cognitive processes and appraised self-concepts. They were then regressed on local metrics of rs brain functional connectivity as to isolate the core areas enabling self-agency. Across subjects, the time window for self-agency was 90-625 ms, while the action-effect integration was impacted by self-evaluated personality traits. Neurally, the brain intrinsic organization sustaining consistency in self-agency attribution was characterized by high connectiveness in the secondary visual cortex, and regional segregation in the primary somatosensory area. Decreased connectiveness in the secondary visual area, regional segregation in the superior parietal lobule, and information control within a primary visual cortex-frontal eye fields network sustained self-agency over long-delayed effects. We thus demonstrate that self-agency is grounded on the intrinsic mode of brain function designed to organize information for visuomotor integration. Our observation is relevant for current models of psychopathology in clinical conditions in which both rs activity and sense of agency are altered.

Action Fluency Facilitates Perceptual Discrimination

Perception and action interact in nearly every moment of daily life. Previous studies have demonstrated not only that perceptual input shapes action but also that various factors associated with action-including individual abilities and biomechanical costs-influence perceptual decisions. However, it is unknown how action fluency affects the sensitivity of early-stage visual perception, such as orientation. To address this question, we used a dual-task paradigm: Participants prepared an action (e.g., grasping), while concurrently performing an orientation-change-detection task. We demonstrated that as actions became more fluent (e.g., as grasping errors decreased), perceptual-discrimination performance also improved. Importantly, we found that grasping training prior to discrimination enhanced subsequent perceptual sensitivity, supporting the notion of a reciprocal relation between perception and action.

Distinct Neural Mechanisms of Spatial Attention and Expectation Guide Perceptual Inference in a Multisensory World

Spatial attention (i.e., task-relevance) and expectation (i.e., signal probability) are two critical top-down mechanisms guiding perceptual inference. Spatial attention prioritizes processing of information at task-relevant locations. Spatial expectations encode the statistical structure of the environment. An unresolved question is how the brain allocates attention and forms expectations in a multisensory environment, where task-relevance and signal probability over space can differ across sensory modalities. We used functional magnetic resonance imaging in human participants (female and male) to investigate whether the brain encodes task-relevance and signal probability over space separately or interactively across sensory modalities. In a novel multisensory paradigm, we manipulated spatial attention and expectation selectively in audition and assessed their effects on behavioral and neural responses to auditory and visual stimuli. Our results show that both auditory and visual stimuli increased activations in a right-lateralized frontoparietal system, when they were presented at locations that were task-irrelevant in audition. Yet, only auditory stimuli increased activations in the medial prefrontal cortex when presented at expected locations and in audiovisual and frontoparietal cortices signaling a prediction error when presented at unexpected locations. This dissociation in multisensory generalization for attention and expectation effects shows that the brain controls attentional resources interactively across the senses but encodes the statistical structure of the environment as spatial expectations independently for each sensory system. Our results demonstrate that spatial attention and expectation engage partly overlapping neural systems via distinct mechanisms to guide perceptual inference in a multisensory world.SIGNIFICANCE STATEMENT In our natural environment the brain is exposed to a constant influx of signals through all our senses. How does the brain allocate attention and form spatial expectations in this multisensory environment? Because observers need to respond to stimuli regardless of their sensory modality, they may allocate attentional resources and encode the probability of events jointly across the senses. This psychophysics and neuroimaging study shows that the brain controls attentional resources interactively across the senses via a frontoparietal system but encodes the statistical structure of the environment independently for each sense in sensory and frontoparietal areas. Thus, spatial attention and expectation engage partly overlapping neural systems via distinct mechanisms to guide perceptual inference in a multisensory world.

Effects of Excitatory Repetitive Transcranial Magnetic Stimulation of the P3 Point in Chronic Stroke Patients—Case Reports

Objective: To evaluate the effects of excitatory repetitive transcranial magnetic stimulation (rTMS) of the international 10–20 system P3 point (intraparietal sulcus region) in chronic patients with a frontal lesion and parietal sparing due to stroke on the impaired upper (UL) and lower limb (LL) as measured by the Fugl-Meyer Assessment (FMA). Methods: Three patients (C1: 49.83/2.75, C2: 53.17/3.83, C3: 63.33/3.08-years-old at stroke/years post-stroke, respectively) received two weeks (five days/week) of rTMS at 10 Hz of P3. A patient was treated in similar conditions with a sham coil (S1: 56.58/4.33). Patients were evaluated before, after, and two months post-treatment (A1, A2, and A3, respectively). Results: For LL, the scores of the motor function subsection of C1 and C3 as well as the sensory function of C2 increased by A2 and remained by A3. For UL, the score of the motor function of C2 and C3 also increased, but the score of C3 decreased by A3. The score of the range of motion subsection of C3 increased by the two follow-up evaluations. Conclusion: This study suggests excitatory rTMS over P3 may be of use for some chronic stroke patients, but these findings need to be verified in a future clinical trial.

Distances on hills look farther than distances on flat ground: Evidence from converging measures

Distances on hills are judged as farther than when the same distance is presented on the flat ground. The hypothesized reason for this difference is because perception is influenced by the increased effort required to walk up a hill than to walk the same distance on flat ground. Alternatively, distances presented up a hill might be judged as farther for other, nonperceptual reasons such as bias from demand characteristics. To test whether distances on hills are perceived as farther or are merely judged as farther, we used a variety of measures, including visual matching and blindwalking tasks, and found similar effects across all measures. This convergence is consistent with a perceptual explanation. Second, we mined our data with the goal of making recommendations for future research on this paradigm. Although all of the perceptual measures used showed similar effects, visual matching was the only measure that had good intrasubject reliability. We recommend that future research on this action-specific effect could use any measure unless the research is geared towards individual differences, in which case, only the visual matching measure of perceived distance should be used.

Effector-based attention systems

Visual processing is known to be enhanced at the end point of eye movements. Feedback within the oculomotor system has been shown to drive these alterations in visual processing. However, we do not simply view the world; we also reach out and interact using our hands. Consequently, it is not surprising that visual processing has also been shown to be altered in near-hand space. A growing body of work documents a myriad of alterations in near-hand visual processing, with little consensus on the neural underpinnings of the effect of the hand. Since movement of the eyes and hands is governed by parallel frontoparietal networks and since within the oculomotor system feedback from these motor control regions has been shown to drive enhanced visual processing at saccade end points, it is plausible that a similar feedback mechanism is at play in near-hand improvements in visual processing. Here, we compare and contrast oculomotor-driven and hand-driven changes in visual processing and provide support for the hypothesis that feedback within the reaching and grasping systems enhances visual processing near the hand in a novel way.

Non-uniform transformation of subjective time during action preparation

Although many studies have reported a distortion of subjective (internal) time during preparation and execution of actions, it is highly controversial whether actions cause a dilation or compression of time. In the present study, we tested a hypothesis that the previous controversy (dilation vs. compression) partly resulted from a mixture of two types of sensory inputs on which a time length was estimated; some studies asked subjects to measure the time of presentation for a single continuous stimulus (stimulus period, e.g. the duration of a long-lasting visual stimulus on a monitor) while others required estimation of a period without continuous stimulations (no-stimulus period, e.g. an inter-stimulus interval between two flashes). Results of our five experiments supported this hypothesis, showing that action preparation induced a dilation of a stimulus period, whereas a no-stimulus period was not subject to this dilation and sometimes can be compressed by action preparation. Those results provided a new insight into a previous view assuming a uniform dilation or compression of subjective time by actions. Our findings about the distinction between stimulus and no-stimulus periods also might contribute to a resolution of mixed results (action-induced dilation vs. compression) in a previous literature.

Functional coupling between frontoparietal and occipitotemporal pathways during action and perception

Several lines of evidence point to areas in the occipitotemporal pathway as being critical in the processes of visual perception and object recognition. Much less appreciated, however, is the role that this pathway plays in object-related processing for the purposes of visually guided action. Here, using functional MRI (fMRI) and functional connectivity (FC) measures, we examined interactions between areas in frontoparietal cortex (FPC) involved in grasping, reaching, eye movements, and tool use and areas in occipitotemporal cortex (OTC) involved in object-, face-, scene-, body-, tool-, and motion-related processing, both during the performance of sensorimotor and visual-perceptual tasks, as well as during passive fixation (resting-state). Cluster analysis of regional time course data identified correspondence in the patterns of FPC and OTC connectivity during the visual-perceptual tasks and rest that both tended to segregate regions along traditional dorsal/ventral pathway boundaries. During the sensorimotor tasks, however, we observed a notable separation in functional coupling between ventral-medial and ventral-lateral regions of OTC, with several of the latter areas often being clustered together with sensorimotor-defined areas in parietal cortex. These findings indicate that the functional coupling of ventral-lateral OTC areas to dorsal parietal and ventral-medial structures is flexible and task-dependent, and suggests that regions in lateral occipital cortex, in particular, may play an important role in mediating interactions between the dorsal and ventral pathways during tasks involving sensorimotor control.

Through the forest of motor representations

Following neuroscience, and using different labels, several philosophers have addressed the idea of the presence of a single representational mechanism lying in between (visual) perceptual processes and motor processes involved in different functions and useful for shaping suitable action performances: a motor representation (MR). MRs are the naturalized mental antecedents of action. This paper presents a new, non-monolithic view of MRs, according to which, contrarily to the received view, when looking at in between (visual) perceptual processes and motor processes, we find not only a single representational mechanism with different functions, but an ensemble of different sub-representational phenomena, each of which with a different function. This new view is able to avoid several issues emerging from the literature and to address something the literature is silent about, which however turns out to be crucial for a theory of MRs.

An Eye in the Palm of Your Hand: Alterations in Visual Processing Near the Hand, a Mini-Review

Feedback within the oculomotor system improves visual processing at eye movement end points, also termed a visual grasp. We do not just view the world around us however, we also reach out and grab things with our hands. A growing body of literature suggests that visual processing in near-hand space is altered. The control systems for moving either the eyes or the hands rely on parallel networks of fronto-parietal regions, which have feedback connections to visual areas. Since the oculomotor system effects on visual processing occur through feedback, both through the motor plan and the motor efference copy, a parallel system where reaching and/or grasping motor-related activity also affects visual processing is likely. Areas in the posterior parietal cortex, for example, receive proprioceptive and visual information used to guide actions, as well as motor efference signals. This trio of information channels is all that would be necessary to produce spatial allocation of reach-related visual attention. We review evidence from behavioral and neurophysiological studies that support the hypothesis that feedback from the reaching and/or grasping motor control networks affects visual processing while noting ways in which it differs from that seen within the oculomotor system. We also suggest that object affordances may represent the neural mechanism through which certain object features are selected for preferential processing when stimuli are near the hand. Finally, we summarize the two effector-based feedback systems and discuss how having separate but parallel effector systems allows for efficient decoupling of eye and hand movements.

Action preparation shapes processing in early visual cortex

Preparation for an action, such as grasping an object, is accompanied by an enhanced perception of the object's action-relevant features, such as orientation and size. Cortical feedback from motor planning areas to early visual areas may drive this enhanced perception. To examine whether action preparation modulates activity in early human visual cortex, subjects grasped or pointed to oriented objects while high-resolution fMRI data were acquired. Using multivoxel pattern analysis techniques, we could decode with >70% accuracy whether a grasping or pointing action was prepared from signals in visual cortex as early as V1. These signals in early visual cortex were observed even when actions were only prepared but not executed. Anterior parietal cortex, on the other hand, showed clearest modulation for actual movements. This demonstrates that preparation of actions, even without execution, modulates relevant neuronal populations in early visual areas.

Hand placement near the visual stimulus improves orientation selectivity in V2 neurons

Often, the brain receives more sensory input than it can process simultaneously. Spatial attention helps overcome this limitation by preferentially processing input from a behaviorally-relevant location. Recent neuropsychological and psychophysical studies suggest that attention is deployed to near-hand space much like how the oculomotor system can deploy attention to an upcoming gaze position. Here we provide the first neuronal evidence that the presence of a nearby hand enhances orientation selectivity in early visual processing area V2. When the hand was placed outside the receptive field, responses to the preferred orientation were significantly enhanced without a corresponding significant increase at the orthogonal orientation. Consequently, there was also a significant sharpening of orientation tuning. In addition, the presence of the hand reduced neuronal response variability. These results indicate that attention is automatically deployed to the space around a hand, improving orientation selectivity. Importantly, this appears to be optimal for motor control of the hand, as opposed to oculomotor mechanisms which enhance responses without sharpening orientation selectivity. Effector-based mechanisms for visual enhancement thus support not only the spatiotemporal dissociation of gaze and reach, but also the optimization of vision for their separate requirements for guiding movements.