Three-dimensional transformations for goal-directed action

Action of the Euclidean versus projective group on an agent's internal space in curiosity driven exploration

According to the Projective Consciousness Model (PCM), in human spatial awareness, 3-dimensional projective geometry structures information integration and action planning through perspective taking within an internal representation space. The way different perspectives are related to and transform a world model defines a specific perception and imagination scheme. In mathematics, such a collection of transformations corresponds to a 'group', whose 'actions' characterize the geometry of a space. Imbuing world models with a group structure may capture different agents' spatial awareness and affordance schemes. We used group action as a special class of policies for perspective-dependent control. We explored how such a geometric structure impacts agents' behaviors, comparing how the Euclidean versus projective groups act on epistemic value in active inference, drive curiosity, and exploration. We formally demonstrate and simulate how the groups induce distinct behaviors in a simple search task. The projective group's nonlinear magnification of information transformed epistemic value according to the choice of frame, generating behaviors of approach toward objects with uncertain locations due to limited sampling. The Euclidean group had no effect on epistemic value: no action was better than the initial idle state. In structuring a priori an agent's internal representation, we show how geometry can play a key role in information integration and action planning. Our results add further support to the PCM.

Neural integration of egocentric and allocentric visual cues in the gaze system

A fundamental question in neuroscience is how the brain integrates egocentric (body-centered) and allocentric (landmark-centered) visual cues, but for many years this question was ignored in sensorimotor studies. This changed in recent behavioral experiments, but the underlying physiology of ego/allocentric integration remained largely unstudied. The specific goal of this review is to explain how prefrontal neurons integrate eye-centered and landmark-centered visual codes for optimal gaze behavior. First, we briefly review the whole brain/behavioral mechanisms for ego/allocentric integration in the human and summarize egocentric coding mechanisms in the primate gaze system. We then focus in more depth on cellular mechanisms for ego/allocentric coding in the frontal and supplementary eye fields. We first explain how prefrontal visual responses integrate eye-centered target and landmark codes to produce a transformation toward landmark-centered coordinates. Next, we describe what happens when a landmark shifts during the delay between seeing and acquiring a remembered target, initially resulting in independently coexisting ego/allocentric memory codes. We then describe how these codes are reintegrated in the motor burst for the gaze shift. Deep network simulations suggest that these properties emerge spontaneously for optimal gaze behavior. Finally, we synthesize these observations and relate them to normal brain function through a simplified conceptual model. Together, these results show that integration of visuospatial features continues well beyond visual cortex and suggest a general cellular mechanism for goal-directed visual behavior.

Neural Encoding of Direction and Distance across Reference Frames in Visually Guided Reaching

Goal-directed actions require transforming sensory information into motor plans defined across multiple parameters and reference frames. Substantial evidence supports the encoding of target direction in gaze- and body-centered coordinates within parietal and premotor regions. However, how the brain encodes the equally critical parameter of target distance remains less understood. Here, using Bayesian pattern component modeling of fMRI data during a delayed reach-to-target task, we dissociated the neural encoding of both target direction and the relative distances between target, gaze, and hand at early and late stages of motor planning. This approach revealed independent representations of direction and distance along the human dorsomedial reach pathway. During early planning, most premotor and superior parietal areas encoded a target's distance in single or multiple reference frames and encoded its direction. In contrast, distance encoding was magnified in gaze- and body-centric reference frames during late planning. These results emphasize a flexible and efficient human central nervous system that achieves goals by remapping sensory information related to multiple parameters, such as distance and direction, in the same brain areas.

Shape configuration of mental targets representation as a holistic measure in a 3D real world pointing test for spatial orientation

Deficits in spatial memory are often early signs of neurological disorders. Here, we analyzed the geometrical shape configuration of 2D-projections of pointing performances to a memorized array of spatially distributed targets in order to assess the feasibility of this new holistic analysis method. The influence of gender differences and cognitive impairment was taken into account in this methodological study. 56 right-handed healthy participants (28 female, mean age 48.89 ± 19.35 years) and 22 right-handed patients with heterogeneous cognitive impairment (12 female, mean age 71.73 ± 7.41 years) underwent a previously validated 3D-real-world pointing test (3D-RWPT). Participants were shown a 9-dot target matrix and afterwards asked to point towards each target in randomized order with closed eyes in different body positions relative to the matrix. Two-dimensional projections of these pointing vectors (i.e., the shapes resulting from the individual dots) were then quantified using morphological analyses. Shape configurations in healthy volunteers largely reflected the real-world target pattern with gender-dependent differences (ANCOVA area males vs. females F(1,73) = 9.00, p 3.69 × 10-3, partial η2 = 0.10, post-hoc difference = 38,350.43, pbonf=3.69 × 10-3**, Cohen's d 0.76, t 3.00). Patients with cognitive impairment showed distorted rectangularity with more large-scale errors, resulting in decreased overall average diameters and solidity (ANCOVA diameter normal cognition/cognitive impairment F(1,71) = 9.30, p 3.22 × 10-3, partial η2 = 0.09, post-hoc difference = 31.22, pbonf=3.19 × 10-3**, Cohen's d 0.92, t 3.05; solidity normal cognition/cognitive impairment F(1,71) = 7.79, p 6.75 × 10-3, partial η2 = 0.08, post-hoc difference = 0.07, pbonf=6.76 × 10-3** Cohen's d 0.84, t 2.79). Shape configuration analysis of the 3D-RWPT target array appears to be a suitable holistic measure of spatial performance in a pointing task. The results of this methodological investigation support further testing in a clinical study for differential diagnosis of disorders with spatial memory deficits.

Perceptual-Cognitive Integration for Goal-Directed Action in Naturalistic Environments

Real-world actions require one to simultaneously perceive, think, and act on the surrounding world, requiring the integration of (bottom-up) sensory information and (top-down) cognitive and motor signals. Studying these processes involves the intellectual challenge of cutting across traditional neuroscience silos, and the technical challenge of recording data in uncontrolled natural environments. However, recent advances in techniques, such as neuroimaging, virtual reality, and motion tracking, allow one to address these issues in naturalistic environments for both healthy participants and clinical populations. In this review, we survey six topics in which naturalistic approaches have advanced both our fundamental understanding of brain function and how neurologic deficits influence goal-directed, coordinated action in naturalistic environments. The first part conveys fundamental neuroscience mechanisms related to visuospatial coding for action, adaptive eye-hand coordination, and visuomotor integration for manual interception. The second part discusses applications of such knowledge to neurologic deficits, specifically, steering in the presence of cortical blindness, impact of stroke on visual-proprioceptive integration, and impact of visual search and working memory deficits. This translational approach-extending knowledge from lab to rehab-provides new insights into the complex interplay between perceptual, motor, and cognitive control in naturalistic tasks that are relevant for both basic and clinical research.

Egocentric cues influence the allocentric spatial memory of object configurations for memory-guided actions

Allocentric and egocentric reference frames are used to code the spatial position of action targets in reference to objects in the environment, i.e., relative to landmarks (allocentric), or the observer (egocentric). Previous research investigated reference frames in isolation, for example, by shifting landmarks relative to the target and asking participants to reach to the remembered target location. Systematic reaching errors were found in the direction of the landmark shift and used as a proxy for allocentric spatial coding. Here, we examined the interaction of both allocentric and egocentric reference frames by shifting the landmarks as well as the observer. We asked participants to encode a three-dimensional configuration of balls and to reproduce this configuration from memory after a short delay followed by a landmark or an observer shift. We also manipulated the number of landmarks to test its effect on the use of allocentric and egocentric reference frames. We found that participants were less accurate when reproducing the configuration of balls after an observer shift, which was reflected in larger configurational errors. In addition, an increase in the number of landmarks led to a stronger reliance on allocentric cues and a weaker contribution of egocentric cues. In sum, our results highlight the important role of egocentric cues for allocentric spatial coding in the context of memory-guided actions.NEW & NOTEWORTHY Objects in our environment are coded relative to each other (allocentrically) and are thought to serve as independent and reliable cues (landmarks) in the context of unreliable egocentric signals. Contrary to this assumption, we demonstrate that egocentric cues alter the allocentric spatial memory, which could reflect recently discovered interactions between allocentric and egocentric neural processing pathways. Furthermore, additional landmarks lead to a higher contribution of allocentric and a lower contribution of egocentric cues.

The behavioral and neural effects of parietal theta burst stimulation on the grasp network are stronger during a grasping task than at rest

Repetitive transcranial magnetic stimulation (TMS) is widely used in neuroscience and clinical settings to modulate human cortical activity. The effects of TMS on neural activity depend on the excitability of specific neural populations at the time of stimulation. Accordingly, the brain state at the time of stimulation may influence the persistent effects of repetitive TMS on distal brain activity and associated behaviors. We applied intermittent theta burst stimulation (iTBS) to a region in the posterior parietal cortex (PPC) associated with grasp control to evaluate the interaction between stimulation and brain state. Across two experiments, we demonstrate the immediate responses of motor cortex activity and motor performance to state-dependent parietal stimulation. We randomly assigned 72 healthy adult participants to one of three TMS intervention groups, followed by electrophysiological measures with TMS and behavioral measures. Participants in the first group received iTBS to PPC while performing a grasping task concurrently. Participants in the second group received iTBS to PPC while in a task-free, resting state. A third group of participants received iTBS to a parietal region outside the cortical grasping network while performing a grasping task concurrently. We compared changes in motor cortical excitability and motor performance in the three stimulation groups within an hour of each intervention. We found that parietal stimulation during a behavioral manipulation that activates the cortical grasping network increased downstream motor cortical excitability and improved motor performance relative to stimulation during rest. We conclude that constraining the brain state with a behavioral task during brain stimulation has the potential to optimize plasticity induction in cortical circuit mechanisms that mediate movement processes.

Integration of landmark and saccade target signals in macaque frontal cortex visual responses

Visual landmarks influence spatial cognition and behavior, but their influence on visual codes for action is poorly understood. Here, we test landmark influence on the visual response to saccade targets recorded from 312 frontal and 256 supplementary eye field neurons in rhesus macaques. Visual response fields are characterized by recording neural responses to various target-landmark combinations, and then we test against several candidate spatial models. Overall, frontal/supplementary eye fields response fields preferentially code either saccade targets (40%/40%) or landmarks (30%/4.5%) in gaze fixation-centered coordinates, but most cells show multiplexed target-landmark coding within intermediate reference frames (between fixation-centered and landmark-centered). Further, these coding schemes interact: neurons with near-equal target and landmark coding show the biggest shift from fixation-centered toward landmark-centered target coding. These data show that landmark information is preserved and influences target coding in prefrontal visual responses, likely to stabilize movement goals in the presence of noisy egocentric signals.

Depth and direction effects in the prediction of static and shifted reaching goals from kinematics

The kinematic parameters of reach-to-grasp movements are modulated by action intentions. However, when an unexpected change in visual target goal during reaching execution occurs, it is still unknown whether the action intention changes with target goal modification and which is the temporal structure of the target goal prediction. We recorded the kinematics of the pointing finger and wrist during the execution of reaching movements in 23 naïve volunteers where the targets could be located at different directions and depths with respect to the body. During the movement execution, the targets could remain static for the entire duration of movement or shifted, with different timings, to another position. We performed temporal decoding of the final goals and of the intermediate trajectory from the past kinematics exploiting a recurrent neural network. We observed a progressive increase of the classification performance from the onset to the end of movement in both horizontal and sagittal dimensions, as well as in decoding shifted targets. The classification accuracy in decoding horizontal targets was higher than the classification accuracy of sagittal targets. These results are useful for establishing how human and artificial agents could take advantage from the observed kinematics to optimize their cooperation in three-dimensional space.

The functions of consciousness in visual processing

Conscious experiences form a relatively diverse class of psychological phenomena, supported by a range of distinct neurobiological mechanisms. This diversity suggests that consciousness occupies a variety of different functional roles across different task domains, individuals, and species; a position I call functional pluralism. In this paper, I begin to tease out some of the functional contributions that consciousness makes to (human) visual processing. Consolidating research from across the cognitive sciences, I discuss semantic and spatiotemporal processing as specific points of comparison between the functional capabilities of the visual system in the presence and absence of conscious awareness. I argue that consciousness contributes a cluster of functions to visual processing; facilitating, among other things, (i) increased capacities for semantically processing informationally complex visual stimuli, (ii) increased spatiotemporal precision, and (iii) increased capacities for representational integration over large spatiotemporal intervals. This sort of analysis should ultimately yield a plurality of functional markers that can be used to guide future research in the philosophy and science of consciousness, some of which are not captured by popular theoretical frameworks like global workspace theory and information integration theory.

Parietofrontal oscillations show hand-specific interactions with top-down movement plans

To generate a hand-specific reach plan, the brain must integrate hand-specific signals with the desired movement strategy. Although various neurophysiology/imaging studies have investigated hand-target interactions in simple reach-to-target tasks, the whole brain timing and distribution of this process remain unclear, especially for more complex, instruction-dependent motor strategies. Previously, we showed that a pro/anti pointing instruction influences magnetoencephalographic (MEG) signals in frontal cortex that then propagate recurrently through parietal cortex (Blohm G, Alikhanian H, Gaetz W, Goltz HC, DeSouza JF, Cheyne DO, Crawford JD. NeuroImage 197: 306-319, 2019). Here, we contrasted left versus right hand pointing in the same task to investigate 1) which cortical regions of interest show hand specificity and 2) which of those areas interact with the instructed motor plan. Eight bilateral areas, the parietooccipital junction (POJ), superior parietooccipital cortex (SPOC), supramarginal gyrus (SMG), medial/anterior interparietal sulcus (mIPS/aIPS), primary somatosensory/motor cortex (S1/M1), and dorsal premotor cortex (PMd), showed hand-specific changes in beta band power, with four of these (M1, S1, SMG, aIPS) showing robust activation before movement onset. M1, SMG, SPOC, and aIPS showed significant interactions between contralateral hand specificity and the instructed motor plan but not with bottom-up target signals. Separate hand/motor signals emerged relatively early and lasted through execution, whereas hand-motor interactions only occurred close to movement onset. Taken together with our previous results, these findings show that instruction-dependent motor plans emerge in frontal cortex and interact recurrently with hand-specific parietofrontal signals before movement onset to produce hand-specific motor behaviors.NEW & NOTEWORTHY The brain must generate different motor signals depending on which hand is used. The distribution and timing of hand use/instructed motor plan integration are not understood at the whole brain level. Using MEG we show that different action planning subnetworks code for hand usage and integrating hand use into a hand-specific motor plan. The timing indicates that frontal cortex first creates a general motor plan and then integrates hand specificity to produce a hand-specific motor plan.

Different strategies in pointing tasks and their impact on clinical bedside tests of spatial orientation

Deficits in spatial memory, orientation, and navigation are often early or neglected signs of degenerative and vestibular neurological disorders. A simple and reliable bedside test of these functions would be extremely relevant for diagnostic routine. Pointing at targets in the 3D environment is a basic well-trained common sensorimotor ability that provides a suitable measure. We here describe a smartphone-based pointing device using the built-in inertial sensors for analysis of pointing performance in azimuth and polar spatial coordinates. Interpretation of the vectors measured in this way is not trivial, since the individuals tested may use at least two different strategies: first, they may perform the task in an egocentric eye-based reference system by aligning the fingertip with the target retinotopically or second, by aligning the stretched arm and the index finger with the visual line of sight in allocentric world-based coordinates similar to using a rifle. The two strategies result in considerable differences of target coordinates. A pilot test with a further developed design of the device and an app for a standardized bedside utilization in five healthy volunteers revealed an overall mean deviation of less than 5° between the measured and the true coordinates. Future investigations of neurological patients comparing their performance before and after changes in body position (chair rotation) may allow differentiation of distinct orientational deficits in peripheral (vestibulopathy) or central (hippocampal or cortical) disorders.

Head Orientation Influences Saccade Directions during Free Viewing

When looking around a visual scene, humans make saccadic eye movements to fixate objects of interest. While the extraocular muscles can execute saccades in any direction, not all saccade directions are equally likely: saccades in horizontal and vertical directions are most prevalent. Here, we asked whether head orientation plays a role in determining saccade direction biases. Study participants (n = 14) viewed natural scenes and abstract fractals (radially symmetric patterns) through a virtual reality headset equipped with eye tracking. Participants' heads were stabilized and tilted at -30°, 0°, or 30° while viewing the images, which could also be tilted by -30°, 0°, and 30° relative to the head. To determine whether the biases in saccade direction changed with head tilt, we calculated polar histograms of saccade directions and cross-correlated pairs of histograms to find the angular displacement resulting in the maximum correlation. During free viewing of fractals, saccade biases largely followed the orientation of the head with an average displacement value of 24° when comparing head upright to head tilt in world-referenced coordinates (t (13) = 17.63, p < 0.001). There was a systematic offset of 2.6° in saccade directions, likely reflecting ocular counter roll (OCR; t (13) = 3.13, p = 0.008). When participants viewed an Earth upright natural scene during head tilt, we found that the orientation of the head still influenced saccade directions (t (13) = 3.7, p = 0.001). These results suggest that nonvisual information about head orientation, such as that acquired by vestibular sensors, likely plays a role in saccade generation.

Integration of allocentric and egocentric visual information in a convolutional/multilayer perceptron network model of goal-directed gaze shifts

Allocentric (landmark-centered) and egocentric (eye-centered) visual codes are fundamental for spatial cognition, navigation, and goal-directed movement. Neuroimaging and neurophysiology suggest these codes are initially segregated, but then reintegrated in frontal cortex for movement control. We created and validated a theoretical framework for this process using physiologically constrained inputs and outputs. To implement a general framework, we integrated a convolutional neural network (CNN) of the visual system with a multilayer perceptron (MLP) model of the sensorimotor transformation. The network was trained on a task where a landmark shifted relative to the saccade target. These visual parameters were input to the CNN, the CNN output and initial gaze position to the MLP, and a decoder transformed MLP output into saccade vectors. Decoded saccade output replicated idealized training sets with various allocentric weightings and actual monkey data where the landmark shift had a partial influence (R² = 0.8). Furthermore, MLP output units accurately simulated prefrontal response field shifts recorded from monkeys during the same paradigm. In summary, our model replicated both the general properties of the visuomotor transformations for gaze and specific experimental results obtained during allocentric–egocentric integration, suggesting it can provide a general framework for understanding these and other complex visuomotor behaviors.

Decoding sensorimotor information from superior parietal lobule of macaque via Convolutional Neural Networks

Despite the well-recognized role of the posterior parietal cortex (PPC) in processing sensory information to guide action, the differential encoding properties of this dynamic processing, as operated by different PPC brain areas, are scarcely known. Within the monkey's PPC, the superior parietal lobule hosts areas V6A, PEc, and PE included in the dorso-medial visual stream that is specialized in planning and guiding reaching movements. Here, a Convolutional Neural Network (CNN) approach is used to investigate how the information is processed in these areas. We trained two macaque monkeys to perform a delayed reaching task towards 9 positions (distributed on 3 different depth and direction levels) in the 3D peripersonal space. The activity of single cells was recorded from V6A, PEc, PE and fed to convolutional neural networks that were designed and trained to exploit the temporal structure of neuronal activation patterns, to decode the target positions reached by the monkey. Bayesian Optimization was used to define the main CNN hyper-parameters. In addition to discrete positions in space, we used the same network architecture to decode plausible reaching trajectories. We found that data from the most caudal V6A and PEc areas outperformed PE area in the spatial position decoding. In all areas, decoding accuracies started to increase at the time the target to reach was instructed to the monkey, and reached a plateau at movement onset. The results support a dynamic encoding of the different phases and properties of the reaching movement differentially distributed over a network of interconnected areas. This study highlights the usefulness of neurons' firing rate decoding via CNNs to improve our understanding of how sensorimotor information is encoded in PPC to perform reaching movements. The obtained results may have implications in the perspective of novel neuroprosthetic devices based on the decoding of these rich signals for faithfully carrying out patient's intentions.

Egocentric Chunking in the Predictive Brain: A Cognitive Basis of Expert Performance in High-Speed Sports

What principles and mechanisms allow humans to encode complex 3D information, and how can it be so fast, so accurately and so flexibly transformed into coordinated action? How do these processes work when developed to the limit of human physiological and cognitive capacity-as they are in high-speed sports, such as alpine skiing or motor racing? High-speed sports present not only physical challenges, but present some of the biggest perceptual-cognitive demands for the brain. The skill of these elite athletes is in many ways an attractive model for studying human performance "in the wild", and its neurocognitive basis. This article presents a framework theory for how these abilities may be realized in high-speed sports. It draws on a careful analysis of the case of the motorsport athlete, as well as theoretical concepts from: (1) cognitive neuroscience of wayfinding, steering, and driving; (2) cognitive psychology of expertise; (3) cognitive modeling and machine learning; (4) human-in-the loop modellling in vehicle system dynamics and human performance engineering; (5) experimental research (in the laboratory and in the field) on human visual guidance. The distinctive contribution is the way these are integrated, and the concept of chunking is used in a novel way to analyze a high-speed sport. The mechanisms invoked are domain-general, and not specific to motorsport or the use of a particular type of vehicle (or any vehicle for that matter); the egocentric chunking hypothesis should therefore apply to any dynamic task that requires similar core skills. It offers a framework for neuroscientists, psychologists, engineers, and computer scientists working in the field of expert sports performance, and may be useful in translating fundamental research into theory-based insight and recommendations for improving real-world elite performance. Specific experimental predictions and applicability of the hypotheses to other sports are discussed.

Distinct networks coupled with parietal cortex for spatial representations inside and outside the visual field

Our mental representation of egocentric space is influenced by the disproportionate sensory perception of the body. Previous studies have focused on the neural architecture for egocentric representations within the visual field. However, the space representation underlying the body is still unclear. To address this problem, we applied both functional Magnitude Resonance Imaging (fMRI) and Magnetoencephalography (MEG) to a spatial-memory paradigm by using a virtual environment in which human participants remembered a target location left, right, or back relative to their own body. Both experiments showed larger involvement of the frontoparietal network in representing a retrieved target on the left/right side than on the back. Conversely, the medial temporal lobe (MTL)-parietal network was more involved in retrieving a target behind the participants. The MEG data showed an earlier activation of the MTL-parietal network than that of the frontoparietal network during retrieval of a target location. These findings suggest that the parietal cortex may represent the entire space around the self-body by coordinating two distinct brain networks.

Retinal optic flow during natural locomotion

We examine the structure of the visual motion projected on the retina during natural locomotion in real world environments. Bipedal gait generates a complex, rhythmic pattern of head translation and rotation in space, so without gaze stabilization mechanisms such as the vestibular-ocular-reflex (VOR) a walker's visually specified heading would vary dramatically throughout the gait cycle. The act of fixation on stable points in the environment nulls image motion at the fovea, resulting in stable patterns of outflow on the retinae centered on the point of fixation. These outflowing patterns retain a higher order structure that is informative about the stabilized trajectory of the eye through space. We measure this structure by applying the curl and divergence operations on the retinal flow velocity vector fields and found features that may be valuable for the control of locomotion. In particular, the sign and magnitude of foveal curl in retinal flow specifies the body's trajectory relative to the gaze point, while the point of maximum divergence in the retinal flow field specifies the walker's instantaneous overground velocity/momentum vector in retinotopic coordinates. Assuming that walkers can determine the body position relative to gaze direction, these time-varying retinotopic cues for the body's momentum could provide a visual control signal for locomotion over complex terrain. In contrast, the temporal variation of the eye-movement-free, head-centered flow fields is large enough to be problematic for use in steering towards a goal. Consideration of optic flow in the context of real-world locomotion therefore suggests a re-evaluation of the role of optic flow in the control of action during natural behavior.

Hierarchical Integration of Communicative and Spatial Perspective‐Taking Demands in Sensorimotor Control of Referential Pointing

Recognized as a simple communicative behavior, referential pointing is cognitively complex because it invites a communicator to consider an addressee's knowledge. Although we know referential pointing is affected by addressees’ physical location, it remains unclear whether and how communicators’ inferences about addressees’ mental representation of the interaction space influence sensorimotor control of referential pointing. The communicative perspective‐taking task requires a communicator to point at one out of multiple referents either to instruct an addressee which one should be selected (communicative, COM) or to predict which one the addressee will select (non‐communicative, NCOM), based on either which referents can be seen (Level‐1 perspective‐taking, PT1) or how the referents were perceived (Level‐2 perspective‐taking, PT2) by the addressee. Communicators took longer to initiate the movements in PT2 than PT1 trials, and they held their pointing fingers for longer at the referent in COM than NCOM trials. The novel findings of this study pertain to trajectory control of the pointing movements. Increasing both communicative and perspective‐taking demands led to longer pointing trajectories, with an under‐additive interaction between those two experimental factors. This finding suggests that participants generate communicative behaviors that are as informative as required rather than overly exaggerated displays, by integrating communicative and perspective‐taking information hierarchically during sensorimotor control. This observation has consequences for models of human communication. It implies that the format of communicative and perspective‐taking knowledge needs to be commensurate with the movement dynamics controlled by the sensorimotor system.

Head and eyes: Looking behavior in 12- to 24-month-old infants

This study demonstrates evidence for a foundational process underlying active vision in older infants during object play. Using head-mounted eye-tracking and motion capture, looks to an object are shown to be tightly linked to and synchronous with a stilled head, regardless of the duration of gaze, for infants 12 to 24 months of age. Despite being a developmental period of rapid and marked changes in motor abilities, the dynamic coordination of head stabilization and sustained gaze to a visual target is developmentally invariant during the examined age range. The findings indicate that looking with an aligned head and eyes is a fundamental property of human vision and highlights the importance of studying looking behavior in freely moving perceivers in everyday contexts, opening new questions about the role of body movement in both typical and atypical development of visual attention.

Gaze control during reaching is flexibly modulated to optimize task outcome

When reaching for an object with the hand, the gaze is usually directed at the target. In a laboratory setting, fixation is strongly maintained at the reach target until the reaching is completed, a phenomenon known as "gaze anchoring." While conventional accounts of such tight eye-hand coordination have often emphasized the internal synergetic linkage between both motor systems, more recent optimal control theories regard motor coordination as the adaptive solution to task requirements. We here investigated to what degree gaze control during reaching is modulated by task demands. We adopted a gaze-anchoring paradigm in which participants had to reach for a target location. During the reach, they additionally had to make a saccadic eye movement to a salient visual cue presented at locations other than the target. We manipulated the task demands by independently changing reward contingencies for saccade reaction time (RT) and reaching accuracy. On average, both saccade RTs and reach error varied systematically with reward condition, with reach accuracy improving when the saccade was delayed. The distribution of the saccade RTs showed two types of eye movements: fast saccades with short RTs, and voluntary saccade with longer RTs. Increased reward for high reach accuracy reduced the probability of fast saccades but left their latency unchanged. The results suggest that gaze anchoring acts through a suppression of fast saccades, a mechanism that can be adaptively adjusted to the current task demands.NEW & NOTEWORTHY During visually guided reaching, our eyes usually fixate the target and saccades elsewhere are delayed ("gaze anchoring"). We here show that the degree of gaze anchoring is flexibly modulated by the reward contingencies of saccade latency and reach accuracy. Reach error became larger when saccades occurred earlier. These results suggest that early saccades are costly for reaching and the brain modulates inhibitory online coordination from the hand to the eye system depending on task requirements.

Interhemispheric interactions between the right angular gyrus and the left motor cortex: a transcranial magnetic stimulation study

The interconnection of the angular gyrus of right posterior parietal cortex (PPC) and the left motor cortex (LM1) is essential for goal-directed hand movements. Previous work with transcranial magnetic stimulation (TMS) showed that right PPC stimulation increases LM1 excitability, but right PPC followed by left PPC-LM1 stimulation (LPPC-LM1) inhibits LM1 corticospinal output compared with LPPC-LM1 alone. It is not clear if right PPC-mediated inhibition of LPPC-LM1 is due to inhibition of left PPC or to combined effects of right and left PPC stimulation on LM1 excitability. We used paired-pulse TMS to study the extent to which combined right and left PPC stimulation, targeting the angular gyri, influences LM1 excitability. We tested 16 healthy subjects in five paired-pulsed TMS experiments using MRI-guided neuronavigation to target the angular gyri within PPC. We tested the effects of different right angular gyrus (RAG) and LM1 stimulation intensities on the influence of RAG on LM1 and on influence of left angular gyrus (LAG) on LM1 (LAG-LM1). We then tested the effects of RAG and LAG stimulation on LM1 short-interval intracortical facilitation (SICF), short-interval intracortical inhibition (SICI), and long-interval intracortical inhibition (LICI). The results revealed that RAG facilitated LM1, inhibited SICF, and inhibited LAG-LM1. Combined RAG-LAG stimulation did not affect SICI but increased LICI. These experiments suggest that RAG-mediated inhibition of LAG-LM1 is related to inhibition of early indirect (I)-wave activity and enhancement of GABA_B receptor-mediated inhibition in LM1. The influence of RAG on LM1 likely involves ipsilateral connections from LAG to LM1 and heterotopic connections from RAG to LM1.NEW & NOTEWORTHY Goal-directed hand movements rely on the right and left angular gyri (RAG and LAG) and motor cortex (M1), yet how these brain areas functionally interact is unclear. Here, we show that RAG stimulation facilitated right hand motor output from the left M1 but inhibited indirect (I)-waves in M1. Combined RAG and LAG stimulation increased GABA_B, but not GABA_A, receptor-mediated inhibition in left M1. These findings highlight unique brain interactions between the RAG and left M1.

Stereoscopic depth constancy from a different direction

To calibrate stereoscopic depth from disparity our visual system must compensate for an object's egocentric location. Ideally, the perceived three-dimensional shape and size of objects in visual space should be invariant with their location such that rigid objects have a consistent identity and shape. These percepts should be accurate enough to support both perceptual judgments and visually-guided interaction. This theoretical note reviews the relationship of stereoscopic depth constancy to the geometry of stereoscopic space and seemingly esoteric concepts like the horopter. We argue that to encompass the full scope of stereoscopic depth constancy, researchers need to consider not just distance but also direction, that is 3D egocentric location in space. Judgements of surface orientation need to take into account the shape of the horopter and the computation of metric depth (when tasks depend on it) must compensate for direction as well as distance to calibrate disparities. We show that the concept of the horopter underlies these considerations and that the relationship between depth constancy and the horopter should be more explicit in the literature.

Evidence for an effector-independent action system from people born without hands

What are the principles of brain organization? In the motor domain, separate pathways were found for reaching and grasping actions performed by the hand. To what extent is this organization specific to the hand or based on abstract action types, regardless of which body part performs them? We tested people born without hands who perform actions with their feet. Activity in frontoparietal association motor areas showed preference for an action type (reaching or grasping), regardless of whether it was performed by the foot in people born without hands or by the hand in typically-developed controls. These findings provide evidence that some association areas are organized based on abstract functions of action types, independent of specific sensorimotor experience and parameters of specific body parts.

Many parts of the visuomotor system guide daily hand actions, like reaching for and grasping objects. Do these regions depend exclusively on the hand as a specific body part whose movement they guide, or are they organized for the reaching task per se, for any body part used as an effector? To address this question, we conducted a neuroimaging study with people born without upper limbs—individuals with dysplasia—who use the feet to act, as they and typically developed controls performed reaching and grasping actions with their dominant effector. Individuals with dysplasia have no prior experience acting with hands, allowing us to control for hand motor imagery when acting with another effector (i.e., foot). Primary sensorimotor cortices showed selectivity for the hand in controls and foot in individuals with dysplasia. Importantly, we found a preference based on action type (reaching/grasping) regardless of the effector used in the association sensorimotor cortex, in the left intraparietal sulcus and dorsal premotor cortex, as well as in the basal ganglia and anterior cerebellum. These areas also showed differential response patterns between action types for both groups. Intermediate areas along a posterior–anterior gradient in the left dorsal premotor cortex gradually transitioned from selectivity based on the body part to selectivity based on the action type. These findings indicate that some visuomotor association areas are organized based on abstract action functions independent of specific sensorimotor parameters, paralleling sensory feature-independence in visual and auditory cortices in people born blind and deaf. Together, they suggest association cortices across action and perception may support specific computations, abstracted from low-level sensorimotor elements.

Transcranial Magnetic Stimulation and the Understanding of Behavior

The development of the use of transcranial magnetic stimulation (TMS) in the study of psychological functions has entered a new phase of sophistication. This is largely due to an increasing physiological knowledge of its effects and to its being used in combination with other experimental techniques. This review presents the current state of our understanding of the mechanisms of TMS in the context of designing and interpreting psychological experiments. We discuss the major conceptual advances in behavioral studies using TMS. There are meaningful physiological and technical achievements to review, as well as a wealth of new perceptual and cognitive experiments. In doing so we summarize the different uses and challenges of TMS in mental chronometry, perception, awareness, learning, and memory.

Spatiotemporal transformations for gaze control

Sensorimotor transformations require spatiotemporal coordination of signals, that is, through both time and space. For example, the gaze control system employs signals that are time-locked to various sensorimotor events, but the spatial content of these signals is difficult to assess during ordinary gaze shifts. In this review, we describe the various models and methods that have been devised to test this question, and their limitations. We then describe a new method that can (a) simultaneously test between all of these models during natural, head-unrestrained conditions, and (b) track the evolving spatial continuum from target (T) to future gaze coding (G, including errors) through time. We then summarize some applications of this technique, comparing spatiotemporal coding in the primate frontal eye field (FEF) and superior colliculus (SC). The results confirm that these areas preferentially encode eye-centered, effector-independent parameters, and show-for the first time in ordinary gaze shifts-a spatial transformation between visual and motor responses from T to G coding. We introduce a new set of spatial models (T-G continuum) that revealed task-dependent timing of this transformation: progressive during a memory delay between vision and action, and almost immediate without such a delay. We synthesize the results from our studies and supplement it with previous knowledge of anatomy and physiology to propose a conceptual model where cumulative transformation noise is realized as inaccuracies in gaze behavior. We conclude that the spatiotemporal transformation for gaze is both local (observed within and across neurons in a given area) and distributed (with common signals shared across remote but interconnected structures).

The role of perception and action on the use of allocentric information in a large-scale virtual environment

In everyday life, our brain constantly builds spatial representations of the objects surrounding us. Many studies have investigated the nature of these spatial representations. It is well established that we use allocentric information in real-time and memory-guided movements. Most studies relied on small-scale and static experiments, leaving it unclear whether similar paradigms yield the same results on a larger scale using dynamic objects. We created a virtual reality task that required participants to encode the landing position of a virtual ball thrown by an avatar. Encoding differed in the nature of the task in that it was either purely perceptual (“view where the ball landed while standing still”—Experiment 1) or involved an action (“intercept the ball with the foot just before it lands”—Experiment 2). After encoding, participants were asked to place a real ball at the remembered landing position in the virtual scene. In some trials, we subtly shifted either the thrower or the midfield line on a soccer field to manipulate allocentric coding of the ball’s landing position. In both experiments, we were able to replicate classic findings from small-scale experiments and to generalize these results to different encoding tasks (perception vs. action) and response modes (reaching vs. walking-and-placing). Moreover, we found that participants preferably encoded the ball relative to the thrower when they had to intercept the ball, suggesting that the use of allocentric information is determined by the encoding task by enhancing task-relevant allocentric information. Our findings indicate that results previously obtained from memory-guided reaching are not restricted to small-scale movements, but generalize to whole-body movements in large-scale dynamic scenes.

Spatial coding for memory-guided reaching in visual and pictorial spaces

An essential difference between pictorial space displayed as paintings, photographs, or computer screens, and the visual space experienced in the real world is that the observer has a defined location, and thus valid information about distance and direction of objects, in the latter but not in the former. Thus egocentric information should be more reliable in visual space, whereas allocentric information should be more reliable in pictorial space. The majority of studies relied on pictorial representations (images on a computer screen), leaving it unclear whether the same coding mechanisms apply in visual space. Using a memory-guided reaching task in virtual reality, we investigated allocentric coding in both visual space (on a table in virtual reality) and pictorial space (on a monitor that is on the table in virtual reality). Our results suggest that the brain uses allocentric information to represent objects in both pictorial and visual space. Contrary to our hypothesis, the influence of allocentric cues was stronger in visual space than in pictorial space, also after controlling for retinal stimulus size, confounding allocentric cues, and differences in presentation depth. We discuss possible reasons for stronger allocentric coding in visual than in pictorial space.

Task Errors Drive Memories That Improve Sensorimotor Adaptation

Traditional views of sensorimotor adaptation (i.e., adaptation of movements to perturbed sensory feedback) emphasize the role of automatic, implicit correction of sensory prediction errors. However, latent memories formed during sensorimotor adaptation, manifest as improved relearning (e.g., savings), have recently been attributed to strategic corrections of task errors (failures to achieve task goals). To dissociate contributions of task errors and sensory prediction errors to latent sensorimotor memories, we perturbed target locations to remove or enforce task errors during learning and/or test, with male/female human participants. Adaptation improved after learning in all conditions where participants were permitted to correct task errors, and did not improve whenever we prevented correction of task errors. Thus, previous correction of task errors was both necessary and sufficient to improve adaptation. In contrast, a history of sensory prediction errors was neither sufficient nor obligatory for improved adaptation. Limiting movement preparation time showed that the latent memories driven by learning to correct task errors take at least two forms: a time-consuming but flexible component, and a rapidly expressible, inflexible component. The results provide strong support for the idea that movement corrections driven by a failure to successfully achieve movement goals underpin motor memories that manifest as savings. Such persistent memories are not exclusively mediated by time-consuming strategic processes but also comprise a rapidly expressible but inflexible component. The distinct characteristics of these putative processes suggest dissociable underlying mechanisms, and imply that identification of the neural basis for adaptation and savings will require methods that allow such dissociations.SIGNIFICANCE STATEMENT Latent motor memories formed during sensorimotor adaptation manifest as improved adaptation when sensorimotor perturbations are reencountered. Conflicting theories suggest that this "savings" is underpinned by different mechanisms, including a memory of successful actions, a memory of errors, or an aiming strategy to correct task errors. Here we show that learning to correct task errors is sufficient to show improved subsequent adaptation with respect to naive performance, even when tested in the absence of task errors. In contrast, a history of sensory prediction errors is neither sufficient nor obligatory for improved adaptation. Finally, we show that latent sensorimotor memories driven by task errors comprise at least two distinct components: a time-consuming, flexible component, and a rapidly expressible, inflexible component.

Spatial updating of allocentric landmark information in real-time and memory-guided reaching

The 2-streams model of vision suggests that egocentric and allocentric reference frames are utilized by the dorsal and the ventral stream for real-time and memory-guided movements, respectively. Recent studies argue against such a strict functional distinction and suggest that real-time and memory-guided movements recruit the same spatial maps. In this study we focus on allocentric spatial coding and updating of targets by using landmark information in real-time and memory-guided reaching. We presented participants with a naturalistic scene which consisted of six objects on a table that served as potential reach targets. Participants were informed about the target object after scene encoding, and were prompted by a go cue to reach to its position. After target identification a brief air-puff was applied to the participant's right eye inducing an eye blink. During the blink the target object disappeared from the scene, and in half of the trials the remaining objects, that functioned as landmarks, were shifted horizontally in the same direction. We found that landmark shifts systematically influenced participants' reaching endpoints irrespective of whether the movements were controlled online based on available target information (real-time movement) or memory-guided based on remembered target information (memory-guided movement). Overall, the effect of landmark shift was stronger for memory-guided than real-time reaching. Our findings suggest that humans can encode and update reach targets in an allocentric reference frame for both real-time and memory-guided movements and show stronger allocentric coding when the movement is based on memory.

Timing Determines Tuning: A Rapid Spatial Transformation in Superior Colliculus Neurons during Reactive Gaze Shifts

Gaze saccades, rapid shifts of the eyes and head toward a goal, have provided fundamental insights into the neural control of movement. For example, it has been shown that the superior colliculus (SC) transforms a visual target (T) code to future gaze (G) location commands after a memory delay. However, this transformation has not been observed in "reactive" saccades made directly to a stimulus, so its contribution to normal gaze behavior is unclear. Here, we tested this using a quantitative measure of the intermediate codes between T and G, based on variable errors in gaze endpoints. We demonstrate that a rapid spatial transformation occurs within the primate's SC (Macaca mulatta) during reactive saccades, involving a shift in coding from T, through intermediate codes, to G. This spatial shift progressed continuously both across and within cell populations [visual, visuomotor (VM), motor], rather than relaying discretely between populations with fixed spatial codes. These results suggest that the SC produces a rapid, noisy, and distributed transformation that contributes to variable errors in reactive gaze shifts.

A review of grasping as the movements of digits in space

It is tempting to describe human reach-to-grasp movements in terms of two, more or less independent visuomotor channels, one relating hand transport to the object’s location and the other relating grip aperture to the object’s size. Our review of experimental work questions this framework for reasons that go beyond noting the dependence between the two channels. Both the lack of effect of size illusions on grip aperture and the finding that the variability in grip aperture does not depend on the object’s size indicate that size information is not used to control grip aperture. An alternative is to describe grip formation as emerging from controlling the movements of the digits in space. Each digit’s trajectory when grasping an object is remarkably similar to its trajectory when moving to tap the same position on its own. The similarity is also evident in the fast responses when the object is displaced. This review develops a new description of the speed-accuracy trade-off for multiple effectors that is applied to grasping. The most direct support for the digit-in-space framework is that prism-induced adaptation of each digit’s tapping movements transfers to that digit’s movements when grasping, leading to changes in grip aperture for adaptation in opposite directions for the two digits. We conclude that although grip aperture and hand transport are convenient variables to describe grasping, treating grasping as movements of the digits in space is a more suitable basis for understanding the neural control of grasping.

Trunk rotation and handedness modulate cortical activation in neglect-associated regions during temporal order judgments

The rotation of the trunk around its vertical midline could be shown to bias visuospatial temporal judgments towards targets in the hemifield ipsilateral to the trunk orientation and to improve visuospatial performance in patients with visual neglect. However, the underlying brain mechanisms are not well understood. Therefore, the goal of the present study was to investigate the neural effects associated with egocentric midplane shifts under consideration of individual handedness. We employed a visuospatial temporal order judgment (TOJ) task in healthy right- and left-handed subjects while their trunk rotation was varied. Participants responded by a saccade towards the stimulus perceived first out of two stimuli presented with different stimulus onset asynchronies (SOA). Apart from gaze behavior, BOLD-fMRI responses were measured using functional magnetic resonance imaging (fMRI). Based on findings from spatial neglect research, analyses of fMRI-BOLD responses were focused on a bilateral fronto-temporo-parietal network comprising Brodmann areas 22, 39, 40, and 44, as well as the basal ganglia core nuclei (caudate, putamen, pallidum).

We observed an acceleration of saccadic speed towards stimuli ipsilateral to the trunk orientation modulated by individual handedness. Left-handed participants showed the strongest behavioral and neural effects, suggesting greater susceptibility to manipulations of trunk orientation. With respect to the dominant hand, a rotation around the vertical trunk midline modulated the activation of an ipsilateral network comprising fronto-temporo-parietal regions and the putamen with the strongest effects for saccades towards the hemifield opposite to the dominant hand. Within the investigated network, the temporo-parietal junction (TPJ) appears to serve as a region integrating sensory, motor, and trunk position information.

Our results are discussed in the context of gain modulatory and laterality effects.

We examined the effect of trunk rotation on brain responses in neglect-associated areas.•

Trunk-related BOLD-fMRI activation patterns depend on handedness.

•

They were modulated most during trunk rotation contralateral to the dominant hand.

•

Trunk rotation and saccade direction show interaction effects at TPJ.

•

TPJ serves as a region integrating sensory, motor, and trunk position information.

Humans Use Predictive Gaze Strategies to Target Waypoints for Steering

A major unresolved question in understanding visually guided locomotion in humans is whether actions are driven solely by the immediately available optical information (model-free online control mechanisms), or whether internal models have a role in anticipating the future path. We designed two experiments to investigate this issue, measuring spontaneous gaze behaviour while steering, and predictive gaze behaviour when future path information was withheld. In Experiment 1 participants (N = 15) steered along a winding path with rich optic flow: gaze patterns were consistent with tracking waypoints on the future path 1–3 s ahead. In Experiment 2, participants (N = 12) followed a path presented only in the form of visual waypoints located on an otherwise featureless ground plane. New waypoints appeared periodically every 0.75 s and predictably 2 s ahead, except in 25% of the cases the waypoint at the expected location was not displayed. In these cases, there were always other visible waypoints for the participant to fixate, yet participants continued to make saccades to the empty, but predictable, waypoint locations (in line with internal models of the future path guiding gaze fixations). This would not be expected based upon existing model-free online steering control models, and strongly points to a need for models of steering control to include mechanisms for predictive gaze control that support anticipatory path following behaviours.

Mixed Spatial and Movement Representations in the Primate Posterior Parietal Cortex

The posterior parietal cortex (PPC) of humans and non-human primates plays a key role in the sensory and motor transformations required to guide motor actions to objects of interest in the environment. Despite decades of research, the anatomical and functional organization of this region is still a matter of contention. It is generally accepted that specialized parietal subregions and their functional counterparts in the frontal cortex participate in distinct segregated networks related to eye, arm and hand movements. However, experimental evidence obtained primarily from single neuron recording studies in non-human primates has demonstrated a rich mixing of signals processed by parietal neurons, calling into question ideas for a strict functional specialization. Here, we present a brief account of this line of research together with the basic trends in the anatomical connectivity patterns of the parietal subregions. We review, the evidence related to the functional communication between subregions of the PPC and describe progress towards using parietal neuron activity in neuroprosthetic applications. Recent literature suggests a role for the PPC not as a constellation of specialized functional subdomains, but as a dynamic network of sensorimotor loci that combine multiple signals and work in concert to guide motor behavior.

Cognition in Sensorimotor Control: Interfacing With the Posterior Parietal Cortex

Millions of people worldwide are afflicted with paralysis from a disruption of neural pathways between the brain and the muscles. Because their cortical architecture is often preserved, these patients are able to plan movements despite an inability to execute them. In such people, brain machine interfaces have great potential to restore lost function through neuroprosthetic devices, circumventing dysfunctional corticospinal circuitry. These devices have typically derived control signals from the motor cortex (M1) which provides information highly correlated with desired movement trajectories. However, sensorimotor control simultaneously engages multiple cognitive processes such as intent, state estimation, decision making, and the integration of multisensory feedback. As such, cortical association regions upstream of M1 such as the posterior parietal cortex (PPC) that are involved in higher order behaviors such as planning and learning, rather than in encoding movement itself, may enable enhanced, cognitive control of neuroprosthetics, termed cognitive neural prosthetics (CNPs). We illustrate in this review, through a small sampling, the cognitive functions encoded in the PPC and discuss their neural representation in the context of their relevance to motor neuroprosthetics. We aim to highlight through examples a role for cortical signals from the PPC in developing CNPs, and to inspire future avenues for exploration in their research and development.

Compatibility between object size and response side in grasping: the left hand prefers smaller objects, the right hand prefers larger objects

It has been proposed that the brain processes quantities such as space, size, number, and other magnitudes using a common neural metric, and that this common representation system reflects a direct link to motor control, because the integration of spatial, temporal, and other quantity-related information is fundamental for sensorimotor transformation processes. In the present study, we examined compatibility effects between physical stimulus size and spatial (response) location during a sensorimotor task. Participants reached and grasped for a small or large object with either their non-dominant left or their dominant right hand. Our results revealed that participants initiated left hand movements faster when grasping the small cube compared to the large cube, whereas they initiated right hand movements faster when grasping the large cube compared to the small cube. Moreover, the compatibility effect influenced the timing of grip aperture kinematics. These findings indicate that the interaction between object size and response hand affects the planning of grasping movements and supports the notion of a strong link between the cognitive representation of (object) size, spatial (response) parameters, and sensorimotor control.

More than skin-deep: Integration of skin-based and musculoskeletal reference frames in localization of touch

The skin of the forearm is, in one sense, a flat 2-dimensional (2D) sheet, but in another sense approximately cylindrical, mirroring the 3-dimensional (3D) volumetric shape of the arm. The role of frames of reference based on the skin as a 2D sheet versus based on the musculoskeletal structure of the arm remains unclear. When we rotate the forearm from a pronated to a supinated posture, the skin on its surface is displaced. Thus, a marked location will slide with the skin across the underlying flesh, and the touch perceived at this location should follow this displacement if it is localized within a skin-based reference frame. We investigated, however, if the perceived tactile locations were also affected by the rearrangement in underlying musculoskeletal structure, that is, displaced medially and laterally on a pronated and supinated forearm, respectively. Participants pointed to perceived touches (Experiment 1), or marked them on a (3D) size-matched forearm on a computer screen (Experiment 2). The perceived locations were indeed displaced medially after forearm pronation in both response modalities. This misperception was reduced (Experiment 1), or absent altogether (Experiment 2) in the supinated posture when the actual stimulus grid moved laterally with the displaced skin. The grid was perceptually stretched at medial-lateral axis, and it was displaced distally, which suggest the influence of skin-based factors. Our study extends the tactile localization literature focused on the skin-based reference frame and on the effects of spatial positions of body parts by implicating the musculoskeletal factors in localization of touch on the body. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Plasticity based on compensatory effector use in the association but not primary sensorimotor cortex of people born without hands

What forces direct brain organization and its plasticity? When brain regions are deprived of their input, which regions reorganize based on compensation for the disability and experience, and which regions show topographically constrained plasticity? People born without hands activate their primary sensorimotor hand region while moving body parts used to compensate for this disability (e.g., their feet). This was taken to suggest a neural organization based on functions, such as performing manual-like dexterous actions, rather than on body parts, in primary sensorimotor cortex. We tested the selectivity for the compensatory body parts in the primary and association sensorimotor cortex of people born without hands (dysplasic individuals). Despite clear compensatory foot use, the primary sensorimotor hand area in the dysplasic subjects showed preference for adjacent body parts that are not compensatorily used as effectors. This suggests that function-based organization, proposed for congenital blindness and deafness, does not apply to the primary sensorimotor cortex deprivation in dysplasia. These findings stress the roles of neuroanatomical constraints like topographical proximity and connectivity in determining the functional development of primary cortex even in extreme, congenital deprivation. In contrast, increased and selective foot movement preference was found in dysplasics' association cortex in the inferior parietal lobule. This suggests that the typical motor selectivity of this region for manual actions may correspond to high-level action representations that are effector-invariant. These findings reveal limitations to compensatory plasticity and experience in modifying brain organization of early topographical cortex compared with association cortices driven by function-based organization.

Reference frames in the decisions of hand choice

For the brain to decide on a reaching movement, it needs to select which hand to use. A number of body-centered factors affect this decision, such as the anticipated movement costs of each arm, recent choice success, handedness, and task demands. While the position of each hand relative to the target is also known to be an important spatial factor, it is unclear which reference frames coordinate the spatial aspects in the decisions of hand choice. Here we tested the role of gaze- and head-centered reference frames in a hand selection task. With their head and gaze oriented in different directions, we measured hand choice of 19 right-handed subjects instructed to make unimanual reaching movements to targets at various directions relative to their body. Using an adaptive procedure, we determined the target angle that led to equiprobable right/left hand choices. When gaze remained fixed relative to the body this balanced target angle shifted systematically with head orientation, and when head orientation remained fixed this choice measure shifted with gaze. These results suggest that a mixture of head- and gaze-centered reference frames is involved in the spatially guided decisions of hand choice, perhaps to flexibly bind this process to the mechanisms of target selection.

NEW & NOTEWORTHY Decisions of target and hand choice are fundamental aspects of human reaching movements. While the reference frames involved in target choice have been identified, it is unclear which reference frames are involved in hand selection. We tested the role of gaze- and head-centered reference frames in a hand selection task. Findings emphasize the role of both spatial reference frames in the decisions of hand choice, in addition to known body-centered computations such anticipated movement costs and handedness.

Neural substrates for allocentric-to-egocentric conversion of remembered reach targets in humans

Targets for goal-directed action can be encoded in allocentric coordinates (relative to another visual landmark), but it is not known how these are converted into egocentric commands for action. Here, we investigated this using a slow event-related fMRI paradigm, based on our previous behavioural finding that the allocentric-to-egocentric (Allo-Ego) conversion for reach is performed at the first possible opportunity. Participants were asked to remember (and eventually reach towards) the location of a briefly presented target relative to another visual landmark. After a first memory delay, participants were forewarned by a verbal instruction if the landmark would reappear at the same location (potentially allowing them to plan a reach following the auditory cue before the second delay), or at a different location where they had to wait for the final landmark to be presented before response, and then reach towards the remembered target location. As predicted, participants showed landmark-centred directional selectivity in occipital-temporal cortex during the first memory delay, and only developed egocentric directional selectivity in occipital-parietal cortex during the second delay for the 'Same cue' task, and during response for the 'Different cue' task. We then compared cortical activation between these two tasks at the times when the Allo-Ego conversion occurred, and found common activation in right precuneus, right presupplementary area and bilateral dorsal premotor cortex. These results confirm that the brain converts allocentric codes to egocentric plans at the first possible opportunity, and identify the four most likely candidate sites specific to the Allo-Ego transformation for reaches.

Role of Rostral Fastigial Neurons in Encoding a Body-Centered Representation of Translation in Three Dimensions

Many daily behaviors rely critically on estimates of our body motion. Such estimates must be computed by combining neck proprioceptive signals with vestibular signals that have been transformed from a head- to a body-centered reference frame. Recent studies showed that deep cerebellar neurons in the rostral fastigial nucleus (rFN) reflect these computations, but whether they explicitly encode estimates of body motion remains unclear. A key limitation in addressing this question is that, to date, cell tuning properties have only been characterized for a restricted set of motions across head-re-body orientations in the horizontal plane. Here we examined, for the first time, how 3D spatiotemporal tuning for translational motion varies with head-re-body orientation in both horizontal and vertical planes in the rFN of male macaques. While vestibular coding was profoundly influenced by head-re-body position in both planes, neurons typically reflected at most a partial transformation. However, their tuning shifts were not random but followed the specific spatial trajectories predicted for a 3D transformation. We show that these properties facilitate the linear decoding of fully body-centered motion representations in 3D with a broad range of temporal characteristics from small groups of 5-7 cells. These results demonstrate that the vestibular reference frame transformation required to compute body motion is indeed encoded by cerebellar neurons. We propose that maintaining partially transformed rFN responses with different spatiotemporal properties facilitates the creation of downstream body motion representations with a range of dynamic characteristics, consistent with the functional requirements for tasks such as postural control and reaching.SIGNIFICANCE STATEMENT Estimates of body motion are essential for many daily activities. Vestibular signals are important contributors to such estimates but must be transformed from a head- to a body-centered reference frame. Here, we provide the first direct demonstration that the cerebellum computes this transformation fully in 3D. We show that the output of these computations is reflected in the tuning properties of deep cerebellar rostral fastigial nucleus neurons in a specific distributed fashion that facilitates the efficient creation of body-centered translation estimates with a broad range of temporal properties (i.e., from acceleration to position). These findings support an important role for the rostral fastigial nucleus as a source of body translation estimates functionally relevant for behaviors ranging from postural control to perception.