Ventral intraparietal area of the macaque: congruent visual and somatic response properties

What's Coming Near? The Influence of Dynamical Visual Stimuli on Nociceptive Processing

Objects approaching us may pose a threat, and signal the need to initiate defensive behavior. Detecting these objects early is crucial to either avoid the object or prepare for contact most efficiently. This requires the construction of a coherent representation of our body, and the space closely surrounding our body, i.e. the peripersonal space. This study, with 27 healthy volunteers, investigated how the processing of nociceptive stimuli applied to the hand is influenced by dynamical visual stimuli either approaching or receding from the hand. On each trial a visual stimulus was either approaching or receding the participant's left or right hand. At different temporal delays from the onset of the visual stimulus, a nociceptive stimulus was applied either at the same or the opposite hand, so that it was presented when the visual stimulus was perceived at varying distances from the hand. Participants were asked to respond as fast as possible at which side they perceived a nociceptive stimulus. We found that reaction times were fastest when the visual stimulus appeared near the stimulated hand. Moreover, investigating the influence of the visual stimuli along the continuous spatial range (from near to far) showed that approaching lights had a stronger spatially dependent effect on nociceptive processing, compared to receding lights. These results suggest that the coding of nociceptive information in a peripersonal frame of reference may constitute a safety margin around the body that is designed to protect it from potential physical threat.

The functional role of time compression

Multisensory integration provides continuous and stable perception from separate sensory inputs. Here, we investigated the functional role of temporal binding between the visual and the tactile senses. To this end we used the paradigm of compression that induces shifts in time when probe stimuli are degraded, e.g., by a visual mask (Zimmermann et al. 2014). Subjects had to estimate the duration of temporal intervals of 500 ms defined by a tactile and a visual, masked stimulus. We observed a strong (~100 ms) underestimation of the temporal interval when the stimuli from both senses appeared to occur at the same position in space. In contrast, when the positions of the visual and tactile stimuli were spatially separate, interval perception was almost veridical. Temporal compression furthermore depended on the correspondence of probe features and was absent when the orientation of the tactile and visual probes was incongruent. An additional experiment revealed that temporal compression also occurs when objects were presented outside the attentional focus. In conclusion, these data support a role for spatiotemporal binding in temporal compression, which is at least in part selective for object features.

Distributed Representation of Curvilinear Self-Motion in the Macaque Parietal Cortex

Information about translations and rotations of the body is critical for complex self-motion perception during spatial navigation. However, little is known about the nature and function of their convergence in the cortex. We measured neural activity in multiple areas in the macaque parietal cortex in response to three different types of body motion applied through a motion platform: translation, rotation, and combined stimuli, i.e., curvilinear motion. We found a continuous representation of motion types in each area. In contrast to single-modality cells preferring either translation-only or rotation-only stimuli, convergent cells tend to be optimally tuned to curvilinear motion. A weighted summation model captured the data well, suggesting that translation and rotation signals are integrated subadditively in the cortex. Interestingly, variation in the activity of convergent cells parallels behavioral outputs reported in human psychophysical experiments. We conclude that representation of curvilinear self-motion perception is widely distributed in the primate sensory cortex.

Joint representation of translational and rotational components of optic flow in parietal cortex

Terrestrial navigation naturally involves translations within the horizontal plane and eye rotations about a vertical (yaw) axis to track and fixate targets of interest. Neurons in the macaque ventral intraparietal (VIP) area are known to represent heading (the direction of self-translation) from optic flow in a manner that is tolerant to rotational visual cues generated during pursuit eye movements. Previous studies have also reported that eye rotations modulate the response gain of heading tuning curves in VIP neurons. We tested the hypothesis that VIP neurons simultaneously represent both heading and horizontal (yaw) eye rotation velocity by measuring heading tuning curves for a range of rotational velocities of either real or simulated eye movements. Three findings support the hypothesis of a joint representation. First, we show that rotation velocity selectivity based on gain modulations of visual heading tuning is similar to that measured during pure rotations. Second, gain modulations of heading tuning are similar for self-generated eye rotations and visually simulated rotations, indicating that the representation of rotation velocity in VIP is multimodal, driven by both visual and extraretinal signals. Third, we show that roughly one-half of VIP neurons jointly represent heading and rotation velocity in a multiplicatively separable manner. These results provide the first evidence, to our knowledge, for a joint representation of translation direction and rotation velocity in parietal cortex and show that rotation velocity can be represented based on visual cues, even in the absence of efference copy signals.

Audio-visual multisensory training enhances visual processing of motion stimuli in healthy participants: an electrophysiological study

Evidence from electrophysiological and imaging studies suggests that audio-visual (AV) stimuli presented in spatial coincidence enhance activity in the subcortical colliculo-dorsal extrastriate pathway. To test whether repetitive AV stimulation might specifically activate this neural circuit underlying multisensory integrative processes, electroencephalographic data were recorded before and after 2 h of AV training, during the execution of two lateralized visual tasks: a motion discrimination task, relying on activity in the colliculo-dorsal MT pathway, and an orientation discrimination task, relying on activity in the striate and early ventral extrastriate cortices. During training, participants were asked to detect and perform a saccade towards AV stimuli that were disproportionally allocated to one hemifield (the trained hemifield). Half of the participants underwent a training in which AV stimuli were presented in spatial coincidence, while the remaining half underwent a training in which AV stimuli were presented in spatial disparity (32°). Participants who received AV training with stimuli in spatial coincidence had a post-training enhancement of the anterior N1 component in the motion discrimination task, but only in response to stimuli presented in the trained hemifield. However, no effect was found in the orientation discrimination task. In contrast, participants who received AV training with stimuli in spatial disparity showed no effects on either task. The observed N1 enhancement might reflect enhanced discrimination for motion stimuli, probably due to increased activity in the colliculo-dorsal MT pathway induced by multisensory training.

Multisensory Integration in Self Motion Perception

Self motion perception involves the integration of visual, vestibular, somatosensory and motor signals. This article reviews the findings from single unit electrophysiology, functional and structural magnetic resonance imaging and psychophysics to present an update on how the human and non-human primate brain integrates multisensory information to estimate one’s position and motion in space. The results indicate that there is a network of regions in the non-human primate and human brain that processes self motion cues from the different sense modalities.

Mental body representations retain homuncular shape distortions: Evidence from Weber's illusion

Mental body representations underlying tactile perception do not accurately reflect the body's true morphology. For example, perceived tactile distance is dependent on both the body part being touched and the stimulus orientation, a phenomenon called Weber's illusion. These findings suggest the presence of size and shape distortions, respectively. However, whereas each morphological feature is typically measured in isolation, a complete morphological characterization requires the concurrent measurement of both size and shape. We did so in three experiments, manipulating both the stimulated body parts (hand; forearm) and stimulus orientation while requiring participants to make tactile distance judgments. We found that the forearm was significantly more distorted than the hand lengthwise but not widthwise. Effects of stimulus orientation are thought to reflect receptive field anisotropies in primary somatosensory cortex. The results of the present study therefore suggest that mental body representations retain homuncular shape distortions that characterize early stages of somatosensory processing.

Body part-centered and full body-centered peripersonal space representations

Dedicated neural systems represent the space surrounding the body, termed Peripersonal space (PPS), by integrating visual or auditory stimuli occurring near the body with somatosensory information. As a behavioral proxy to PPS, we measured participants' reaction time to tactile stimulation while task-irrelevant auditory or visual stimuli were presented at different distances from their body. In 7 experiments we delineated the critical distance at which auditory or visual stimuli boosted tactile processing on the hand, face, and trunk as a proxy of the PPS extension. Three main findings were obtained. First, the size of PPS varied according to the stimulated body part, being progressively bigger for the hand, then face, and largest for the trunk. Second, while approaching stimuli always modulated tactile processing in a space-dependent manner, receding stimuli did so only for the hand. Finally, the extension of PPS around the hand and the face varied according to their relative positioning and stimuli congruency, whereas the trunk PPS was constant. These results suggest that at least three body-part specific PPS representations exist, differing in extension and directional tuning. These distinct PPS representations, however, are not fully independent from each other, but referenced to the common reference frame of the trunk.

True and Perceived Synchrony are Preferentially Associated With Particular Sensory Pairings

Perception and behavior are fundamentally shaped by the integration of different sensory modalities into unique multisensory representations, a process governed by spatio-temporal correspondence. Prior work has characterized temporal perception using the point in time at which subjects are most likely to judge multisensory stimuli to be simultaneous (PSS) and the temporal binding window (TBW) over which participants are likely to do so. Here we examine the relationship between the PSS and the TBW within and between individuals, and within and between three sensory combinations: audiovisual, audiotactile and visuotactile. We demonstrate that TBWs correlate within individuals and across multisensory pairings, but PSSs do not. Further, we reveal that while the audiotactile and audiovisual pairings show tightly related TBWs, they also exhibit a differential relationship with respect to true and perceived multisensory synchrony. Thus, audiotactile and audiovisual temporal processing share mechanistic features yet are respectively functionally linked to objective and subjective synchrony.

Probing the reaching-grasping network in humans through multivoxel pattern decoding

INTRODUCTION

The quest for a putative human homolog of the reaching-grasping network identified in monkeys has been the focus of many neuropsychological and neuroimaging studies in recent years. These studies have shown that the network underlying reaching-only and reach-to-grasp movements includes the superior parieto-occipital cortex (SPOC), the anterior part of the human intraparietal sulcus (hAIP), the ventral and the dorsal portion of the premotor cortex, and the primary motor cortex (M1). Recent evidence for a wider frontoparietal network coding for different aspects of reaching-only and reach-to-grasp actions calls for a more fine-grained assessment of the reaching-grasping network in humans by exploiting pattern decoding methods (multivoxel pattern analysis--MVPA).

METHODS

Here, we used MPVA on functional magnetic resonance imaging (fMRI) data to assess whether regions of the frontoparietal network discriminate between reaching-only and reach-to-grasp actions, natural and constrained grasping, different grasp types, and object sizes. Participants were required to perform either reaching-only movements or two reach-to-grasp types (precision or whole hand grasp) upon spherical objects of different sizes.

RESULTS

Multivoxel pattern analysis highlighted that, independently from the object size, all the selected regions of both hemispheres contribute in coding for grasp type, with the exception of SPOC and the right hAIP. Consistent with recent neurophysiological findings on monkeys, there was no evidence for a clear-cut distinction between a dorsomedial and a dorsolateral pathway that would be specialized for reaching-only and reach-to-grasp actions, respectively. Nevertheless, the comparison of decoding accuracy across brain areas highlighted their different contributions to reaching-only and grasping actions.

CONCLUSIONS

Altogether, our findings enrich the current knowledge regarding the functional role of key brain areas involved in the cortical control of reaching-only and reach-to-grasp actions in humans, by revealing novel fine-grained distinctions among action types within a wide frontoparietal network.

An invisible touch: Body-related multisensory conflicts modulate visual consciousness

The majority of scientific studies on consciousness have focused on vision, exploring the cognitive and neural mechanisms of conscious access to visual stimuli. In parallel, studies on bodily consciousness have revealed that bodily (i.e. tactile, proprioceptive, visceral, vestibular) signals are the basis for the sense of self. However, the role of bodily signals in the formation of visual consciousness is not well understood. Here we investigated how body-related visuo-tactile stimulation modulates conscious access to visual stimuli. We used a robotic platform to apply controlled tactile stimulation to the participants' back while they viewed a dot moving either in synchrony or asynchrony with the touch on their back. Critically, the dot was rendered invisible through continuous flash suppression. Manipulating the visual context by presenting the dot moving on either a body form, or a non-bodily object we show that: (i) conflict induced by synchronous visuo-tactile stimulation in a body context is associated with a delayed conscious access compared to asynchronous visuo-tactile stimulation, (ii) this effect occurs only in the context of a visual body form, and (iii) is not due to detection or response biases. The results indicate that body-related visuo-tactile conflicts impact visual consciousness by facilitating access of non-conflicting visual information to awareness, and that these are sensitive to the visual context in which they are presented, highlighting the interplay between bodily signals and visual experience.

Looming sensitive cortical regions without V1 input: evidence from a patient with bilateral cortical blindness

Fast and automatic behavioral responses are required to avoid collision with an approaching stimulus. Accordingly, looming stimuli have been found to be highly salient and efficient attractors of attention due to the implication of potential collision and potential threat. Here, we address the question of whether looming motion is processed in the absence of any functional primary visual cortex and consequently without awareness. For this, we investigated a patient (TN) suffering from complete, bilateral damage to his primary visual cortex. Using an fMRI paradigm, we measured TN's brain activation during the presentation of looming, receding, rotating, and static point lights, of which he was unaware. When contrasted with other conditions, looming was found to produce bilateral activation of the middle temporal areas, as well as the superior temporal sulcus and inferior parietal lobe (IPL). The latter are generally thought to be involved in multisensory processing of motion in extrapersonal space, as well as attentional capture and saliency. No activity was found close to the lesioned V1 area. This demonstrates that looming motion is processed in the absence of awareness through direct subcortical projections to areas involved in multisensory processing of motion and saliency that bypass V1.

Tactile representation of the head and shoulders assessed by fMRI in the nonhuman primate

In nonhuman primates, tactile representation at the cortical level has mostly been studied using single-cell recordings targeted to specific cortical areas. In this study, we explored the representation of tactile information delivered to the face or the shoulders at the whole brain level, using functional magnetic resonance imaging (fMRI) in the nonhuman primate. We used air puffs delivered to the center of the face, the periphery of the face, or the shoulders. These stimulations elicited activations in numerous cortical areas, encompassing the primary and secondary somatosensory areas, prefrontal and premotor areas, and parietal, temporal, and cingulate areas as well as low-level visual cortex. Importantly, a specific parieto-temporo-prefrontal network responded to the three stimulations but presented a marked preference for air puffs directed to the center of the face. This network corresponds to areas that are also involved in near-space representation, as well as in the multisensory integration of information at the interface between this near space and the skin of the face, and is probably involved in the construction of a peripersonal space representation around the head.

Frequency tagging of steady-state evoked potentials to explore the crossmodal links in spatial attention between vision and touch

The sustained periodic modulation of a stimulus induces an entrainment of cortical neurons responding to the stimulus, appearing as a steady-state evoked potential (SS-EP) in the EEG frequency spectrum. Here, we used frequency tagging of SS-EPs to study the crossmodal links in spatial attention between touch and vision. We hypothesized that a visual stimulus approaching the left or right hand orients spatial attention toward the approached hand, and thereby enhances the processing of vibrotactile input originating from that hand. Twenty-five subjects took part in the experiment: 16-s trains of vibrotactile stimuli (4.2 and 7.2 Hz) were applied simultaneously to the left and right hand, concomitantly with a punctate visual stimulus blinking at 9.8 Hz. The visual stimulus was approached toward the left or right hand. The hands were either uncrossed (left and right hands to the left and right of the participant) or crossed (left and right hands to the right and left of the participant). The vibrotactile stimuli elicited two distinct SS-EPs with scalp topographies compatible with activity in the contralateral primary somatosensory cortex. The visual stimulus elicited a third SS-EP with a topography compatible with activity in visual areas. When the visual stimulus was over one of the hands, the amplitude of the vibrotactile SS-EP elicited by stimulation of that hand was enhanced, regardless of whether the hands were uncrossed or crossed. This demonstrates a crossmodal effect of spatial attention between vision and touch, integrating proprioceptive and/or visual information to map the position of the limbs in external space.

Visual rehabilitation: visual scanning, multisensory stimulation and vision restoration trainings

Neuropsychological training methods of visual rehabilitation for homonymous vision loss caused by postchiasmatic damage fall into two fundamental paradigms: “compensation” and “restoration”. Existing methods can be classified into three groups: Visual Scanning Training (VST), Audio-Visual Scanning Training (AViST) and Vision Restoration Training (VRT). VST and AViST aim at compensating vision loss by training eye scanning movements, whereas VRT aims at improving lost vision by activating residual visual functions by training light detection and discrimination of visual stimuli. This review discusses the rationale underlying these paradigms and summarizes the available evidence with respect to treatment efficacy. The issues raised in our review should help guide clinical care and stimulate new ideas for future research uncovering the underlying neural correlates of the different treatment paradigms. We propose that both local “within-system” interactions (i.e., relying on plasticity within peri-lesional spared tissue) and changes in more global “between-system” networks (i.e., recruiting alternative visual pathways) contribute to both vision restoration and compensatory rehabilitation, which ultimately have implications for the rehabilitation of cognitive functions.

Evolution of posterior parietal cortex and parietal-frontal networks for specific actions in primates

Posterior parietal cortex (PPC) is an extensive region of the human brain that develops relatively late and is proportionally large compared with that of monkeys and prosimian primates. Our ongoing comparative studies have led to several conclusions about the evolution of this posterior parietal region. In early placental mammals, PPC likely was a small multisensory region much like PPC of extant rodents and tree shrews. In early primates, PPC likely resembled that of prosimian galagos, in which caudal PPC (PPCc) is visual and rostral PPC (PPCr) has eight or more multisensory domains where electrical stimulation evokes different complex motor behaviors, including reaching, hand-to-mouth, looking, protecting the face or body, and grasping. These evoked behaviors depend on connections with functionally matched domains in premotor cortex (PMC) and motor cortex (M1). Domains in each region compete with each other, and a serial arrangement of domains allows different factors to influence motor outcomes successively. Similar arrangements of domains have been retained in New and Old World monkeys, and humans appear to have at least some of these domains. The great expansion and prolonged development of PPC in humans suggest the addition of functionally distinct territories. We propose that, across primates, PMC and M1 domains are second and third levels in a number of parallel, interacting networks for mediating and selecting one type of action over others.

Grasping actions and social interaction: neural bases and anatomical circuitry in the monkey

The study of the neural mechanisms underlying grasping actions showed that cognitive functions are deeply embedded in motor organization. In the first part of this review, we describe the anatomical structure of the motor cortex in the monkey and the cortical and sub-cortical connections of the different motor areas. In the second part, we review the neurophysiological literature showing that motor neurons are not only involved in movement execution, but also in the transformation of object physical features into motor programs appropriate to grasp them (through visuo-motor transformations). We also discuss evidence indicating that motor neurons can encode the goal of motor acts and the intention behind action execution. Then, we describe one of the mechanisms-the mirror mechanism-considered to be at the basis of action understanding and intention reading, and describe the anatomo-functional pathways through which information about the social context can reach the areas containing mirror neurons. Finally, we briefly show that a clear similarity exists between monkey and human in the organization of the motor and mirror systems. Based on monkey and human literature, we conclude that the mirror mechanism relies on a more extended network than previously thought, and possibly subserves basic social functions. We propose that this mechanism is also involved in preparing appropriate complementary response to observed actions, allowing two individuals to become attuned and cooperate in joint actions.

Proprioceptive body illusions modulate the visual perception of reaching distance

The neurobiology of reaching has been extensively studied in human and non-human primates. However, the mechanisms that allow a subject to decide—without engaging in explicit action—whether an object is reachable are not fully understood. Some studies conclude that decisions near the reach limit depend on motor simulations of the reaching movement. Others have shown that the body schema plays a role in explicit and implicit distance estimation, especially after motor practice with a tool. In this study we evaluate the causal role of multisensory body representations in the perception of reachable space. We reasoned that if body schema is used to estimate reach, an illusion of the finger size induced by proprioceptive stimulation should propagate to the perception of reaching distances. To test this hypothesis we induced a proprioceptive illusion of extension or shrinkage of the right index finger while participants judged a series of LEDs as reachable or non-reachable without actual movement. Our results show that reach distance estimation depends on the illusory perceived size of the finger: illusory elongation produced a shift of reaching distance away from the body whereas illusory shrinkage produced the opposite effect. Combining these results with previous findings, we suggest that deciding if a target is reachable requires an integration of body inputs in high order multisensory parietal areas that engage in movement simulations through connections with frontal premotor areas.

Organization and evolution of parieto-frontal processing streams in macaque monkeys and humans

The functional organization of the parieto-frontal system is crucial for understanding cognitive-motor behavior and provides the basis for interpreting the consequences of parietal lesions in humans from a neurobiological perspective. The parieto-frontal connectivity defines some main information streams that, rather than being devoted to restricted functions, underlie a rich behavioral repertoire. Surprisingly, from macaque to humans, evolution has added only a few, new functional streams, increasing however their complexity and encoding power. In fact, the characterization of the conduction times of parietal and frontal areas to different target structures has recently opened a new window on cortical dynamics, suggesting that evolution has amplified the probability of dynamic interactions between the nodes of the network, thanks to communication patterns based on temporally-dispersed conduction delays. This might allow the representation of sensory-motor signals within multiple neural assemblies and reference frames, as to optimize sensory-motor remapping within an action space characterized by different and more complex demands across evolution.

Optic-flow selective cortical sensory regions associated with self-reported states of vection

Optic flow is one of the most important visual cues to the estimation of self-motion. It has repeatedly been demonstrated that a cortical network including visual, multisensory, and vestibular areas is implicated in processing optic flow; namely, visual areas middle temporal cortex (MT+), V6; multisensory areas ventral intra-parietal area (VIP), cingulate sulcus visual area, precuneus motion area (PcM); and vestibular areas parieto-insular vestibular cortex (PIVC) and putative area 2v (p2v). However, few studies have investigated the roles of and interaction between the optic-flow selective sensory areas within the context of self-motion perception. When visual information (i.e., optic flow) is the sole cue to computing self-motion parameters, the discrepancy amongst the sensory signals may induce an illusion of self-motion referred to as ‘vection.’ This study aimed to identify optic-flow selective sensory areas that are involved in the processing of visual cues to self-motion, by introducing vection as an index and assessing activation in which of those areas reflect vection, using functional magnetic resonance imaging. The results showed that activity in visual areas MT+ and V6, multisensory area VIP and vestibular area PIVC was significantly greater while participants were experiencing vection, as compared to when they were experiencing no vection, which may indicate that activation in MT+, V6, VIP, and PIVC reflects vection. The results also place VIP in a good position to integrate visual cues related to self-motion and vestibular information.

Neural substrates underlying the passive observation and active control of translational egomotion

Moving or static obstacles often get in the way while walking in daily life. Avoiding obstacles involves both perceptual processing of motion information and controlling appropriate defensive movements. Several higher-level motion areas, including the ventral intraparietal area (VIP), medial superior temporal area, parieto-insular vestibular cortex (PIVC), areas V6 and V6A, and cingulate sulcus visual area, have been identified in humans by passive viewing of optic flow patterns that simulate egomotion and object motion. However, the roles of these areas in the active control of egomotion in the real world remain unclear. Here, we used functional magnetic resonance imaging (fMRI) to map the neural substrates underlying the passive observation and active control of translational egomotion in humans. A wide-field virtual reality environment simulated a daily scenario where doors randomly swing outward while walking in a hallway. The stimuli of door-dodging events were essentially the same in two event-related fMRI experiments, which compared passive and active dodges in response to swinging doors. Passive dodges were controlled by a computer program, while active dodges were controlled by the subject. Passive dodges activated several higher-level areas distributed across three dorsal motion streams in the temporal, parietal, and cingulate cortex. Active dodges most strongly activated the temporal-vestibular stream, with peak activation located in the right PIVC. Other higher-level motion areas including VIP showed weaker to no activation in active dodges. These results suggest that PIVC plays an active role in sensing and guiding translational egomotion that moves an observer aside from impending obstacles.

Impact prediction by looming visual stimuli enhances tactile detection

From an ecological point of view, approaching objects are potentially more harmful than receding objects. A predator, a dominant conspecific, or a mere branch coming up at high speed can all be dangerous if one does not detect them and produce the appropriate escape behavior fast enough. And indeed, looming stimuli trigger stereotyped defensive responses in both monkeys and human infants. However, while the heteromodal somatosensory consequences of visual looming stimuli can be fully predicted by their spatiotemporal dynamics, few studies if any have explored whether visual stimuli looming toward the face predictively enhance heteromodal tactile sensitivity around the expected time of impact and at its expected location on the body. In the present study, we report that, in addition to triggering a defensive motor repertoire, looming stimuli toward the face provide the nervous system with predictive cues that enhance tactile sensitivity on the face. Specifically, we describe an enhancement of tactile processes at the expected time and location of impact of the stimulus on the face. We additionally show that a looming stimulus that brushes past the face also enhances tactile sensitivity on the nearby cheek, suggesting that the space close to the face is incorporated into the subjects' body schema. We propose that this cross-modal predictive facilitation involves multisensory convergence areas subserving the representation of a peripersonal space and a safety boundary of self.

Development of space perception in relation to the maturation of the motor system in infant rhesus macaques (Macaca mulatta)

To act on the environment, organisms must perceive object locations in relation to their body. Several neuroscientific studies provide evidence of neural circuits that selectively represent space within reach (i.e., peripersonal) and space outside of reach (i.e., extrapersonal). However, the developmental emergence of these space representations remains largely unexplored. We investigated the development of space coding in infant macaques and found that they exhibit different motor strategies and hand configurations depending on the objects' size and location. Reaching-grasping improved from 2 to 4 weeks of age, suggesting a broadly defined perceptual body schema at birth, modified by the acquisition and refinement of motor skills through early sensorimotor experience, enabling the development of a mature capacity for coding space.

Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception

Which is my body and how do I distinguish it from the bodies of others, or from objects in the surrounding environment? The perception of our own body and more particularly our sense of body ownership is taken for granted. Nevertheless, experimental findings from body ownership illusions (BOIs), show that under specific multisensory conditions, we can experience artificial body parts or fake bodies as our own body parts or body, respectively. The aim of the present paper is to discuss how and why BOIs are induced. We review several experimental findings concerning the spatial, temporal, and semantic principles of crossmodal stimuli that have been applied to induce BOIs. On the basis of these principles, we discuss theoretical approaches concerning the underlying mechanism of BOIs. We propose a conceptualization based on Bayesian causal inference for addressing how our nervous system could infer whether an object belongs to our own body, using multisensory, sensorimotor, and semantic information, and we discuss how this can account for several experimental findings. Finally, we point to neural network models as an implementational framework within which the computational problem behind BOIs could be addressed in the future.

Role of visual and non-visual cues in constructing a rotation-invariant representation of heading in parietal cortex

As we navigate through the world, eye and head movements add rotational velocity patterns to the retinal image. When such rotations accompany observer translation, the rotational velocity patterns must be discounted to accurately perceive heading. The conventional view holds that this computation requires efference copies of self-generated eye/head movements. Here we demonstrate that the brain implements an alternative solution in which retinal velocity patterns are themselves used to dissociate translations from rotations. These results reveal a novel role for visual cues in achieving a rotation-invariant representation of heading in the macaque ventral intraparietal area. Specifically, we show that the visual system utilizes both local motion parallax cues and global perspective distortions to estimate heading in the presence of rotations. These findings further suggest that the brain is capable of performing complex computations to infer eye movements and discount their sensory consequences based solely on visual cues.

DOI:http://dx.doi.org/10.7554/eLife.04693.001

When strolling along a path beside a busy street, we can look around without losing our stride. The things we see change as we walk forward, and our view also changes if we turn our head—for example, to look at a passing car. Nevertheless, we can still tell that we are walking in a straight-line because our brain is able to compute the direction in which we are heading by discounting the visual changes caused by rotating our head or eyes.

It remains unclear how the brain gets the information about head and eye movements that it would need to be able to do this. Many researchers had proposed that the brain estimates these rotations by using a copy of the neural signals that are sent to the muscles to move the eyes or head. However, it is possible that the brain can estimate head and eye rotations by directly analyzing the visual information from the eyes. One region of the brain that may contribute to this process is the ventral intraparietal area or ‘area VIP’ for short.

Sunkara et al. devised an experiment that can help distinguish the effects of visual cues from copies of neural signals sent to the muscles during eye rotations. This involved training monkeys to look at a 3D display of moving dots, which gives the impression of moving through space. Sunkara et al. then measured the electrical signals in area VIP either when the monkey moved its eyes (to follow a moving target), or when the display changed to give the monkey the same visual cues as if it had rotated its eyes, when in fact it had not.

Sunkara et al. found that the electrical signals recorded in area VIP when the monkey was given the illusion of rotating its eyes were similar to the signals recorded when the monkey actually rotated its eyes. This suggests that visual cues play an important role in correcting for the effects of eye rotations and correctly estimating the direction in which we are heading. Further research into the mechanisms behind this neural process could lead to new vision-based treatments for medical disorders that cause people to have balance problems. Similar research could also help to identify ways to improve navigation in automated vehicles, such as driverless cars.

DOI:http://dx.doi.org/10.7554/eLife.04693.002

Extending peripersonal space representation without tool-use: evidence from a combined behavioral-computational approach

Stimuli from different sensory modalities occurring on or close to the body are integrated in a multisensory representation of the space surrounding the body, i.e., peripersonal space (PPS). PPS dynamically modifies depending on experience, e.g., it extends after using a tool to reach far objects. However, the neural mechanism underlying PPS plasticity after tool use is largely unknown. Here we use a combined computational-behavioral approach to propose and test a possible mechanism accounting for PPS extension. We first present a neural network model simulating audio-tactile representation in the PPS around one hand. Simulation experiments showed that our model reproduced the main property of PPS neurons, i.e., selective multisensory response for stimuli occurring close to the hand. We used the neural network model to simulate the effects of a tool-use training. In terms of sensory inputs, tool use was conceptualized as a concurrent tactile stimulation from the hand, due to holding the tool, and an auditory stimulation from the far space, due to tool-mediated action. Results showed that after exposure to those inputs, PPS neurons responded also to multisensory stimuli far from the hand. The model thus suggests that synchronous pairing of tactile hand stimulation and auditory stimulation from the far space is sufficient to extend PPS, such as after tool-use. Such prediction was confirmed by a behavioral experiment, where we used an audio-tactile interaction paradigm to measure the boundaries of PPS representation. We found that PPS extended after synchronous tactile-hand stimulation and auditory-far stimulation in a group of healthy volunteers. Control experiments both in simulation and behavioral settings showed that the same amount of tactile and auditory inputs administered out of synchrony did not change PPS representation. We conclude by proposing a simple, biological-plausible model to explain plasticity in PPS representation after tool-use, which is supported by computational and behavioral data.

Computational cognitive models of spatial memory in navigation space: a review

Spatial memory refers to the part of the memory system that encodes, stores, recognizes and recalls spatial information about the environment and the agent's orientation within it. Such information is required to be able to navigate to goal locations, and is vitally important for any embodied agent, or model thereof, for reaching goals in a spatially extended environment. In this paper, a number of computationally implemented cognitive models of spatial memory are reviewed and compared. Three categories of models are considered: symbolic models, neural network models, and models that are part of a systems-level cognitive architecture. Representative models from each category are described and compared in a number of dimensions along which simulation models can differ (level of modeling, types of representation, structural accuracy, generality and abstraction, environment complexity), including their possible mapping to the underlying neural substrate. Neural mappings are rarely explicated in the context of behaviorally validated models, but they could be useful to cognitive modeling research by providing a new approach for investigating a model's plausibility. Finally, suggested experimental neuroscience methods are described for verifying the biological plausibility of computational cognitive models of spatial memory, and open questions for the field of spatial memory modeling are outlined.

The what, where, and why of priority maps and their interactions with visual working memory

Priority maps are winner-take-all neural mechanisms thought to guide the allocation of covert and overt attention. Here, we go beyond this standard definition and argue that priority maps play a much broader role in controlling goal-directed behavior. We start by defining what priority maps are and where they might be found in the brain; we then ask why they exist-the function that they serve. We propose that this function is to communicate a goal state to the different effector systems, thereby guiding behavior. Within this framework, we speculate on how priority maps interact with visual working memory and introduce our common source hypothesis, the suggestion that this goal state is maintained in visual working memory and used to construct all of the priority maps controlling the various motor systems. Finally, we look ahead and suggest questions about priority maps that should be asked next.

Boosting pitch encoding with audiovisual interactions in congenital amusia

The combination of information across senses can enhance perception, as revealed for example by decreased reaction times or improved stimulus detection. Interestingly, these facilitatory effects have been shown to be maximal when responses to unisensory modalities are weak. The present study investigated whether audiovisual facilitation can be observed in congenital amusia, a music-specific disorder primarily ascribed to impairments of pitch processing. Amusic individuals and their matched controls performed two tasks. In Task 1, they were required to detect auditory, visual, or audiovisual stimuli as rapidly as possible. In Task 2, they were required to detect as accurately and as rapidly as possible a pitch change within an otherwise monotonic 5-tone sequence that was presented either only auditorily (A condition), or simultaneously with a temporally congruent, but otherwise uninformative visual stimulus (AV condition). Results of Task 1 showed that amusics exhibit typical auditory and visual detection, and typical audiovisual integration capacities: both amusics and controls exhibited shorter response times for audiovisual stimuli than for either auditory stimuli or visual stimuli. Results of Task 2 revealed that both groups benefited from simultaneous uninformative visual stimuli to detect pitch changes: accuracy was higher and response times shorter in the AV condition than in the A condition. The audiovisual improvements of response times were observed for different pitch interval sizes depending on the group. These results suggest that both typical listeners and amusic individuals can benefit from multisensory integration to improve their pitch processing abilities and that this benefit varies as a function of task difficulty. These findings constitute the first step towards the perspective to exploit multisensory paradigms to reduce pitch-related deficits in congenital amusia, notably by suggesting that audiovisual paradigms are effective in an appropriate range of unimodal performance.

Predictive coding accounts of shared representations in parieto-insular networks

The discovery of mirror neurons in the ventral premotor cortex (area F5) and inferior parietal cortex (area PFG) in the macaque monkey brain has provided the physiological evidence for direct matching of the intrinsic motor representations of the self and the visual image of the actions of others. The existence of mirror neurons implies that the brain has mechanisms reflecting shared self and other action representations. This may further imply that the neural basis self-body representations may also incorporate components that are shared with other-body representations. It is likely that such a mechanism is also involved in predicting other's touch sensations and emotions. However, the neural basis of shared body representations has remained unclear. Here, we propose a neural basis of body representation of the self and of others in both human and non-human primates. We review a series of behavioral and physiological findings which together paint a picture that the systems underlying such shared representations require integration of conscious exteroception and interoception subserved by a cortical sensory-motor network involving parieto-inner perisylvian circuits (the ventral intraparietal area [VIP]/inferior parietal area [PFG]-secondary somatosensory cortex [SII]/posterior insular cortex [pIC]/anterior insular cortex [aIC]). Based on these findings, we propose a computational mechanism of the shared body representation in the predictive coding (PC) framework. Our mechanism proposes that processes emerging from generative models embedded in these specific neuronal circuits play a pivotal role in distinguishing a self-specific body representation from a shared one. The model successfully accounts for normal and abnormal shared body phenomena such as mirror-touch synesthesia and somatoparaphrenia. In addition, it generates a set of testable experimental predictions.

Sliding perspectives: dissociating ownership from self-location during full body illusions in virtual reality

Bodily illusions have been used to study bodily self-consciousness and disentangle its various components, among other the sense of ownership and self-location. Congruent multimodal correlations between the real body and a fake humanoid body can in fact trigger the illusion that the fake body is one's own and/or disrupt the unity between the perceived self-location and the position of the physical body. However, the extent to which changes in self-location entail changes in ownership is still matter of debate. Here we address this problem with the support of immersive virtual reality. Congruent visuotactile stimulation was delivered on healthy participants to trigger full body illusions from different visual perspectives, each resulting in a different degree of overlap between real and virtual body. Changes in ownership and self-location were measured with novel self-posture assessment tasks and with an adapted version of the cross-modal congruency task. We found that, despite their strong coupling, self-location and ownership can be selectively altered: self-location was affected when having a third person perspective over the virtual body, while ownership toward the virtual body was experienced only in the conditions with total or partial overlap. Thus, when the virtual body is seen in the far extra-personal space, changes in self-location were not coupled with changes in ownership. If a partial spatial overlap is present, ownership was instead typically experienced with a boosted change in the perceived self-location. We discussed results in the context of the current knowledge of the multisensory integration mechanisms contributing to self-body perception. We argue that changes in the perceived self-location are associated to the dynamical representation of peripersonal space encoded by visuotactile neurons. On the other hand, our results speak in favor of visuo-proprioceptive neuronal populations being a driving trigger in full body ownership illusions.

Visual selectivity for heading in the macaque ventral intraparietal area

The patterns of optic flow seen during self-motion can be used to determine the direction of one's own heading. Tracking eye movements which typically occur during everyday life alter this task since they add further retinal image motion and (predictably) distort the retinal flow pattern. Humans employ both visual and nonvisual (extraretinal) information to solve a heading task in such case. Likewise, it has been shown that neurons in the monkey medial superior temporal area (area MST) use both signals during the processing of self-motion information. In this article we report that neurons in the macaque ventral intraparietal area (area VIP) use visual information derived from the distorted flow patterns to encode heading during (simulated) eye movements. We recorded responses of VIP neurons to simple radial flow fields and to distorted flow fields that simulated self-motion plus eye movements. In 59% of the cases, cell responses compensated for the distortion and kept the same heading selectivity irrespective of different simulated eye movements. In addition, response modulations during real compared with simulated eye movements were smaller, being consistent with reafferent signaling involved in the processing of the visual consequences of eye movements in area VIP. We conclude that the motion selectivities found in area VIP, like those in area MST, provide a way to successfully analyze and use flow fields during self-motion and simultaneous tracking movements.

Eye position modulates retinotopic responses in early visual areas: a bias for the straight-ahead direction

Even though the eyes constantly change position, the location of a stimulus can be accurately represented by a population of neurons with retinotopic receptive fields modulated by eye position gain fields. Recent electrophysiological studies, however, indicate that eye position gain fields may serve an additional function since they have a non-uniform spatial distribution that increases the neural response to stimuli in the straight-ahead direction. We used functional magnetic resonance imaging and a wide-field stimulus display to determine whether gaze modulations in early human visual cortex enhance the blood-oxygenation-level dependent (BOLD) response to stimuli that are straight-ahead. Subjects viewed rotating polar angle wedge stimuli centered straight-ahead or vertically displaced by ±20° eccentricity. Gaze position did not affect the topography of polar phase-angle maps, confirming that coding was retinotopic, but did affect the amplitude of the BOLD response, consistent with a gain field. In agreement with recent electrophysiological studies, BOLD responses in V1 and V2 to a wedge stimulus at a fixed retinal locus decreased when the wedge location in head-centered coordinates was farther from the straight-ahead direction. We conclude that stimulus-evoked BOLD signals are modulated by a systematic, non-uniform distribution of eye-position gain fields.

Crossmodal integration: a glimpse into the development of sensory remapping

Correctly localising sensory stimuli in space is a formidable challenge for the newborn brain. A new study provides a first glimpse into how human brain mechanisms for sensory remapping develop in the first year of life.