Functional properties of rotation-sensitive neurons in the posterior parietal association cortex of the monkey

The Ventral Posterior Lateral Thalamus Preferentially Encodes Externally Applied Versus Active Movement: Implications for Self-Motion Perception

Successful interaction with our environment requires that voluntary behaviors be precisely coordinated with our perception of self-motion. The vestibular sensors in the inner ear detect self-motion and in turn send projections via the vestibular nuclei to multiple cortical areas through 2 principal thalamocortical pathways, 1 anterior and 1 posterior. While the anterior pathway has been extensively studied, the role of the posterior pathway is not well understood. Accordingly, here we recorded responses from individual neurons in the ventral posterior lateral thalamus of macaque monkeys during externally applied (passive) and actively generated self-motion. The sensory responses of neurons that robustly encoded passive rotations and translations were canceled during comparable voluntary movement (~80% reduction). Moreover, when both passive and active self-motion were experienced simultaneously, neurons selectively encoded the detailed time course of the passive component. To examine the mechanism underlying the selective elimination of vestibular sensitivity to active motion, we experimentally controlled correspondence between intended and actual head movement. We found that suppression only occurred if the actual sensory consequences of motion matched the motor-based expectation. Together, our findings demonstrate that the posterior thalamocortical vestibular pathway selectively encodes unexpected motion, thereby providing a neural correlate for ensuring perceptual stability during active versus externally generated motion.

Going with the Flow: The Neural Mechanisms Underlying Illusions of Complex-Flow Motion

Studying the mismatch between perception and reality helps us better understand the constructive nature of the visual brain. The Pinna-Brelstaff motion illusion is a compelling example illustrating how a complex moving pattern can generate an illusory motion perception. When an observer moves toward (expansion) or away (contraction) from the Pinna-Brelstaff figure, the figure appears to rotate. The neural mechanisms underlying the illusory complex-flow motion of rotation, expansion, and contraction remain unknown. We studied this question at both perceptual and neuronal levels in behaving male macaques by using carefully parametrized Pinna-Brelstaff figures that induce the above motion illusions. We first demonstrate that macaques perceive illusory motion in a manner similar to that of human observers. Neurophysiological recordings were subsequently performed in the middle temporal area (MT) and the dorsal portion of the medial superior temporal area (MSTd). We find that subgroups of MSTd neurons encoding a particular global pattern of real complex-flow motion (rotation, expansion, contraction) also represent illusory motion patterns of the same class. They require an extra 15 ms to reliably discriminate the illusion. In contrast, MT neurons encode both real and illusory local motions with similar temporal delays. These findings reveal that illusory complex-flow motion is first represented in MSTd by the same neurons that normally encode real complex-flow motion. However, the extraction of global illusory motion in MSTd from other classes of real complex-flow motion requires extra processing time. Our study illustrates a cascaded integration mechanism from MT to MSTd underlying the transformation from external physical to internal nonveridical flow-motion perception.SIGNIFICANCE STATEMENT The neural basis of the transformation from objective reality to illusory percepts of rotation, expansion, and contraction remains unknown. We demonstrate psychophysically that macaques perceive these illusory complex-flow motions in a manner similar to that of human observers. At the neural level, we show that medial superior temporal (MSTd) neurons represent illusory flow motions as if they were real by globally integrating middle temporal area (MT) local motion signals. Furthermore, while MT neurons reliably encode real and illusory local motions with similar temporal delays, MSTd neurons take a significantly longer time to process the signals associated with illusory percepts. Our work extends previous complex-flow motion studies by providing the first detailed analysis of the neuron-specific mechanisms underlying complex forms of illusory motion integration from MT to MSTd.

A Rotational Motion Perception Neural Network Based on Asymmetric Spatiotemporal Visual Information Processing

All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.

Dependence of Illusory Motion on Directional Consistency in Oblique Components

Pinna and Brelstaff (2000 Vision Research40 2091–2096) reported a motion illusion on viewing two concentric circles consisting of quadrangular components with black and white sides on a grey background. Our results suggest that the illusion is based on the integration of motion signals derived from oblique components, and on the consistency in the direction among those components. Furthermore, arrays of these oblique components can elicit the perception of motion not only for the oblique components themselves, but also for other objects in the picture. We propose that the motion illusion depends not only upon detection of the illusory motion signal at each local oblique component, but also upon the accumulation of the signal all over the stimulus configuration.

The effect of rotating random dot motion on visuospatial line orientation in patients with right-sided stroke

Spatial deficits are frequent after brain damage, particularly right hemisphere stroke. Visual judgments of line orientation (LINE) are often impaired after right parietal lesions. Perception of line orientation is an important visuoperceptual component of visuoconstructive capacities. Yet, little is known about modulating factors in LINE and effective treatments are rare for this disorder. Studies in patients with spatial neglect show that horizontal random dot motion (RDM) significantly modulates horizontal spatial disorders, both transiently and permanently after treatment. In the current study, we investigated whether rotational RDM modulates judgements in an oblique LINE task in 20 patients with right-hemispheric first ever stroke (10 of them with a disorder in LINE and 10 without such a disorder), and 10 healthy, age-matched subjects. Subjects were tested under three experimental conditions: (1) with a static background of small white dots, (2) with slow clockwise or (3) counterclockwise circular RDM of these background stimuli, while they performed the LINE task. In the baseline condition with static background, the impaired patient group showed a significant counterclockwise tilt. Clockwise rotating RDM normalized this deficit transiently but completely, while counterclockwise rotating RDM slightly aggravated it, though not significantly. Tilts in the LINE task were significantly correlated with left visuospatial neglect. Similar but much smaller effects were obtained in the spatially unimpaired patients and the normal controls. These results show that rotational RDM modulates deficits of line orientation in patients with right-sided stroke, possibly by influencing higher spatial representations devoted to the perception of oblique lines.

Performance characterization of Watson Ahumada motion detector using random dot rotary motion stimuli

The performance of Watson & Ahumada's model of human visual motion sensing is compared against human psychophysical performance. The stimulus consists of random dots undergoing rotary motion, displayed in a circular annulus. The model matches psychophysical observer performance with respect to most parameters. It is able to replicate some key psychophysical findings such as invariance of observer performance to dot density in the display, and decrease of observer performance with frame duration of the display.

Associated with the concept of rotary motion is the notion of a center about which rotation occurs. One might think that for accurate estimation of rotary motion in the display, this center must be accurately known. A simple vector analysis reveals that this need not be the case. Numerical simulations confirm this result, and may explain the position invariance of MST(d) cells. Position invariance is the experimental finding that rotary motion sensitive cells are insensitive to where in their receptive field rotation occurs.

When all the dots in the display are randomly drawn from a uniform distribution, illusory rotary motion is perceived. This case was investigated by Rose & Blake previously, who termed the illusory rotary motion the omega effect. Two important experimental findings are reported concerning this effect. First, although the display of random dots evokes perception of rotary motion, the direction of motion perceived does not depend on what dot pattern is shown. Second, the time interval between spontaneous flips in perceived direction is lognormally distributed (mode≈2 s). These findings suggest the omega effect fits in the category of a typical bistable illusion, and therefore the processes that give rise to this illusion may be the same processes that underlie much of other bistable phenomenon.

Multimodal coding of three-dimensional rotation and translation in area MSTd: comparison of visual and vestibular selectivity

Recent studies have shown that most neurons in the dorsal medial superior temporal area (MSTd) signal the direction of self-translation (i.e., heading) in response to both optic flow and inertial motion. Much less is currently known about the response properties of MSTd neurons during self-rotation. We have characterized the three-dimensional tuning of MSTd neurons while monkeys passively fixated a central, head-fixed target. Rotational stimuli were either presented using a motion platform or simulated visually using optic flow. Nearly all MSTd cells were significantly tuned for the direction of rotation in the absence of optic flow, with more neurons preferring roll than pitch or yaw rotations. The preferred rotation axis in response to optic flow was generally the opposite of that during physical rotation. This result differs sharply from our findings for translational motion, where approximately half of MSTd neurons have congruent visual and vestibular preferences. By testing a subset of neurons with combined visual and vestibular stimulation, we also show that the contributions of visual and vestibular cues to MSTd responses depend on the relative reliabilities of the two stimulus modalities. Previous studies of MSTd responses to motion in darkness have assumed a vestibular origin for the activity observed. We have directly verified this assumption by recording from MSTd neurons after bilateral labyrinthectomy. Selectivity for physical rotation and translation stimuli was eliminated after labyrinthectomy, whereas selectivity to optic flow was unaffected. Overall, the lack of MSTd neurons with congruent rotation tuning for visual and vestibular stimuli suggests that MSTd does not integrate these signals to produce a robust perception of self-rotation. Vestibular rotation signals in MSTd may instead be used to compensate for the confounding effects of rotatory head movements on optic flow.

Translation speed compensation in the dorsal aspect of the medial superior temporal area

The dorsal aspect of the medial superior temporal area (MSTd) is involved in the computation of heading direction from the focus of expansion (FOE) of the visual image. Our laboratory previously found that MSTd neurons adjust their focus tuning curves to compensate for shifts in the FOE produced by eye rotation (Bradley et al., 1996) as well as for changes in pursuit speed (Shenoy et al., 2002). The translation speed of an observer also affects the shift of the FOE. To investigate whether MSTd neurons can adjust their focus tuning curves to compensate for varying translation speeds, we recorded extracellular responses from 93 focus-tuned MSTd neurons in two rhesus monkeys (Macaca mulatta) performing pursuit eye movements across displays of varying translation speeds. We found that MSTd neurons had larger shifts in their tuning curves for slow translation speeds and smaller shifts for fast translation speeds. These shifts aligned the focus tuning curves with the true heading direction and not with the retinal position of the FOE. Because the eye was pursuing at the same rate for varying translation speeds, these results indicate that retinal cues related both to translation speed and extraretinal signals from pursuit eye movements are used by MSTd neurons to compute heading direction.

The continuous Wagon Wheel Illusion is object-based

The occurrence of perceived reversed motion while observers view a periodic, continuously moving stimulus (the "continuous Wagon Wheel Illusion") has been taken as evidence that some aspects of motion perception rely on discrete sampling of visual information. The spatial extent of this sampling is currently under debate. When two separate motion stimuli are viewed simultaneously, the illusion of reversed motion rarely occurs for both objects together: this rules out global sampling of the visual field. The same result holds when the objects are superimposed by transparency: this argues against location-based sampling. Here we show that the sampling is in fact object-based: we use a rotating ring stimulus split in two halves. When the two halves move in opposite directions, appearing to belong to separate objects, perceptual reversals occur in either half at a time, but rarely in both. When the two halves physically move in compatible directions, they generally appear to reverse simultaneously: the illusion keeps the perceptual object united. Rather than the local low-level properties of the motion stimulus (which are comparable in both cases), it is thus the high-level organization of the scene that determines the extent of perceived motion reversals. These results imply that the continuous Wagon Wheel Illusion, and any discrete perceptual sampling that may cause it, is restricted to the object of our attention.

Competition in bistable vision is attribute-specific

We employ ambiguous figures and rivalrous stimuli that have multiple ambiguous properties to show that the different attributes of an ambiguous stimulus can undergo independent switching dynamics. This suggests that competition is distributed and attribute-specific, consistent with the known functional segregation of visual processing. Conflicting evidence that binocular rivalry is an early or late visual process may be better understood as evidence for attribute-specific competition occurring at multiple stages of visual processing. Specifically, we show that whether perceptual selection during binocular rivalry is early and eye-based or late and percept-based depends on the particular ambiguous attributes of the rivalrous stimulus.

Neural mechanisms of three-dimensional vision

We can see things in three dimensions because the visual system re-constructs the three-dimensional (3D) configurations of objects from their two-dimensional (2D) images projected onto the retinas. The purpose of this paper is to give an overview of the psychological background and recent physiological findings concerning three-dimensional vision. Psychophysical and computational studies have suggested that in the visual system the 3D surface orientation is first estimated independently from individual depth cues--such as binocular disparity, as well as various monocular cues including texture gradients--and then the information from these different depth cues is integrated to construct a generalized representation of the 3D surface geometry. Neurons involved in low-level disparity processing, or the detection of local absolute disparity, were found mainly in the occipital cortex, whereas neurons involved in high-level disparity processing, or the reconstruction of 3D surface orientation through the computation of disparity gradients, were found mainly in the parietal area caudal intraparietal sulcus (CIP). Neurons sensitive to texture gradients, which is one of the major monocular cues, were also found in CIP. The majority of these neurons were sensitive to disparity gradients as well, suggesting their involvement in the computation of 3D surface orientation. In CIP, neurons sensitive to multiple depth cues were widely distributed together with those sensitive to a specific depth cue, suggesting CIP's involvement in the integration of depth information from different sources. In addition, human and monkey imaging studies have indicated convergence of multiple depth cues in CIP. These neurophysiological findings suggest that CIP plays a critical role in 3D vision by constructing a generalized representation of the 3D surface geometry of objects.

Three-dimensional structure-from-motion selectivity in the anterior superior temporal polysensory area, STPa, of the behaving monkey

Human and non-human primates are able to perceive three-dimensional structure from motion displays. Three-dimensional structure-from-motion (object-motion) displays were used to test the hypothesis that neurons in the anterior division of the superior temporal polysensory area (STPa) of monkeys can selectively respond to three-dimensional structure-from-motion. Monkeys performed a reaction time task that required the detection of a change in the fraction of structure in three-dimensional transparent sphere displays. Neurons were able to distinguish structured and unstructured three-dimensional optic flow. These cells could differentiate the change in structure-from-motion at stimulus presentation and when the animal was detecting the amount of structure in the display. Some of these neurons were also tuned for characteristics of the sphere stimuli. Cells were also tested with navigational motion and many were found to respond both to three-dimensional structure-from-motion and navigational motion. These results suggest that STPa neurons represent specific aspects of three-dimensional surface structure and that neurons within STPa contribute to the perception of three-dimensional structure-from-motion.

Stabilized Structure from Motion without Disparity Induces Disparity Adaptation

3D structures can be perceived based on the patterns of 2D motion signals. With orthographic projection of a 3D stimulus onto a 2D plane, the kinetic information can give a vivid impression of depth, but the depth order is intrinsically ambiguous, resulting in bistable or even multistable interpretations. For example, an orthographic projection of dots on the surface of a rotating cylinder is perceived as a rotating cylinder with ambiguous direction of rotation. We show that the bistable rotation can be stabilized by adding information, not to the dots themselves, but to their spatial context. More interestingly, the stabilized bistable motion can generate consistent rotation aftereffects. The rotation aftereffect can only be observed when the adapting and test stimuli are presented at the same stereo depth and the same retinal location, and it is not due to attentional tracking. The observed rotation aftereffect is likely due to direction-contingent disparity adaptation, implying that stimuli with kinetic depth may have activated neurons sensitive to different disparities, even though the stimuli have zero relative disparity. Stereo depth and kinetic depth may be supported by a common neural mechanism at an early stage in the visual system.

Vestibular signals of posterior parietal cortex neurons during active and passive head movements in macaque monkeys

The posterior parietal cortex may function as an interface between sensory and motor cortices and thus could be involved in the formation of motor plans as well as abstract representations of space. We have recorded from neurons in the intraparietal sulcus, namely, the ventral and medial intraparietal areas (VIP and MIP, respectively), and analyzed their head-movement-related signals in relation to passive and active movements. To generate active head movements, we made the animals track a moving fixation spot in the horizontal plane under head-free conditions. When under certain circumstances the animals were tracking the fixation spot almost exclusively via head movements, a clear correlation between neuronal firing rate and head movement could be established. Furthermore, a newly employed paradigm, the "replay method," made available direct comparison of neuronal firing behavior under active and passive movement conditions. In such case, the animals were allowed to make spontaneous head movements in darkness. Subsequently, the heads were fixed and the previously recorded active head-movement profile was reproduced by a turntable as passive stimulation. Neuronal responses ranged from total extinction of the vestibular signal during active movement to presence of activity only during active movement. Furthermore, in approximately one-third of the neurons, a change of vestibular on-direction depending on active versus passive movement mode was observed, that is, type I neurons became type II neurons, etc. We suggest that the role of parietal vestibular neurons has to be sought in sensory space representation rather than reflex behavior and motor control contexts.

Vestibular response kinematics in posterior parietal cortex neurons of macaque monkeys

Perception of extrapersonal space is a fundamental requirement for accurate interaction with the environment and moving in it. Parietal cortical areas are thought to play an important role in this function. A significant sensory input to this area arrives from the vestibular system. We quantified neuronal responses in the ventral intraparietal area and the medial intraparietal area of awake head-fixed macaque monkeys during classical vestibular sinusoidal stimulation protocols and with a newly developed random vestibular testing paradigm. The goal was to study more specifically the signal content of parietal vestibular neurons with respect to head movement kinematics. Traditional sinusoidal stimulation analysis revealed that about one-third of the neurons responded in phase with either head position or head acceleration, besides classical head velocity tuning. Random vestibular stimulation revealed more complex signal profiles in the majority of neurons, although quantification of the kinematic variables that drove the neurons most effectively led to similar results to phase shift analysis. Thus, a majority of cells was principally driven by head velocity, and a minority by either acceleration or position. Nevertheless, random stimulation also revealed the simultaneous presence of all three kinematic response parameters (i.e. velocity, position and acceleration) in a majority of neurons. A minority of cells coded only two kinematic variables, i.e. head velocity coupled with either acceleration or position. Neurons coding only one kinematic variable were not found. We hereby demonstrate for the first time that central vestibular neurons carry several head movement kinematic variables simultaneously.

Functional organization of parietal neuronal responses to optic-flow stimuli

We analyzed the dissimilarity matrix of neuronal responses to moving visual stimuli using tree clustering and multidimensional scaling (MDS). Single-cell activity was recorded in area 7a while random dots moving coherently in eight different kinds of motion (right-, left-, up-, and downward, clockwise, counterclockwise, expansion, contraction) were presented to behaving monkeys with eyes fixated. Tree clustering analyses showed that the [rightward, leftward], [upward, downward], and [clockwise, counterclockwise]] motions were clustered in three separate branches (i.e., horizontal, vertical, and rotatory motion, respectively). In contrast, expansion was in a lone branch, whereas contraction was also separate but within a larger cluster. The distances among these clusters were then subjected to an MDS analysis to identify the dimensions underlying the tree clustering observed. This analysis revealed two major factors in operation. The first factor separated expansion from all other stimulus motions, which seems to reflect the prominence of expansion during the common activity of locomotion. In contrast, the second factor separated planar motions from motion in depth, which suggests that the latter may hold a special place in visual motion processing.

Visual response properties of neurons in the parahippocampal cortex of monkeys

We examined visual response properties of single neurons in the parahippocampal (PH) cortex of alert monkeys using various visual stimuli (bars, geometrical shapes such as a circle, and images such as a human face) while the monkey fixated a spot for a juice reward. Of the investigated PH neurons 104 of 359 (29%) were found to be visually responsive. The investigation was focused on spatial and object aspects of visual processing. We investigated a visual receptive field (RF) property and a direction selectivity for a moving bar with respect to spatial processing. For half of these PH neurons (53%), the optimal stimulus position, where a visual stimulus elicited the maximal response, located peripherally, that is, with an eccentricity of more than 10 deg. More than 20% of these PH neurons had an RF that does not include the center of gaze. There were neurons in the PH cortex that appeared to convey motion signals. In addition, some PH neurons showed eye-position-dependent activity. With respect to object processing, we investigated selectivities for images, geographical shapes, orientations of a bar, and colors. For comparison purposes, we also examined responses of perirhinal (PR) neurons. PH neurons showed selective responses to these stimuli, but PR neurons were found to be more selective for images than PH neurons. These results suggest that the PH cortex is involved in both spatial and object processing, but less involved than the PR cortex in processing of complex images.

Differential ambiguity reduces grouping of metastable objects

Two coaxial, ambiguously rotating objects tend to be perceived as corotating. Such grouping could be the consequence of bottom-up, cooperative interactions between the stimuli, or the top-down selection of object properties consistent with a model of the objects or scene. However, we find that the coupling between an ambiguous and unambiguous object is sharply reduced, presenting a challenge for both explanations of grouping. We describe experiments that support the idea that top-down feedback is necessary to select and stabilize a perceptual interpretation for ambiguous figures. Reduced coupling between objects of differing ambiguity can be explained if the feedback is global and proportional to ambiguity.

Pursuit speed compensation in cortical area MSTd

When we move forward the visual images on our retinas expand. Humans rely on the focus, or center, of this expansion to estimate their direction of self-motion or heading and, as long as the eyes are still, the retinal focus corresponds to the heading. However, smooth pursuit eye movements add visual motion to the expanding retinal image and displace the focus of expansion. In spite of this, humans accurately judge their heading during pursuit eye movements even though the retinal focus no longer corresponds to the heading. Recent studies in macaque suggest that correction for pursuit may occur in the dorsal aspect of the medial superior temporal area (MSTd); neurons in this area are tuned to the retinal position of the focus and they modify their tuning to partially compensate for the focus shift caused by pursuit. However, the question remains whether these neurons shift focus tuning more at faster pursuit speeds, to compensate for the larger focus shifts created by faster pursuit. To investigate this question, we recorded from 40 MSTd neurons while monkeys made pursuit eye movements at a range of speeds across simulated self- or object motion displays. We found that most MSTd neurons modify their focus tuning more at faster pursuit speeds, consistent with the idea that they encode heading and other motion parameters regardless of pursuit speed. Across the population, the median rate of compensation increase with pursuit speed was 51% as great as required for perfect compensation. We recorded from the same neurons in a simulated pursuit condition, in which gaze was fixed but the entire display counter-rotated to produce the same retinal image as during real pursuit. This condition yielded the result that retinal cues contribute to pursuit compensation; the rate of compensation increase was 30% of that required for accurate encoding of heading. The difference between these two conditions was significant (P < 0.05), indicating that extraretinal cues also contribute significantly. We found a systematic antialignment between preferred pursuit and preferred visual motion directions. Neurons may use this antialignment to combine retinal and extraretinal compensatory cues. These results indicate that many MSTd neurons compensate for pursuit velocity, pursuit direction as previously reported and pursuit speed, and further implicate MSTd as a critical stage in the computation of egomotion.

Visual-vestibular interactive responses in the macaque ventral intraparietal area (VIP)

Self-motion detection requires the interaction of a number of sensory systems for correct perceptual interpretation of a given movement and an eventual motor response. Parietal cortical areas are thought to play an important role in this function, and we have thus studied the encoding of multimodal signals and their spatiotemporal interactions in the ventral intraparietal area of macaque monkeys. Thereby, we have identified for the first time the presence of vestibular sensory input to this area and described its interaction with somatosensory and visual signals, via extracellular single-cell recordings in awake head-fixed animals. Visual responses were driven by large field stimuli that simulated either backward or forward self-motion (contraction or expansion stimuli, respectively), or movement in the frontoparallel plane (visual increments moving simultaneously in the same direction). While the dominant sensory modality in most neurons was visual, about one third of all recorded neurons responded to horizontal rotation. These vestibular responses were typically in phase with head velocity, but in some cases they could signal acceleration or even showed integration to position. The associated visual responses were always codirectional with the vestibular on-direction, i.e. noncomplementary. Somatosensory responses were in register with the visual preferred direction, either in the same or in the opposite direction, thus signalling translation or rotation in the horizontal plane. These results, taken together with data on responses to optic flow stimuli obtained in a parallel study, strongly suggest an involvement of area VIP in the analysis and the encoding of self-motion.

Response of MSTd neurons to simulated 3D orientation of rotating planes

We studied whether the dorsal division of the medial superior temporal area (MSTd) in the macaque has activity related to structure-from-motion (SFM) processing. As the simplest form of three-dimensional (3D) structure, we chose a planar stimulus and examined the relation between the neural responses and the simulated 3D orientation of the plane defined by motion cues. We recorded from 114 MSTd neurons while monkeys were performing a visual fixation task. These neurons responded to a basic set of optic flow patterns such as translation, expansion/contraction, and rotation. Responses of these neurons to rotating plane stimuli were examined to see whether the MSTd neurons exhibited selectivity to the tilt and slant that characterize the 3D orientation of the plane. We found that most MSTd neurons tested (97 of 114) responded to the plane stimuli, and many neurons (65 of 97) exhibited selectivity to tilt and/or slant. Of 97 neurons, 18% (17/97) were selective only to tilt, 24% (23/97) only to slant, and 26% (25/97) to both. Control experiments rejected the possibility that the selectivity could be explained solely by the sensitivity to local stimulus components such as local translation, local speed, and local speed gradients. These results suggest that MSTd neurons are sensitive to stimulus features specific to the perceived 3D orientation of the rotating plane stimuli and suggest that area MSTd is involved in SFM processing.

Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP

In this study, we mainly investigated the visual selectivity of hand-manipulation-related neurons in the anterior intraparietal area (area AIP) while the animal was grasping or fixating on three-dimensional (3D) objects of different geometric shapes, sizes, and orientations. We studied the activity of 132 task-related neurons during the hand-manipulation tasks in the light and in the dark, as well as during object fixation. Seventy-seven percent (101/132) of the hand-manipulation-related neurons were visually responsive, showing either lesser activity during manipulation in the dark than during that in the light (visual-motor neurons) or no activation in the dark (visual-dominant neurons). Of these visually responsive neurons, more than half (n = 66) responded during the object-fixation task (object-type). Among these, 55 were tested for their shape selectivity during the object-fixation task, and many (n = 25) were highly selective, preferring one particular shape of the six different shapes presented (ring, cube, cylinder, cone, sphere, and square plate). For 28 moderately selective object-type neurons, we performed multidimensional scaling (MDS) to examine how the neurons encode the similarity of objects. The results suggest that some moderately selective neurons responded preferentially to common geometric features shared by similar objects (flat, round, elongated, etc.). Moderately selective nonobject-type visually responsive neurons, which did not respond during object fixation, were found by MDS to be more closely related to the handgrip than to the object shape. We found a similar selectivity for handgrip in motor-dominant neurons that did not show any visual response. With regard to the size of the objects, 16 of 26 object-type neurons tested were selective for both size and shape, whereas 9 object-type neurons were selective for shape but not for size. Seven of 12 nonobject-type and all (8/8) of the motor-dominant neurons examined were selective for size, and almost all of them were also selective for objects. Many hand-manipulation-related neurons that preferred the plate and/or ring were selective for the orientation of the objects (17/20). These results suggest that the visual responses of object-type neurons represent the shape, size, and/or orientation of 3D objects, whereas those of the nonobject-type neurons probably represent the shape of the handgrip, grip size, or hand-orientation. The activity of motor-dominant neurons was also, in part, likely to represent these parameters of hand movement. This suggests that the dorsal visual pathway is concerned with the aspect of form, orientation, and/or size perception that is relevant for the visual control of movements.

Selective right parietal lobe activation during mental rotation: a parametric PET study

Regional cerebral blood flow (rCBF) was measured with PET in seven healthy subjects while they carried out a mental rotation task in which they decided whether alphanumeric characters presented in different orientations were in their canonical form or mirror-reversed. Consistent with previous findings, subjects took proportionally longer to respond as characters were rotated further from the upright, indicating that they were mentally rotating the characters to the upright position before making a decision. We used a parametric design in which we varied the mental rotation demands in an incremental fashion while keeping all other aspects of the task constant. In four different scanning conditions, 10, 40, 70 or 100% of the stimuli presented during the scan required mental rotation while the rest were upright. The statistical parametric mapping technique was used to identify areas where changes in rCBF were correlated with the rotational demands of the task. Significant activation was found in only one area located in the right posterior parietal lobe, centred on the intraparietal sulcus (Brodmann area 7). The experimental literature on monkeys and humans suggests that this area is involved in a variety of spatial transformations. Our results contribute evidence that such transformations are recruited during mental rotation and add to a body of evidence which suggests that the right posterior parietal lobe is important for carrying out visuospatial transformations.

Influence of gaze rotation on the visual response of primate MSTd neurons

When we move forward, the visual image on our retina expands. Humans rely on the focus, or center, of this expansion to estimate their direction of heading and, as long as the eyes are still, the retinal focus corresponds to the heading. However, smooth rotation of the eyes adds nearly uniform visual motion to the expanding retinal image and causes a displacement of the retinal focus. In spite of this, humans accurately judge their heading during pursuit eye movements and during active, smooth head rotations even though the retinal focus no longer corresponds to the heading. Recent studies in macaque suggest that correction for pursuit may occur in the dorsal aspect of the medial superior temporal area (MSTd) because these neurons are tuned to the retinal position of the focus and they modify their tuning during pursuit to compensate partially for the focus shift. However, the question remains whether these neurons also shift focus tuning to compensate for smooth head rotations that commonly occur during gaze tracking. To investigate this question, we recorded from 80 MSTd neurons while monkeys tracked a visual target either by pursuing with their eyes or by vestibulo-ocular reflex cancellation (VORC; whole-body rotation with eyes fixed in head and head fixed on body). VORC is a passive, smooth head rotation condition that selectively activates the vestibular canals. We found that neurons shift their focus tuning in a similar way whether focus displacement is caused by pursuit or by VORC. Across the population, compensation averaged 88 and 77% during pursuit and VORC, respectively (tuning shift divided by the retinal focus to true heading difference). Moreover the degree of compensation during pursuit and VORC was correlated in individual cells (P < 0.001). Finally neurons that did not compensate appreciably tended to be gain-modulated during pursuit and VORC and may constitute an intermediate stage in the compensation process. These results indicate that many MSTd cells compensate for general gaze rotation, whether produced by eye-in-head or head-in-world rotation, and further implicate MSTd as a critical stage in the computation of heading. Interestingly vestibular cues present during VORC allow many cells to compensate even though humans do not accurately judge their heading in this condition. This suggests that MSTd may use vestibular information to create a compensated heading representation within at least a subpopulation of cells, which is accessed perceptually only when additional cues related to active head rotations are also present.

The contributions of vestibular signals to the representations of space in the posterior parietal cortex

Vestibular signals play an important role in spatial orientation, perception of object location, and control of self-motion. Prior physiological research on vestibular information processing has focused on brainstem mechanisms; relatively little is known about the processing of vestibular information at the level of the cerebral cortex. Recent electrophysiological experiments examining the use of vestibular canal signals in two different perceptual tasks are described: computation of self motion and localization of visual stimuli in a world-centered reference frame. These two perceptual functions are mediated by different parts of the posterior parietal cortex, the former in the dorsal aspect of the medial superior temporal area (MSTd) and the latter in area 7a.

A theory of geometric constraints on neural activity for natural three-dimensional movement

Although the orientation of an arm in space or the static view of an object may be represented by a population of neurons in complex ways, how these variables change with movement often follows simple linear rules, reflecting the underlying geometric constraints in the physical world. A theoretical analysis is presented for how such constraints affect the average firing rates of sensory and motor neurons during natural movements with low degrees of freedom, such as a limb movement and rigid object motion. When applied to nonrigid reaching arm movements, the linear theory accounts for cosine directional tuning with linear speed modulation, predicts a curl-free spatial distribution of preferred directions, and also explains why the instantaneous motion of the hand can be recovered from the neural population activity. For three-dimensional motion of a rigid object, the theory predicts that, to a first approximation, the response of a sensory neuron should have a preferred translational direction and a preferred rotation axis in space, both with cosine tuning functions modulated multiplicatively by speed and angular speed, respectively. Some known tuning properties of motion-sensitive neurons follow as special cases. Acceleration tuning and nonlinear speed modulation are considered in an extension of the linear theory. This general approach provides a principled method to derive mechanism-insensitive neuronal properties by exploiting the inherently low dimensionality of natural movements.

From visual affordances in monkey parietal cortex to hippocampo-parietal interactions underlying rat navigation

This paper explores the hypothesis that various subregions (but by no means all) of the posterior parietal cortex are specialized to process visual information to extract a variety of affordances for behaviour. Two biologically based models of regions of the posterior parietal cortex of the monkey are introduced. The model of the lateral intraparietal area (LIP) emphasizes its roles in dynamic remapping of the representation of targets during a double saccade task, and in combining stored, updated input with current visual input. The model of the anterior intraparietal area (AIP) addresses parietal-premotor interactions involved in grasping, and analyses the interaction between the AIP and premotor area F5. The model represents the role of other intraparietal areas working in concert with the inferotemporal cortex as well as with corollary discharge from F5 to provide and augment the affordance information in the AIP, and suggests how various constraints may resolve the action opportunities provided by multiple affordances. Finally, a systems-level model of hippocampo parietal interactions underlying rat navigation is developed, motivated by the monkey data used in developing the above two models as well as by data on neurones in the posterior parietal cortex of the monkey that are sensitive to visual motion. The formal similarity between dynamic remapping (primate saccades) and path integration (rat navigation) is noted, and certain available data on rat posterior parietal cortex in terms of affordances for locomotion are explained. The utility of further modelling, linking the World Graph model of cognitive maps for motivated behaviour with hippocampal-parietal interactions involved in navigation, is also suggested. These models demonstrate that posterior parietal cortex is not only itself a network of interacting subsystems, but functions through cooperative computation with many other brain regions.

The TINS Lecture. The parietal association cortex in depth perception and visual control of hand action

Recent neurophysiological studies in alert monkeys have revealed that the parietal association cortex plays a crucial role in depth perception and visually guided hand movement. The following five classes of parietal neurons covering various aspects of these functions have been identified: (1) depth-selective visual-fixation (VF) neurons of the inferior parietal lobule (IPL), representing egocentric distance; (2) depth-movement sensitive (DMS) neurons of V5A and the ventral intraparietal (VIP) area representing direction of linear movement in 3-D space; (3) depth-rotation-sensitive (RS) neurons of V5A and the posterior parietal (PP) area representing direction of rotary movement in space; (4) visually responsive manipulation-related neurons (visual-dominant or visual-and-motor type) of the anterior intraparietal (AIP) area, representing 3-D shape or orientation (or both) of objects for manipulation; and (5) axis-orientation-selective (AOS) and surface-orientation-selective (SOS) neurons in the caudal intraparietal sulcus (cIPS) sensitive to binocular disparity and representing the 3-D orientation of the longitudinal axes and flat surfaces, respectively. Some AOS and SOS neurons are selective in both orientation and shape. Thus the dorsal visual pathway is divided into at least two subsystems, V5A, PP and VIP areas for motion vision and V6, LIP and cIPS areas for coding position and 3-D features. The cIPS sends the signals of 3-D features of objects to the AIP area, which is reciprocally connected to the ventral premotor (F5) area and plays an essential role in matching hand orientation and shaping with 3-D objects for manipulation.

Functional activation of the human brain during mental rotation

Regional cerebral blood flow was measured with positron emission tomography during the performance of tasks that required cognitive spatial transformations of alphanumeric stimuli. In the mirror image task, the subjects were required to discriminate between the normal and the mirror images of alphanumeric stimuli presented in the upright orientation. In the mental rotation task, the same judgement was required, but now the stimuli were presented in various orientations other than the upright one. The subjects therefore had to rotate the stimuli, in mind, into the upright position before making their decision. In relation to the control task, which involved discrimination of these same stimuli but not any form of mental transformation, there were significant increases in the right postero-superior parietal cortex and the left inferior parietal cortex in both experimental tasks. For mental rotation, specific activity was seen only within the left inferior parietal region and the right head of the caudate nucleus. These results specified the parietal areas involved in a purely cognitive spatial process and demonstrated a close interaction between these areas and the anterior neostriatum and certain lateral frontal cortical areas in the discrimination of rotated forms of stimuli.

Specific involvement of human parietal systems and the amygdala in the perception of biological motion

To explore the extent to which functional systems within the human posterior parietal cortex and the superior temporal sulcus are involved in the perception of action, we measured cerebral metabolic activity in human subjects by positron emission tomography during the perception of simulations of biological motion with point-light displays. The experimental design involved comparisons of activity during the perception of goal-directed hand action, whole body motion, object motion, and random motion. The results demonstrated that the perception of scripts of goal-directed hand action implicates the cortex in the intraparietal sulcus and the caudal part of the superior temporal sulcus, both in the left hemisphere. By contrast, the rostrocaudal part of the right superior temporal sulcus and adjacent temporal cortex, and limbic structures such as the amygdala, are involved in the perception of signs conveyed by expressive body movements.

Intercepting real and path-guided apparent motion targets

Human subjects were instructed to intercept with a cursor real and apparent motion targets presented on a computer screen. Targets traveled counterclockwise (CCW) in a circle at one of five angular velocities (180, 300, 420, 480 and 540 deg/s), either smoothly (real motion) or in path-guided apparent motion. Subjects operated a computer mouse and were instructed to intercept targets at the 12 o'clock position; there were no constraints on when to initiate the response, which was a movement from the center of the screen towards and past 12 o'clock. We found the following: (a) for both motion conditions and all target velocities, subjects were late in intercepting the target, especially at higher target velocities; (b) for both motion conditions, the directional variability of the response increased as a linear function of the target velocity; (c) the directional variability of the response was systematically higher for the apparent than the real motion condition; there was no significant interaction between target velocity and target motion type; (d) the response time did not vary significantly with velocity, but was consistently longer for apparent than real motion targets; (e) the movement time was very similar for different target velocities; and (f) the moment of initiation of the interception movement was delayed appreciably at higher target velocities, relative to that dictated for perfect interception at a given target velocity. This delay was greater for the apparent motion target. These results demonstrated the following: (a) for both target motion conditions, interception was not fully predictive but lagged the target in spite of the constant target velocity and the unconstrained time allowed for initiating the interception movement; (b) subjects can intercept an apparent motion target but, compared with real motion, the performance is somewhat degraded overall; (c) the similarities in performance between the two target motion conditions, and the fact that target velocity influenced performance in a similar fashion, suggest that the motor system can access the visual information provided by the moving target; and (d) since movement time was similar for different target velocities, the strategy for interception relied on controlling the moment of initiation of the interception movement. This was successful for low target velocities but became unsuccessful at higher target velocities.

Is there an effect of weightlessness on mental rotation of three-dimensional objects?

We studied the performance of eight cosmonauts in a mental rotation paradigm with simultaneously presented perspective views of three-dimensional objects. The cosmonauts were tested successively on earth, in microgravity aboard the Russian MIR station and again on earth. Their performance was compared to performance of a control group of five subjects tested on earth on the same dates. We particularly tried to disambiguate the effect of microgravity, procedural bias and practice. Our results show that the microgravity did not alter the mental rotation process. The performance of cosmonauts increased with practice, similarly to the performance of control group's subjects suggesting that the weightlessness did not impair implicit learning as well. Finally, we propose an explanation of previous contradictory results.

Influence of graviceptives cues at different level of visual information processing: the effect of prolonged weightlessness

We evaluated the influence of prolonged weightlessness on the performance of visual tasks in the course of the Russian-French missions ANTARES, Post-ANTARES and ALTAIR aboard the MIR station. Eight cosmonauts were subjects in two experiments executed pre-flight, in-flight and post-flight sessions. In the first experiment, cosmonauts performed a task of symmetry detection in 2-D polygons. The results indicate that this detection is locked in a head retinal reference frame rather than in an environmentally defined one as meridional orientations of symmetry axis (vertical and horizontal) elicited faster response times than oblique ones. However, in weightlessness the saliency of a retinally vertical axis of symmetry is no longer significantly different from an horizontal axis. In the second experiment, cosmonauts performed a mental rotation task in which they judged whether two 3-D objects presented in different orientations were identical. Performance on this task is basically identical in weightlessness and normal gravity.