Surface orientation discrimination activates caudal and anterior intraparietal sulcus in humans: an event-related fMRI study

The neuroscience of vision-based grasping: a functional review for computational modeling and bio-inspired robotics

The topic of vision-based grasping is being widely studied in humans and in other primates using various techniques and with different goals. The fundamental related findings are reviewed in this paper, with the aim of providing researchers from different fields, including intelligent robotics and neural computation, a comprehensive but accessible view on the subject. A detailed description of the principal sensorimotor processes and the brain areas involved is provided following a functional perspective, in order to make this survey especially useful for computational modeling and bio-inspired robotic applications.

3D surface perception from motion involves a temporal-parietal network

Previous research has suggested that three-dimensional (3D) structure-from-motion (SFM) perception in humans involves several motion-sensitive occipital and parietal brain areas. By contrast, SFM perception in nonhuman primates seems to involve the temporal lobe including areas MT, MST and FST. The present functional magnetic resonance imaging study compared several motion-sensitive regions of interest including the superior temporal sulcus (STS) while human observers viewed horizontally moving dots that defined either a 3D corrugated surface or a 3D random volume. Low-level stimulus features such as dot density and velocity vectors as well as attention were tightly controlled. Consistent with previous research we found that 3D corrugated surfaces elicited stronger responses than random motion in occipital and parietal brain areas including area V3A, the ventral and dorsal intraparietal sulcus, the lateral occipital sulcus and the fusiform gyrus. Additionally, 3D corrugated surfaces elicited stronger activity in area MT and the STS but not in area MST. Brain activity in the STS but not in area MT correlated with interindividual differences in 3D surface perception. Our findings suggest that area MT is involved in the analysis of optic flow patterns such as speed gradients and that the STS in humans plays a greater role in the analysis of 3D SFM than previously thought.

TMS activation of interhemispheric pathways between the posterior parietal cortex and the contralateral motor cortex

Using a twin coil transcranial magnetic stimulation (tc-TMS) approach we have previously demonstrated that facilitation may be detected in the primary motor cortex (M1) following stimulation over the ipsilateral caudal intraparietal sulcus (cIPS). Here we tested the interhemispheric interactions between the IPS and the contralateral motor cortex (M1). We found that conditioning the right cIPS facilitated contralateral M1 when the conditioning stimulus had an intensity of 90% resting motor threshold (RMT) but not at 70% or 110% RMT. Facilitation was maximal when the interstimulus interval (ISI) between cIPS and M1 was 6 or 12 ms. These facilitatory effects were mediated by interactions with specific groups of interneurons in the contralateral M1. In fact, short intracortical inhibition (SICI) was reduced following cIPS stimulation. Moreover, additional comparison of facilitation of responses evoked by anterior-posterior versus posterior-anterior stimulation of M1 suggested that facilitation was more effective on early I1/I2 circuits than on I3 circuits. In contrast to these effects, stimulation of anterior IPS (aIPS) at 90% RMT induced inhibition, instead of facilitation, of contralateral M1 at ISIs of 10-12 ms. Finally, we found similar facilitation between left cIPS and right M1 although the conditioning stimuli had to have a higher intensity compared with stimulation of right cIPS (110% instead of 90% RMT). These findings demonstrate that different subregions of the posterior parietal cortex (PPC) in humans exert both facilitatory and inhibitory effects towards the contralateral primary motor cortex. These corticocortical projections could contribute to a variety of motor tasks such as bilateral manual coordination, movement planning in space and grasping.

Activation of sensory-motor areas in sentence comprehension

The sensory-motor account of conceptual processing suggests that modality-specific attributes play a central role in the organization of object and action knowledge in the brain. An opposing view emphasizes the abstract, amodal, and symbolic character of concepts, which are thought to be represented outside the brain's sensory-motor systems. We conducted a functional magnetic resonance imaging study in which the participants listened to sentences describing hand/arm action events, visual events, or abstract behaviors. In comparison to visual and abstract sentences, areas associated with planning and control of hand movements, motion perception, and vision were activated when understanding sentences describing actions. Sensory-motor areas were activated to a greater extent also for sentences with actions that relied mostly on hands, as opposed to arms. Visual sentences activated a small area in the secondary visual cortex, whereas abstract sentences activated superior temporal and inferior frontal regions. The results support the view that linguistic understanding of actions partly involves imagery or simulation of actions, and relies on some of the same neural substrate used for planning, performing, and perceiving actions.

Dynamic changes in the mental rotation network revealed by pattern recognition analysis of fMRI data

We investigated the temporal dynamics and changes in connectivity in the mental rotation network through the application of spatio-temporal support vector machines (SVMs). The spatio-temporal SVM [Mourao-Miranda, J., Friston, K. J., et al. (2007). Dynamic discrimination analysis: A spatial-temporal SVM. Neuroimage, 36, 88-99] is a pattern recognition approach that is suitable for investigating dynamic changes in the brain network during a complex mental task. It does not require a model describing each component of the task and the precise shape of the BOLD impulse response. By defining a time window including a cognitive event, one can use spatio-temporal fMRI observations from two cognitive states to train the SVM. During the training, the SVM finds the discriminating pattern between the two states and produces a discriminating weight vector encompassing both voxels and time (i.e., spatio-temporal maps). We showed that by applying spatio-temporal SVM to an event-related mental rotation experiment, it is possible to discriminate between different degrees of angular disparity (0 degrees vs. 20 degrees , 0 degrees vs. 60 degrees , and 0 degrees vs. 100 degrees ), and the discrimination accuracy is correlated with the difference in angular disparity between the conditions. For the comparison with highest accuracy (0 degrees vs. 100 degrees ), we evaluated how the most discriminating areas (visual regions, parietal regions, supplementary, and premotor areas) change their behavior over time. The frontal premotor regions became highly discriminating earlier than the superior parietal cortex. There seems to be a parcellation of the parietal regions with an earlier discrimination of the inferior parietal lobe in the mental rotation in relation to the superior parietal. The SVM also identified a network of regions that had a decrease in BOLD responses during the 100 degrees condition in relation to the 0 degrees condition (posterior cingulate, frontal, and superior temporal gyrus). This network was also highly discriminating between the two conditions. In addition, we investigated changes in functional connectivity between the most discriminating areas identified by the spatio-temporal SVM. We observed an increase in functional connectivity between almost all areas activated during the 100 degrees condition (bilateral inferior and superior parietal lobe, bilateral premotor area, and SMA) but not between the areas that showed a decrease in BOLD response during the 100 degrees condition.

Separate processing of texture and form in the ventral stream: evidence from FMRI and visual agnosia

Real-life visual object recognition requires the processing of more than just geometric (shape, size, and orientation) properties. Surface properties such as color and texture are equally important, particularly for providing information about the material properties of objects. Recent neuroimaging research suggests that geometric and surface properties are dealt with separately within the lateral occipital cortex (LOC) and the collateral sulcus (CoS), respectively. Here we compared objects that differed either in aspect ratio or in surface texture only, keeping all other visual properties constant. Results on brain-intact participants confirmed that surface texture activates an area in the posterior CoS, quite distinct from the area activated by shape within LOC. We also tested 2 patients with visual object agnosia, one of whom (DF) performed well on the texture task but at chance on the shape task, whereas the other (MS) showed the converse pattern. This behavioral double dissociation was matched by a parallel neuroimaging dissociation, with activation in CoS but not LOC in patient DF and activation in LOC but not CoS in patient MS. These data provide presumptive evidence that the areas respectively activated by shape and texture play a causally necessary role in the perceptual discrimination of these features.

Dorsal-ventral integration in the recognition of motion-defined unfamiliar faces

The primate visual system is organized into two parallel anatomical pathways, both originating in early visual areas but terminating in posterior parietal or inferior temporal regions. Classically, these two pathways have been thought to subserve spatial vision and visual guided actions (dorsal pathway) and object identification (ventral pathway). However, evidence is accumulating that dorsal visual areas may also represent many aspects of object shape in absence of demands for attention or action. Dorsal visual areas exhibit selectivity for three-dimensional cues of depth and are considered necessary for the extraction of surfaces from depth cues and can carry out cognitive functions with such cues as well. These results suggest that dorsal visual areas may participate in object recognition, but it is unclear to what capacity. Here, we tested whether three-dimensional structure-from-motion (SFM) cues, thought to be computed exclusively by dorsal stream mechanisms, are sufficient to drive complex object recognition. We then tested whether recognition of such stimuli relies on dorsal stream mechanisms alone, or whether dorsal-ventral integration is invoked. Results suggest that such cues are sufficient to drive unfamiliar face recognition in normal participants and that ventral stream areas are necessary for both identification and learning of unfamiliar faces from SFM cues.

Perceptual incongruence influences bistability and cortical activation

We employed a parametric psychophysical design in combination with functional imaging to examine the influence of metric changes in perceptual incongruence on perceptual alternation rates and cortical responses. Subjects viewed a bistable stimulus defined by incongruent depth cues; bistability resulted from incongruence between binocular disparity and monocular perspective cues that specify different slants (slant rivalry). Psychophysical results revealed that perceptual alternation rates were positively correlated with the degree of perceived incongruence. Functional imaging revealed systematic increases in activity that paralleled the psychophysical results within anterior intraparietal sulcus, prior to the onset of perceptual alternations. We suggest that this cortical activity predicts the frequency of subsequent alternations, implying a putative causal role for these areas in initiating bistable perception. In contrast, areas implicated in form and depth processing (LOC and V3A) were sensitive to the degree of slant, but failed to show increases in activity when these cues were in conflict.

Three-dimensional shape from second-order orientation flows

In images of textured surfaces, orientation flows formed by perspective convergence invariably convey 3D shape. We show that orientation flows formed by contrast-modulated (CM) and illusory contours (IC) convey 3D shape, and that both stimulus types induce 3D shape aftereffects on CM and IC test stimuli. Adaptation to luminance-modulated (LM) orientation flows induce robust 3D shape aftereffects on CM and IC tests, however, aftereffects using CM/IC adapting stimuli on LM tests were substantially weaker. These results can be explained by the adaptation of 3D shape-selective neurons that invariantly extract first- and second-order orientation flows from striate and extra-striate signals, which receive stronger input from neurons selective for first-order orientation flows.

Common cortical loci are activated during visuospatial interpolation and orientation discrimination judgements

There is a wealth of literature on the role of short-range interactions between low-level orientation-tuned filters in the perception of discontinuous contours. However, little is known about how spatial information is integrated across more distant regions of the visual field in the absence of explicit local orientation cues, a process referred to here as visuospatial interpolation (VSI). To examine the neural correlates of VSI high field functional magnetic resonance imaging was used to study brain activity while observers either judged the alignment of three Gabor patches by a process of interpolation or discriminated the local orientation of the individual patches. Relative to a fixation baseline the two tasks activated a largely over-lapping network of regions within the occipito-temporal, occipito-parietal and frontal cortices. Activated clusters specific to the orientation task (orientation>interpolation) included the caudal intraparietal sulcus, an area whose role in orientation encoding per se has been hotly disputed. Surprisingly, there were few task-specific activations associated with visuospatial interpolation (VSI>orientation) suggesting that largely common cortical loci were activated by the two experimental tasks. These data are consistent with previous studies that suggest higher level grouping processes -putatively involved in VSI- are automatically engaged when the spatial properties of a stimulus (e.g. size, orientation or relative position) are used to make a judgement.

Remembrance of things touched: how sensorimotor experience affects the neural instantiation of object form

Numerous neuroimaging and neuropsychological studies have highlighted the role of the ventral, occipitotemporal visual processing stream in the representation and retrieval of semantic memory for the appearance of objects. Here, we examine the role of the parietal cortex in retrieval of object shape information. fMRI data were acquired as subjects listened to the names of common objects and made judgments about their shape. Recruitment of the left inferior parietal lobule (IPL) during shape retrieval was modulated by the amount of prior tactile experience associated with the objects. In addition, the same pattern of activity was observed in right postcentral gyrus, suggesting that the representation of object shape is distributed amongst regions that are relevant to the sensorimotor acquisition history of this attribute, as predicted by distributed models of semantic memory.

Localization of human intraparietal areas AIP, CIP, and LIP using surface orientation and saccadic eye movement tasks

In monkeys, areas in the intraparietal sulcus (IPS) play a crucial role in visuospatial information processing. Despite many human neuroimaging studies, the location of the human functional homologs of some IPS areas is still a matter of debate. The aim of the present functional magnetic resonance imaging (fMRI) study was to identify the distinct locations of specific human IPS areas based on their functional properties using stimuli adapted from nonhuman primate experiments, in particular, surface orientation discrimination and memory guided saccadic eye movements (SEM). Intersubject anatomical variability likely accounts for much of the debate. By applying subject by subject analysis, we can demonstrate that sufficient intersubject anatomical and functional commonalities exist. Both the lateral bank of the anterior part of IPS, the putative human homolog of the area AIP, and the caudal part of the IPS (putative CIP) showed activation related to spatial discrimination of surface orientation. Eye tracking conducted during fMRI data acquisition allowed us to show that both areas were separated by an area related to SEM. This area was located in the middle region of the IPS (most probably including LIP), i.e., similar to the location observed in nonhuman primates. In 10 of 11 subjects our putative CIP activation was located in a medial side branch of the posterior part of the IPS, on the opposite side as described in nonhuman primates, making this landmark a useful anatomical marker for the location of CIP.

Effects of right parietal transcranial magnetic stimulation on object identification and orientation judgments

We investigated the role played by the right parietal lobe in object identification and the ability to interpret object orientation, using transcranial magnetic stimulation (TMS) to momentarily interfere with ongoing cortical activity. Short trains of TMS pulses (12 Hz) were applied to a site overlying the right intraparietal sulcus/inferior parietal lobe while subjects performed either object identification tasks (i.e., picture-word verification and categorizing objects as natural or manufactured) or object orientation judgment tasks (i.e., picture-arrow verification and deciding whether an object was rotated clockwise or counterclockwise). Across different tasks, right parietal TMS impaired orientation judgments, but facilitated object identification, compared to TMS applied to a brain vertex control site. These complementary findings demonstrate that the right parietal lobe--a region belonging to the dorsal visual stream--is critical for processing the spatial attributes of objects, but not their identity. The observed improvement in object recognition, however, suggests an indirect role for the right parietal lobe in object recognition. We propose that this involves the creation of a spatial reference frame for the object, which allows interaction with the object and the individuation of specific viewing instances.

The cerebellum predicts the timing of perceptual events

Prospective (forward) temporal-spatial models are essential for both action and perception, but the literature on perceptual prediction has primarily been limited to the spatial domain. In this study we asked how the neural systems of perceptual prediction change, when change-over-time must be modeled. We used a naturalistic paradigm in which observers had to extrapolate the trajectory of an occluded moving object to make perceptual judgments based on the spatial (direction) or temporal-spatial (velocity) characteristics of object motion. Using functional magnetic resonance imaging we found that a region in posterior cerebellum (lobule VII crus 1) was engaged specifically when a temporal-spatial model was required (velocity judgment task), suggesting that circuitry involved in motor forward-modeling may also be engaged in perceptual prediction when a model of change-over-time is required. This cerebellar region appears to supply a temporal signal to cortical networks involved in spatial orienting: a frontal-parietal network associated with attentional orienting was engaged in both (spatial and temporal-spatial) tasks, but functional connectivity between these regions and the posterior cerebellum was enhanced in the temporal-spatial prediction task. In addition to the oculomotor spatial orienting network, regions involved in hand movements (aIP and PMv) were recruited in the temporal-spatial task, suggesting that the nature of perceptual prediction may bias the recruitment of sensory-motor networks in orienting. Finally, in temporal-spatial prediction, functional connectivity was enhanced between the cerebellum and the putamen, a structure which has been proposed to supply the brain's metric of time, in the temporal-spatial prediction task.

The extraction of 3D shape from texture and shading in the human brain

We used functional magnetic resonance imaging to investigate the human cortical areas involved in processing 3-dimensional (3D) shape from texture (SfT) and shading. The stimuli included monocular images of randomly shaped 3D surfaces and a wide variety of 2-dimensional (2D) controls. The results of both passive and active experiments reveal that the extraction of 3D SfT involves the bilateral caudal inferior temporal gyrus (caudal ITG), lateral occipital sulcus (LOS) and several bilateral sites along the intraparietal sulcus. These areas are largely consistent with those involved in the processing of 3D shape from motion and stereo. The experiments also demonstrate, however, that the analysis of 3D shape from shading is primarily restricted to the caudal ITG areas. Additional results from psychophysical experiments reveal that this difference in neuronal substrate cannot be explained by a difference in strength between the 2 cues. These results underscore the importance of the posterior part of the lateral occipital complex for the extraction of visual 3D shape information from all depth cues, and they suggest strongly that the importance of shading is diminished relative to other cues for the analysis of 3D shape in parietal regions.

Parietal mapping of visuomotor transformations during human tool grasping

During daily life, we reach and grasp objects located in a variety of positions in our visual-field. Where is the information regarding the visual (position) and motor (acting-hand) aspects integrated in the brain? To address this question, a functional magnetic resonance imaging experiment was conducted, in which 10 right-handed subjects used their right or left hand to grasp 3-dimensional tools, located to the right or left of a central fixation point. The posterior part of the intraparietal sulcus (IPS), the putative human homolog of caudal-IPS, was found to be primarily involved in representing the visual location of the tools, whereas more anterior regions, the human homologs of medial intraparietal area and anterior intraparietal, primarily encoded the identity of the contralateral acting-hand. Quantitative analysis revealed 2 opposite visual and motor gradients along the posterior-anterior axis within the IPS: although the importance of the visual-field gradually diminished, the weight of the acting-hand became increasingly greater. Moreover, direct evidence for visuomotor interaction was found in all 3 IPS subregions, but not in occipital or frontal regions. These findings support the hypothesis that the human IPS is comprised of subregions that have different properties, and that it is engaged in visuomotor transformations necessary for visually guided prehension.

Representation of shapes, edges, and surfaces across multiple cues in the human visual cortex

The lateral occipital complex (LOC) responds preferentially to objects compared with random stimuli or textures independent of the visual cue. However, it is unknown whether the LOC (or other cortical regions) are involved in the processing of edges or global surfaces without shape information. Here, we examined processing of 1) global shape, 2) disconnected edges without a global shape, and 3) global surfaces without edges versus random stimuli across motion and stereo cues. The LOC responded more strongly to global shapes than to edges, surfaces, or random stimuli, for both motion and stereo cues. However, its responses to local edges or global surfaces were not different from random stimuli. This suggests that the LOC processes shapes, not edges or surfaces. LOC also responded more strongly to objects than to holes with the same shape, suggesting sensitivity to border ownership. V7 responded more strongly to edges than to surfaces or random stimuli for both motion and stereo cues, whereas V3a and V4 preferred motion edges. Finally, a region in the caudal intraparietal sulcus (cIPS) responded more strongly to both stereo versus motion and to stereo surfaces versus random stereo (but not to motion surfaces vs. random motion). Thus we found evidence for cue-specific responses to surfaces in the cIPS, both cue-specific and cue-independent responses to edges in intermediate visual areas, and shape-selective responses across multiple cues in the LOC. Overall, these data suggest that integration of visual information across multiple cues is mainly achieved at the level of shape and underscore LOC's role in shape computations.

Neural basis of 3-D shape aftereffects

We used selective adaptation to identify the neural mechanisms responsible for 3-D shape perception from orientation flows in retinal images [Li, A., & Zaidi, Q. (2000). Perception of three-dimensional shape from texture is based on patterns of oriented energy. Vision Research 40 (2), 217-242)]. Three-dimensional shape adaptation could involve stages from photoreceptors to non-oriented retinal cells, oriented cells in striate cortex, and extra-striate cells that respond to 3-D slants. To psychophysically isolate the relevant stage, we used 3-D adapting stimuli created from real and illusory orientations, and test stimuli different from the adapting stimuli in phases and frequencies. The results showed that mechanisms that adapt to 3-D shapes combine real and illusory 2-D orientation information over a range of spatial frequencies.

Posteromedial parietal cortical activity and inputs predict tactile spatial acuity

We used functional magnetic resonance imaging (fMRI) to investigate the neural circuitry underlying tactile spatial acuity at the human finger pad. Stimuli were linear, three-dot arrays, applied to the immobilized right index finger pad using a computer-controlled, MRI-compatible, pneumatic stimulator. Activity specific for spatial processing was isolated by contrasting discrimination of left-right offsets of the central dot in the array with discrimination of the duration of stimulation by an array without a spatial offset. This contrast revealed activity in a distributed frontoparietal cortical network, within which the levels of activity in right posteromedial parietal cortical foci [right posterior intraparietal sulcus (pIPS) and right precuneus] significantly predicted individual acuity thresholds. Connectivity patterns were assessed using both bivariate analysis of Granger causality with the right pIPS as a reference region and multivariate analysis of Granger causality for a selected set of regions. The strength of inputs into the right pIPS was significantly greater in subjects with better acuity than those with poorer acuity. In the better group, the paths predicting acuity converged from the left postcentral sulcus and right frontal eye field onto the right pIPS and were selective for the spatial task, and their weights predicted the level of right pIPS activity. We propose that the optimal strategy for fine tactile spatial discrimination involves interaction in the pIPS of a top-down control signal, possibly attentional, with somatosensory cortical inputs, reflecting either visualization of the spatial configurations of tactile stimuli or engagement of modality-independent circuits specialized for fine spatial processing.

Spatio-temporal brain activity related to rotation method during a mental rotation task of three-dimensional objects: An MEG study

During mental rotation tasks, subjects perform mental simulation to solve tasks. However, detailed neural mechanisms underlying mental rotation of three-dimensional (3D) objects, particularly, whether higher motor areas related to mental simulation are activated, remain unknown. We hypothesized that environmental monitoring-a process based on environmental information and is included in motor execution-is as a key factor affecting the utilization of higher motor areas. Therefore, using magnetoencephalography (MEG), we measured spatio-temporal brain activities during two types (two-dimensional (2D) and 3D rotation tasks) of mental rotation of 3D objects. Only the 3D rotation tasks required subjects to mentally rotate objects in a depth plane with visualization of hidden parts of the visual stimuli by acquiring and retrieving 3D information. In cases showing significant differences in the averaged activities at 100-ms intervals between the two rotations, the activities were located in the right dorsal premotor (PMd) at approximately 500 ms. In these cases, averaged activities during 3D rotation were greater than those during 2D rotation, implying that the right PMd activities are related to environmental monitoring. During 3D rotation, higher activities were observed from 200 to 300 ms in the left PMd and from 400 to 700 ms in the right PMd. It is considered that the left PMd is related to primary motor control, whereas the right PMd plays a supplementary role during mental simulation. Further, during 3D rotation, late higher activities related to mental simulation are observed in the right superior parietal lobule (SPL), which is connected to PMd.

FMRI reveals a dissociation between grasping and perceiving the size of real 3D objects

BACKGROUND

Almost 15 years after its formulation, evidence for the neuro-functional dissociation between a dorsal action stream and a ventral perception stream in the human cerebral cortex is still based largely on neuropsychological case studies. To date, there is no unequivocal evidence for separate visual computations of object features for performance of goal-directed actions versus perceptual tasks in the neurologically intact human brain. We used functional magnetic resonance imaging to test explicitly whether or not brain areas mediating size computation for grasping are distinct from those mediating size computation for perception.

METHODOLOGY/PRINCIPAL FINDINGS

Subjects were presented with the same real graspable 3D objects and were required to perform a number of different tasks: grasping, reaching, size discrimination, pattern discrimination or passive viewing. As in prior studies, the anterior intraparietal area (AIP) in the dorsal stream was more active during grasping, when object size was relevant for planning the grasp, than during reaching, when object properties were irrelevant for movement planning (grasping>reaching). Activity in AIP showed no modulation, however, when size was computed in the context of a purely perceptual task (size = pattern discrimination). Conversely, the lateral occipital (LO) cortex in the ventral stream was modulated when size was computed for perception (size>pattern discrimination) but not for action (grasping = reaching).

CONCLUSIONS/SIGNIFICANCE

While areas in both the dorsal and ventral streams responded to the simple presentation of 3D objects (passive viewing), these areas were differentially activated depending on whether the task was grasping or perceptual discrimination, respectively. The demonstration of dual coding of an object for the purposes of action on the one hand and perception on the other in the same healthy brains offers a substantial contribution to the current debate about the nature of the neural coding that takes place in the dorsal and ventral streams.

Beyond grasping: representation of action in human anterior intraparietal sulcus

The fronto-parietal network has been implicated in the processing of multisensory information for motor control. Recent methodological advances with both fMRI and TMS provide the opportunity to dissect the functionality of this extensive network in humans and may identify distinct contributions of local neural populations within this circuit that are not only related to motor planning, but to goal oriented behavior as a whole. Herein, we review and make parallels between experiments in monkeys and humans on a broad array of motor as well as non-motor tasks in order to characterize the specific contribution of a region in the parietal lobe, the anterior intraparietal sulcus (aIPS). The intent of this article is to review: (1) the historical perspectives on the parietal lobe, particularly the aIPS; (2) extend and update these perspectives based on recent empirical data; and (3) discuss the potential implications of the revised functionality of the aIPS in relationship to complex goal oriented behavior and social interaction. Our contention is that aIPS is a critical node within a network involved in the higher order dynamic control of action, including representation of intended action goals. These findings may be important not only for guiding the design of future experiments investigating related issues but may also have valuable utility in other fields, such social neuroscience and biomedical engineering.

Orientation sensitivity to graspable objects: An fMRI adaptation study

It has been proposed that vision-for-perception and vision-for-action are subserved by distinct streams of visual processing, the ventral and dorsal stream, respectively [Milner, A. D., Goodale, M. A., 1995. The visual brain in action. Oxford University Press, Oxford]. Such a distinction has been supported by a recent functional magnetic resonance (fMR) adaptation study [Valyear, K. F., Culham, J. C., Sharif, N., Westwood, D., Goodale, M. A., 2006. A double dissociation between sensitivity to changes in object identity and object orientation in the ventral and dorsal visual streams: A human fMRI study. Neuropsychologia 44, 218-228], which demonstrated selectivity to object identity but not object orientation within the ventral stream, and selectivity to object orientation but not object identity within the dorsal stream. These results were interpreted as suggesting that changes to object identity (but not to orientation) would alter the representation of the stimulus in the perceptual/recognition system, whereas changes in object orientation (but not necessarily identity) would alter the coding of the stimulus within a visuomotor system concerned with behaviour such as grasping. If orientation sensitivity in the dorsal stream does reflect such a potential for action, then this sensitivity should be specific to graspable objects. Using an fMR adaptation paradigm, we presented participants with an image of either a graspable or non-graspable stimulus, followed by the same image in either the original orientation or its mirror image. One region within the dorsal stream, the lateral occipito-parietal junction (lOPJ), was shown to be sensitive to orientation changes for graspable stimuli; this region did not show orientation sensitivity for non-graspable stimuli. Thus, it appears that the sensitivity to orientation changes in this region is specific to graspable objects, presumably because such changes affect the affordances of graspable but not non-graspable objects.

Neural correlates of disparity-defined shape discrimination in the human brain

Binocular disparity, the slight differences between the images registered by our two eyes, provides an important cue when estimating the three-dimensional (3D) structure of the complex environment we inhabit. Sensitivity to binocular disparity is evident at multiple levels of the visual hierarchy in the primate brain, from early visual cortex to parietal and temporal areas. However, the relationship between activity in these areas and key perceptual functions that exploit disparity information for 3D shape perception remains an important open question. Here we investigate the link between human cortical activity and the perception of disparity-defined shape, measuring fMRI responses concurrently with psychophysical shape judgments. We parametrically degraded the coherence of shapes by shuffling the spatial position of dots whose disparity defined the 3D structure and investigated the effect of this stimulus manipulation on both cortical activity and shape discrimination. We report significant relationships between shape coherence and fMRI response in both dorsal (V3, hMT+/V5) and ventral (LOC) visual areas that correspond to the observers' discrimination performance. In contrast to previous suggestions of a dichotomy of disparity-related processes in the ventral and dorsal streams, these findings are consistent with proposed interactions between these pathways that may mediate a continuum of processes important in perceiving 3D shape from coarse contour segmentation to fine curvature estimation.

Neural correlates of monocular and binocular depth cues based on natural images: A LORETA analysis

Functional imaging studies investigating perception of depth rely solely on one type of depth cue based on non-natural stimulus material. To overcome these limitations and to provide a more realistic and complete set of depth cues natural stereoscopic images were used in this study. Using slow cortical potentials and source localization we aimed to identify the neural correlates of monocular and binocular depth cues. This study confirms and extends functional imaging studies, showing that natural images provide a good, reliable, and more realistic alternative to artificial stimuli, and demonstrates the possibility to separate the processing of different depth cues.

Multiple brain networks for visual self-recognition with different sensitivity for motion and body part

Multiple brain networks may support visual self-recognition. It has been hypothesized that the left ventral occipito-temporal cortex processes one's own face as a symbol, and the right parieto-frontal network processes self-image in association with motion-action contingency. Using functional magnetic resonance imaging, we first tested these hypotheses based on the prediction that these networks preferentially respond to a static self-face and to moving one's whole body, respectively. Brain activation specifically related to self-image during familiarity judgment was compared across four stimulus conditions comprising a two factorial design: factor Motion contrasted picture (Picture) and movie (Movie), and factor Body part a face (Face) and whole body (Body). Second, we attempted to segregate self-specific networks using a principal component analysis (PCA), assuming an independent pattern of inter-subject variability in activation over the four stimulus conditions in each network. The bilateral ventral occipito-temporal and the right parietal and frontal cortices exhibited self-specific activation. The left ventral occipito-temporal cortex exhibited greater self-specific activation for Face than for Body, in Picture, consistent with the prediction for this region. The activation profiles of the right parietal and frontal cortices did not show preference for Movie Body predicted by the assumed roles of these regions. The PCA extracted two cortical networks, one with its peaks in the right posterior, and another in frontal cortices; their possible roles in visuo-spatial and conceptual self-representations, respectively, were suggested by previous findings. The results thus supported and provided evidence of multiple brain networks for visual self-recognition.

The predictive value of white matter organization in posterior parietal cortex for spatial visualization ability

Humans differ substantially in their ability to imagine spatial transformations of novel stimuli (i.e., mental rotation). Whereas "high-spatial" individuals are able to maintain high-quality representations even after complex mental transformations, "low-spatial" individuals often experience substantial degradation of the initial representation. Even though subdivisions of the posterior parietal cortex are known to instantiate the necessary spatial transformations, a direct demonstration of neuroanatomical differences predicting this behavioral variability is currently missing. Because recent evidence suggests that interindividual differences on the behavioral level might be related to regionally specific white matter organization, we addressed this question using diffusion tensor imaging in combination with well-established psychometric tests. As expected, behavioral results revealed a substantial disparity in mental rotation performance. Most importantly, despite controlling for differences in spatial short-term memory capacity, we observed a tight relationship between mental rotation proficiency and white matter organization near the anterior part of the intraparietal sulcus. Whereas high-level proficiency was paralleled by high fractional anisotropy (FA) values, the opposite pattern was observed in "low spatials". The present results strongly indicate that the efficiency of information transfer between posterior parietal regions involved in the mental transformation process could be one decisive factor in individual spatial visualization proficiency.

Sustained activity in topographic areas of human posterior parietal cortex during memory-guided saccades

In a previous study, we identified three cortical areas in human posterior parietal cortex that exhibited topographic responses during memory-guided saccades [visual area 7 (V7), intraparietal sulcus 1 (IPS1), and IPS2], which are candidate homologs of macaque parietal areas such as the lateral intraparietal area and parietal reach region. Here, we show that these areas exhibit sustained delay-period activity, a critical physiological signature of areas in macaque parietal cortex. By varying delay duration, we disambiguated delay-period activity from sensory and motor responses. Mean time courses in the parietal areas were well fit by a linear model comprising three components representing responses to (1) the visual target, (2) the delay period, and (3) the eye movement interval. We estimated the contributions of each component: the response amplitude during the delay period was substantially smaller (<30%) than that elicited by the transient visual target. All three parietal regions showed comparable delay-period response amplitudes, with a trend toward larger responses from V7 to IPS1 and IPS2. Responses to the cue and during the delay period showed clear lateralization with larger responses to trials in which the target was placed in the contralateral visual field, suggesting that both of these components contributed to the topography we measured.

Visual Physiology: Perceived Size Looms Large

Visual illusions tell us that size perception depends heavily upon complex contextual cues, often thought to be extracted by brain areas high in the visual hierarchy. Now, a new study shows that perceived size is reflected in activity as early as the primary visual cortex.

Cytoarchitectonic identification and probabilistic mapping of two distinct areas within the anterior ventral bank of the human intraparietal sulcus

Anatomical studies in the macaque cortex and functional imaging studies in humans have demonstrated the existence of different cortical areas within the intraparietal sulcus (IPS). Such functional segregation, however, does not correlate with presently available architectonic maps of the human brain. This is particularly true for the classical Brodmann map, which is still widely used as an anatomical reference in functional imaging studies. The aim of this cytoarchitectonic mapping study was to use previously defined algorithms to determine whether consistent regions and borders can be found within the cortex of the anterior IPS in a population of 10 post-mortem human brains. Two areas, the human intraparietal area 1 (hIP1) and the human intraparietal area 2 (hIP2), were delineated in serial histological sections of the anterior, lateral bank of the human IPS. The region hIP1 is located posterior and medial to hIP2, and the former is always within the depths of the IPS. The latter, on the other hand, sometimes reaches the free surface of the superior parietal lobule. The delineations were registered to standard reference space, and probabilistic maps were calculated, thereby quantifying the intersubject variability in location and extent of both areas. In the future, they can be a tool for analyzing structure-function relationships and a basis for determining degrees of homology in the IPS among anthropoid primates. We conclude that the human IPS has a more finely grained parcellation than shown in Brodmann's map.

Functional clusters in the human parietal cortex as revealed by an observer-independent meta-analysis of functional activation studies

The human parietal cortex is a highly differentiated structure consisting of cytoarchitectonically defined subareas that are specifically connected with other cortical and subcortical areas. Based on evidence from neurophysiological studies in subhuman primates these subareas are supposed to be functionally highly specialized. Here, we reviewed 51 different neuroimaging studies on healthy subjects with activation of the parietal lobe in statistical parametric maps. Running a cluster analysis on the stereotactic coordinates of the centers of gravity of the activation areas and plotting them into Talairach space showed a high consistency of the mean activation foci for similar paradigms across different laboratories and functional imaging modalities. Our meta-analysis exposed seven distinct pairs of quite symmetrically distributed subareas of the parietal cortex of each hemisphere as well as three unpaired regions that are critically involved in the generation of limb and eye movements in egocentric and allocentric coordinates, but also in attention, memory and cognitive problem solving. These data highlights the modular organization of the human parietal lobe. By its locally interspersed distributed circuits it orchestrates specialized cognitive subfunctions interfacing perception and action. Our meta-analysis provides a new framework for understanding information processing in the human parietal cortex.

TMS in the parietal cortex: updating representations for attention and action

Transcranial magnetic stimulation (TMS) is one of the most recent techniques to have been used in investigations of the parietal cortex but already a number of studies have employed it as a tool in investigations of attentional and sensorimotor processes in the human parietal cortices. The high temporal resolution of TMS has proved to be a particular strength of the technique and the experiments have led to hypotheses about when circumscribed regions of parietal cortex are critical for specific attentional and sensorimotor processes. A consistent theme that runs through many reports is that of a critical contribution of parietal areas when attention or movements are re-directed and representations for attention or action must be updated.

The specificity of cortical region KO to depth structure

Functional MRI studies have identified a cortical region designated as KO between retinotopic areas V3A/B and motion area V5 in human cortex as particularly responsive to motion-defined or kinetic borders. To determine the response of the KO region to more general aspects of structure, we used stereoscopic depth borders and disparate planes with no borders, together with three stimulus types that evoked no depth percept: luminance borders, line contours and illusory phase borders. Responses to these stimuli in the KO region were compared with the responses in retinotopically defined areas that have been variously associated with disparity processing in neurophysiological and fMRI studies. The strongest responses in the KO region were to stimuli evoking perceived depth structure from either disparity or motion cues, but it showed negligible responses either to luminance-based contour stimuli or to edgeless disparity stimuli. We conclude that the region designated as KO is best regarded as a primary center for the generic representation of depth structure rather than any kind of contour specificity.

Mapping the parietal cortex of human and non-human primates

The present essay reviews a series of functional magnetic resonance imaging (fMRI) studies conducted in parallel in humans and awake monkeys, concentrating on the intraparietal sulcus (IPS). MR responses to a range of visual stimuli indicate that the human IPS contains more functional regions along its anterior-posterior extent than are known in the monkey. Human IPS includes four motion sensitive regions, ventral IPS (VIPS), parieto-occipital IPS (POIPS), dorsal IPS medial (DIPSM) and dorsal IPS anterior (DIPSA), which are also sensitive to three-dimensional structure from motion (3D SFM). On the other hand, the monkey IPS contains only one motion sensitive area (VIP), which is not particularly sensitive to 3D SFM. The human IPS includes four regions sensitive to two-dimensional shape and three representations of central vision, while monkey IPS appears to contain only two shape sensitive regions and one central representation. These data support the hypothesis that monkey LIP corresponds to the region of human IPS between DIPSM and POIPS and that a portion of the anterior part of human IPS is evolutionarily new. This additional cortical tissue may provide the capacity for an enhanced visual analysis of moving images necessary for sophisticated control of manipulation and tool handling.

The role of parietal cortex in visuomotor control: what have we learned from neuroimaging?

Research from macaque neurophysiology and human neuropsychology has implicated the parietal cortex in the sensory control of action. Functional neuroimaging has been very valuable in localizing and characterizing specific regions of the human brain involved in visuomotor actions involving different effectors, such as the eyes, head, arms and hands. Here, we review the areas discovered by human neuroimaging, including the putative functional equivalents of the following macaque regions: parietal eye fields (PEF), ventral intraparietal (VIP) area, parietal reach region (PRR) and the anterior intraparietal (AIP) area. We discuss the challenges of studying realistic movements in the imaging environment, the lateralization of visuomotor function, caveats involved in proposing interspecies homologies and the limitations and future directions for neuroimaging studies of visuomotor control.

At what stage of neural processing do perspective depth cues make a difference?

The present study investigated the cortical processing of three-dimensional (3D) perspective cues in humans, to determine how the brain computes depth from a bidimensional retinal image. We recorded visual evoked potentials in 12 subjects in response to flat and in-perspective stimuli, which evoked biphasic potentials over posterior electrodes. The first, positive component (P1, at 90 ms) was not sensitive to perspective, while the second, negative peak (N1 at approximately 150 ms) was significantly larger for 3D stimuli, regardless of attention. The amplitude increase due to perspective was seen on all posterior electrodes, but was largest over the right hemisphere, particularly at parietal sites. Source modeling low-resolution electromagnetic tomography (LORETA) confirmed that among the different areas participating in two- and three-dimensional stimuli processing, the right parietal source is the most enhanced by perspective depth cues. We conclude that the extraction of depth from perspective cues occurs at a second level of stimulus processing, by increasing the activity of the regions involved in 2D stimuli processing, particularly in the right hemisphere, possibly through feedback loops from higher cortical areas. These modulations would participate in the fine-tuned analysis of the 3D features of stimuli.

The functional organization of the intraparietal sulcus in humans and monkeys

In macaque monkeys, the posterior parietal cortex (PPC) is concerned with the integration of multimodal information for constructing a spatial representation of the external world (in relation to the macaque's body or parts thereof), and planning and executing object-centred movements. The areas within the intraparietal sulcus (IPS), in particular, serve as interfaces between the perceptive and motor systems for controlling arm and eye movements in space. We review here the latest evidence for the existence of the IPS areas AIP (anterior intraparietal area), VIP (ventral intraparietal area), MIP (medial intraparietal area), LIP (lateral intraparietal area) and CIP (caudal intraparietal area) in macaques, and discuss putative human equivalents as assessed with functional magnetic resonance imaging. The data suggest that anterior parts of the IPS comprising areas AIP and VIP are relatively well preserved across species. By contrast, posterior areas such as area LIP and CIP have been found more medially in humans, possibly reflecting differences in the evolution of the dorsal visual stream and the inferior parietal lobule. Despite interspecies differences in the precise functional anatomy of the IPS areas, the functional relevance of this sulcus for visuomotor tasks comprising target selections for arm and eye movements, object manipulation and visuospatial attention is similar in humans and macaques, as is also suggested by studies of neurological deficits (apraxia, neglect, Bálint's syndrome) resulting from lesions to this region.

Interactions between number and space in parietal cortex

Since the time of Pythagoras, numerical and spatial representations have been inextricably linked. We suggest that the relationship between the two is deeply rooted in the brain's organization for these capacities. Many behavioural and patient studies have shown that numerical-spatial interactions run far deeper than simply cultural constructions, and, instead, influence behaviour at several levels. By combining two previously independent lines of research, neuroimaging studies of numerical cognition in humans, and physiological studies of spatial cognition in monkeys, we propose that these numerical-spatial interactions arise from common parietal circuits for attention to external space and internal representations of numbers.

A double dissociation between sensitivity to changes in object identity and object orientation in the ventral and dorsal visual streams: a human fMRI study

We used an event-related fMR-adaptation paradigm to investigate changes in BOLD activity in the dorsal and ventral visual streams as a function of object identity and object orientation. Participants viewed successive paired images of real-world, graspable objects, separated by a visual mask. The second image of each pair was either: (i) the same as the first image, (ii) different only in identity, (iii) different only in orientation, or (iv) different in both identity and orientation. A region in the parieto-occipital cortex (dorsal stream) showed a selective increase in BOLD activity with changes in object orientation, but was insensitive to changes in object identity. In contrast, a region in the temporo-occipital cortex (ventral stream) showed a selective increase in activity with changes in identity, but was insensitive to changes in orientation. The differential sensitivity to orientation and identity is consistent with the idea that the dorsal stream plays a critical role in the visual control of object-directed actions while the ventral stream plays a critical role in object perception.

A distributed left hemisphere network active during planning of everyday tool use skills

Determining the relationship between mechanisms involved in action planning and/or execution is critical to understanding the neural bases of skilled behaviors, including tool use. Here we report findings from two fMRI studies of healthy, right-handed adults in which an event-related design was used to distinguish regions involved in planning (i.e. identifying, retrieving and preparing actions associated with a familiar tools' uses) versus executing tool use gestures with the dominant right (experiment 1) and non-dominant left (experiment 2) hands. For either limb, planning tool use actions activates a distributed network in the left cerebral hemisphere consisting of: (i) posterior superior temporal sulcus, along with proximal regions of the middle and superior temporal gyri; (ii) inferior frontal and ventral premotor cortices; (iii) two distinct parietal areas, one located in the anterior supramarginal gyrus (SMG) and another in posterior SMG and angular gyrus; and (iv) dorsolateral prefrontal cortex (DLFPC). With the exception of left DLFPC, adjacent and partially overlapping sub-regions of left parietal, frontal and temporal cortex are also engaged during action execution. We suggest that this left lateralized network constitutes a neural substrate for the interaction of semantic and motoric representations upon which meaningful skills depend.

3D shape perception from combined depth cues in human visual cortex

Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception.

Virtual lesions of the anterior intraparietal area disrupt goal-dependent on-line adjustments of grasp

Adaptive motor behavior requires efficient error detection and correction. The posterior parietal cortex is critical for on-line control of reach-to-grasp movements. Here we show a causal relationship between disruption of cortical activity within the anterior intraparietal sulcus (aIPS) by transcranial magnetic stimulation (TMS) and disruption of goal-directed prehensile actions (either grip size or forearm rotation, depending on the task goal, with reaching preserved in either case). Deficits were elicited by applying TMS within 65 ms after object perturbation, which attributes a rapid control process on the basis of visual feedback to aIPS. No aperture deficits were produced when TMS was applied to a more caudal region within the intraparietal sulcus, to the parieto-occipital complex (putative V6, V6A) or to the hand area of primary motor cortex. We contend that aIPS is critical for dynamic error detection during goal-dependent reach-to-grasp action that is visually guided.

Neural mechanisms of three-dimensional vision

We can see things in three dimensions because the visual system re-constructs the three-dimensional (3D) configurations of objects from their two-dimensional (2D) images projected onto the retinas. The purpose of this paper is to give an overview of the psychological background and recent physiological findings concerning three-dimensional vision. Psychophysical and computational studies have suggested that in the visual system the 3D surface orientation is first estimated independently from individual depth cues--such as binocular disparity, as well as various monocular cues including texture gradients--and then the information from these different depth cues is integrated to construct a generalized representation of the 3D surface geometry. Neurons involved in low-level disparity processing, or the detection of local absolute disparity, were found mainly in the occipital cortex, whereas neurons involved in high-level disparity processing, or the reconstruction of 3D surface orientation through the computation of disparity gradients, were found mainly in the parietal area caudal intraparietal sulcus (CIP). Neurons sensitive to texture gradients, which is one of the major monocular cues, were also found in CIP. The majority of these neurons were sensitive to disparity gradients as well, suggesting their involvement in the computation of 3D surface orientation. In CIP, neurons sensitive to multiple depth cues were widely distributed together with those sensitive to a specific depth cue, suggesting CIP's involvement in the integration of depth information from different sources. In addition, human and monkey imaging studies have indicated convergence of multiple depth cues in CIP. These neurophysiological findings suggest that CIP plays a critical role in 3D vision by constructing a generalized representation of the 3D surface geometry of objects.