Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in macaque ventral premotor cortex

Neurons responding during action execution and action observation were discovered in the ventral premotor cortex 3 decades ago. However, the visual features that drive the responses of action observation/execution neurons (AOENs) have not been revealed at present. We investigated the neural responses of AOENs in ventral premotor area F5c of 4 macaques during the observation of action videos and crucial control stimuli. The large majority of AOENs showed highly phasic responses during the action videos, with a preference for the moment that the hand made contact with the object. They also responded to an abstract shape moving towards but not interacting with an object, even when the shape moved on a scrambled background, implying that most AOENs in F5c do not require the perception of causality or a meaningful action. Additionally, the majority of AOENs responded to static frames of the videos. Our findings show that very elementary stimuli, even without a grasping context, are sufficient to drive responses in F5c AOENs.

Mixed selectivity in monkey anterior intraparietal area during visual and motor processes

Classical studies suggest that the anterior intraparietal area (AIP) contributes to the encoding of specific information such as objects and actions of self and others, through a variety of neuronal classes, such as canonical, motor and mirror neurons. However, these studies typically focused on a single variable, leaving it unclear whether distinct sets of AIP neurons encode a single or multiple sources of information and how multimodal coding emerges. Here, we chronically recorded monkey AIP neurons in a variety of tasks and conditions classically employed in separate experiments. Most cells exhibited mixed selectivity for observed objects, executed actions, and observed actions, enhanced when this information came from the monkey's peripersonal working space. In contrast with the classical view, our findings indicate that multimodal coding emerges in AIP from partially-mixed selectivity of individual neurons for a variety of information relevant for planning actions directed to both physical objects and other subjects.

The neural mechanisms of manual dexterity

The hand endows us with unparalleled precision and versatility in our interactions with objects, from mundane activities such as grasping to extraordinary ones such as virtuoso pianism. The complex anatomy of the human hand combined with expansive and specialized neuronal control circuits allows a wide range of precise manual behaviours. To support these behaviours, an exquisite sensory apparatus, spanning the modalities of touch and proprioception, conveys detailed and timely information about our interactions with objects and about the objects themselves. The study of manual dexterity provides a unique lens into the sensorimotor mechanisms that endow the nervous system with the ability to flexibly generate complex behaviour.

Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types

We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.

A distinction concerning vision-for-action and affordance perception

In this paper, I offer a discussion concerning the conceptual connection between the notion of vision-for-action and the one of affordance perception. I first analyze the notion of vision-for-action. I then analyze a notion usually coupled with it: the notion of affordance perception, the main insights behind which are guiding several current neuroscientific enterprises and the related philosophical speculations. Then, I argue that we should not couple these two notions with a light heart: though these two processes can be, from a theoretical point of view, related, we should be careful in inferring the actual and effective occurrence of the latter in the presence of the former. This will be done by carrying out a conceptual analysis of the experimental evidence coming from the 'Two Visual Systems Model', which is the main reference in the literature on affordance perception and vision-for-action. My point has strong philosophical implications for our view concerning the best interpretation of how vision-for-action really works, and the specific relation it actually entertains with affordance perception.

Effective Connectivity Reveals an Interconnected Inferotemporal Network for Three-Dimensional Structure Processing

Disparity-defined 3D shape is processed in both the ventral and the dorsal visual stream. The network of cortical areas that is activated during the processing of disparity-defined 3D shape includes, in addition to parietal and premotor areas, three clearly distinct regions in inferotemporal cortex (ITC). To investigate the connectivity of the latter regions, we combined electrical stimulation with fMRI in male macaque monkeys. Electrical stimulation of each of the 3D-structure nodes in ITC mainly elicited increased fMRI activations in the other 3D-structure nodes and more variably in other parts of ventral visual cortex. Importantly, no increased activation was found in parietal areas, nor in PFC, whereas microstimulation in posterior parietal cortex did activate the ITC. Our results indicate that 3D-structure nodes in ITC form a strongly interconnected network, receiving input from parietal areas implicated in 3D-structure processing.SIGNIFICANCE STATEMENT Previous studies combining electrical microstimulation with functional imaging showed an interconnected set of regions in the ventral stream processing faces or bodies, but is has been unclear whether the same is true for other visual categories. Here the authors show that there is a connected system of stereo-selective regions in inferotemporal cortex, receiving input from parietal areas in the dorsal stream.

Localization of movable electrodes in a multi-electrode microdrive in nonhuman primates

BACKGROUND

Recently, large-scale semi-chronic recording systems have been developed, unique in their capability to record simultaneously from multiple individually moveable electrodes. As these recording systems can cover a large area, knowledge of the exact location of each individual electrode is crucial. Currently, the only method of keeping track of electrode depth and thus location is through detailed notebook keeping on neural activity.

NEW METHOD

We have improved the electrode localization by combining pre- and postoperative anatomical magnetic resonance imaging (MRI) scans with high resolution computed tomography (CT) scans throughout the experiment, and validated our method by comparing the resulting location estimates with traditional notebook-keeping. Finally, the actual location of a selection of electrodes was marked at the end of the experiment by creating small metallic depositions using electrical stimulation, and thereby made visible on MRI.

RESULTS

Combining CT scans with a high resolution, artefact reducing sequence during the experiment with a preoperative MRI scan provides crucial information about the exact electrode location of multielectrode arrays with individually moveable electrodes.

COMPARISON WITH EXISTING METHODS

The information obtained from the hybrid CT-MR image and the notes on spiking activity showed a similar pattern, with the clear advantage of the visualization of the exact position of the electrodes using our method.

CONCLUSIONS

The described technique allows for a precise anatomical identification of the recorded brain areas and thus to draw strong conclusions about the role of each targeted cortical area in the behavior under study.

Investigating Object Representations in the Macaque Dorsal Visual Stream Using Single-unit Recordings

Previous studies have shown that neurons in parieto-frontal areas of the macaque brain can be highly selective for real-world objects, disparity-defined curved surfaces, and images of real-world objects (with and without disparity) in a similar manner as described in the ventral visual stream. In addition, parieto-frontal areas are believed to convert visual object information into appropriate motor outputs, such as the pre-shaping of the hand during grasping. To better characterize object selectivity in the cortical network involved in visuomotor transformations, we provide a battery of tests intended to analyze the visual object selectivity of neurons in parieto-frontal regions.

Human infants can generate vergence responses to retinal disparity by 5 to 10 weeks of age

Vergence is defined as a binocular eye movement during which the two eyes move in opposite directions to align to a target in depth. In adults, fine vergence control is driven primarily by interocular retinal image disparity. Although infants have not typically been shown to respond to disparity until 3 to 5 months postpartum, they have been shown to align their eyes from hours after birth. It remains unclear what drives these responses in young infants. In this experiment, 5- to 10-week-old human infants were presented with a dynamic random noise stimulus oscillating in disparity at 0.1 Hz over an amplitude of 2° for 30 s. Fourier transforms of the horizontal eye movements revealed significant disparity-driven responses at the frequency of the stimulus in over half of the tested infants. Because the stimulus updated dynamically, this experiment precluded the possibility of independent monocular fixations to a sustained target. These data demonstrate cortical binocular function in humans by five weeks, the youngest age tested here, which is as much as two months younger than previously believed.

Towards a unified perspective of object shape and motion processing in human dorsal cortex

Although object-related areas were discovered in human parietal cortex a decade ago, surprisingly little is known about the nature and purpose of these representations, and how they differ from those in the ventral processing stream. In this article, we review evidence for the unique contribution of object areas of dorsal cortex to three-dimensional (3-D) shape representation, the localization of objects in space, and in guiding reaching and grasping actions. We also highlight the role of dorsal cortex in form-motion interaction and spatiotemporal integration, possible functional relationships between 3-D shape and motion processing, and how these processes operate together in the service of supporting goal-directed actions with objects. Fundamental differences between the nature of object representations in the dorsal versus ventral processing streams are considered, with an emphasis on how and why dorsal cortex supports veridical (rather than invariant) representations of objects to guide goal-directed hand actions in dynamic visual environments.

Hierarchical Organization Within the Ventral Premotor Cortex of the Macaque Monkey

Recent studies have revealed that the ventral premotor cortex (PMv) of nonhuman primates plays a pivotal role in various behaviors that require the transformation of sensory cues to appropriate actions. Examples include decision-making based on various sensory cues, preparation for upcoming motor behavior, adaptive sensorimotor transformation, and the generation of motor commands using rapid sensory feedback. Although the PMv has frequently been regarded as a single entity, it can be divided into at least five functionally distinct regions: F4, a dorsal convexity region immediately rostral to the primary motor cortex (M1); F5p, a cortical region immediately rostral to F4, lying within the arcuate sulcus; F5c, a ventral convexity region rostral to F4; and F5a, located in the caudal bank of the arcuate sulcus inferior limb lateral to F5p. Among these, F4 can be further divided into dorsal and ventral subregions (F4d and F4v), which are involved in forelimb and orofacial movements, respectively. F5p contains "mirror neurons" to understand others' actions based on visual and other types of information, and F4d and F5p work together as a functional complex involved in controlling forelimb and eye movements, most efficiently in the execution and completion of coordinated eye-hand movements for reaching and grasping under visual guidance. In contrast, F5c and F5a are hierarchically higher than the F4d, F5p, and F5v complexes, and play a role in decision-making based on various sensory discriminations. Hence, the PMv subregions form a hierarchically organized integral system from decision-making to eye-hand coordination under various behavioral circumstances.

More than Action: The Dorsal Pathway Contributes to the Perception of 3-D Structure

An evolving view in cognitive neuroscience is that the dorsal visual pathway not only plays a key role in visuomotor behavior but that it also contributes functionally to the recognition of objects. To characterize the nature of the object representations derived by the dorsal pathway, we assessed perceptual performance in the context of the continuous flash suppression paradigm, which suppresses object processing in the ventral pathway while sparing computation in the dorsal pathway. In a series of experiments, prime stimuli, which were rendered imperceptible by the continuous flash suppression, still contributed to perceptual decisions related to the subsequent perceptible target stimuli. However, the contribution of the prime to perception was contingent on the prime's structural coherence, in that a perceptual advantage was observed only for targets primed by objects with legitimate 3-D structure. Finally, we obtained additional evidence to demonstrate that the processing of the suppressed objects was contingent on the magnocellular, rather than the parvocellular, system, further linking the processing of the suppressed stimuli to the dorsal pathway. Together, these results provide novel evidence that the dorsal pathway does not only support visuomotor control but, rather, that it also derives the structural description of 3-D objects and contributes to shape perception.

Thresholds for sine-wave corrugations defined by binocular disparity in random dot stereograms: Factor analysis of individual differences reveals two stereoscopic mechanisms tuned for spatial frequency

Threshold functions for sinusoidal depth corrugations typically reach their minimum (highest sensitivity) at spatial frequencies of 0.2-0.4 cycles/degree (cpd), with lower thresholds for horizontal than vertical corrugations at low spatial frequencies. To elucidate spatial frequency and orientation tuning of stereoscopic mechanisms, we measured the disparity sensitivity functions, and used factor analytic techniques to estimate the existence of independent underlying stereo channels. The data set (N = 30 individuals) was for horizontal and vertical corrugations of spatial frequencies ranging from 0.1 to 1.6 cpd. A principal component analysis of disparity sensitivities (log-arcsec) revealed that two significant factors accounted for 70% of the variability. Following Varimax rotation to approximate "simple structure", one factor clearly loaded onto low spatial frequencies (≤0.4 cpd), and a second was tuned to higher spatial frequencies (≥0.8 cpd). Each factor had nearly identical tuning (loadings) for horizontal and vertical patterns. The finding of separate factors for low and high spatial frequencies is consistent with previous studies. The failure to find separate factors for horizontal and vertical corrugations is somewhat surprising because the neuronal mechanisms are believed to be different. Following an oblique rotation (Direct Oblimin), the two factors correlated significantly, suggesting some interdependence rather than full independence between the two factors.

The large-scale organization of shape processing in the ventral and dorsal pathways

Although shape perception is considered a function of the ventral visual pathway, evidence suggests that the dorsal pathway also derives shape-based representations. In two psychophysics and neuroimaging experiments, we characterized the response properties, topographical organization and perceptual relevance of these representations. In both pathways, shape sensitivity increased from early visual cortex to extrastriate cortex but then decreased in anterior regions. Moreover, the lateral aspect of the ventral pathway and posterior regions of the dorsal pathway were sensitive to the availability of fundamental shape properties, even for unrecognizable images. This apparent representational similarity between the posterior-dorsal and lateral-ventral regions was corroborated by a multivariate analysis. Finally, as with ventral pathway, the activation profile of posterior dorsal regions was correlated with recognition performance, suggesting a possible contribution to perception. These findings challenge a strict functional dichotomy between the pathways and suggest a more distributed model of shape processing.

We rely on our sense of vision to perceive the world around us and the objects within it. We also use vision to guide our interactions with objects. One of the most influential theories in cognitive neuroscience is the idea that separate pathways within the brain support these two processes. The ventral pathway is in charge of vision-for-perception. It analyses the features that help us recognize objects, such as their color, size or shape, enabling us to identify the hammer in a toolbox, for example. The dorsal pathway is responsible for vision-for-action. It processes features that help us interact with objects, such as their movement and location, enabling us to use the hammer to strike a nail.

However, recent studies have suggested that the ventral and dorsal pathways may not be as independent as originally thought. Freud et al. now test this idea by examining if the dorsal vision-for-action pathway can also perceive and process objects.

Healthy volunteers viewed pictures of objects while lying inside a brain scanner. Some of the objects in the pictures were intact, whereas others had been distorted. If a brain region shows greater activation when viewing intact objects than distorted ones, it implies that that region is sensitive to the normal shapes of objects. Freud et al. found that both the ventral and dorsal pathways were sensitive to shape, with some areas in the two pathways showing highly similar responses. Furthermore, the shape sensitivity of certain regions within the dorsal pathway correlated with the volunteers’ ability to recognize the objects. This suggests that regions distributed across both pathways – and not just the ventral one – may contribute to object recognition.

The two-pathways hypothesis has governed our understanding of vision and of other sensory systems including hearing for several decades. By challenging the binary distinction between the two pathways, the results of Freud et al. suggest that models of sensory processing may require updating. This improved understanding may ultimately improve diagnosis and treatment of perceptual disorders such as agnosia, in which patients struggle to recognize objects.

Functional interactions between the macaque dorsal and ventral visual pathways during three-dimensional object vision

The division of labor between the dorsal and the ventral visual stream in the primate brain has inspired numerous studies on the visual system in humans and in nonhuman primates. However, how and under which circumstances the two visual streams interact is still poorly understood. Here we review evidence from anatomy, modelling, electrophysiology, electrical microstimulation (EM), reversible inactivation and functional imaging in the macaque monkey aimed at clarifying at which levels in the hierarchy of visual areas the two streams interact, and what type of information might be exchanged between the two streams during three-dimensional (3D) object viewing. Neurons in both streams encode 3D structure from binocular disparity, synchronized activity between parietal and inferotemporal areas is present during 3D structure categorization, and clusters of 3D structure-selective neurons in parietal cortex are anatomically connected to ventral stream areas. In addition, caudal intraparietal cortex exerts a causal influence on 3D-structure related activations in more anterior parietal cortex and in inferotemporal cortex. Thus, both anatomical and functional evidence indicates that the dorsal and the ventral visual stream interact during 3D object viewing.

Object vision to hand action in macaque parietal, premotor, and motor cortices

Grasping requires translating object geometries into appropriate hand shapes. How the brain computes these transformations is currently unclear. We investigated three key areas of the macaque cortical grasping circuit with microelectrode arrays and found cooperative but anatomically separated visual and motor processes. The parietal area AIP operated primarily in a visual mode. Its neuronal population revealed a specialization for shape processing, even for abstract geometries, and processed object features ultimately important for grasping. Premotor area F5 acted as a hub that shared the visual coding of AIP only temporarily and switched to highly dominant motor signals towards movement planning and execution. We visualize these non-discrete premotor signals that drive the primary motor cortex M1 to reflect the movement of the grasping hand. Our results reveal visual and motor features encoded in the grasping circuit and their communication to achieve transformation for grasping.

DOI:http://dx.doi.org/10.7554/eLife.15278.001

In order to grasp and manipulate objects, our brains have to transform information about an object (such as its size, shape and position) into commands about movement that are sent to our hands. Previous work suggests that in primates (including humans and monkeys), this transformation is coordinated in three key brain areas: the parietal cortex, the premotor cortex and the motor cortex. But exactly how these transformations are computed is still not clear.

Schaffelhofer and Scherberger attempted to find out how this transformation happens by recording the electrical activity from different brain areas as monkeys reached out to grasp different objects. The specific brain areas studied were the anterior intraparietal (AIP) area of the parietal cortex, a part of the premotor cortex known as F5, and the region of the motor cortex that controls hand movements. The exact movement made by the monkeys’ hands was also recorded.

Analysing the recorded brain activity revealed that the three brain regions worked together to transform information about an object into commands for the hand, although each region also had its own specific, separate role in this process. Neurons in the AIP area of the parietal cortex mostly dealt with visual information about the object. These neurons specialized in processing information about the shape of an object, including information that was ultimately important for grasping it. In contrast, the premotor area F5 represented visual information about the object only briefly, quickly switching to representing information about the upcoming movement as it was planned and carried out. Finally, the neurons in the primary motor cortex were only active during the actual hand movement, and their activity strongly reflected the action of hand as it grasped the object.

Overall, the results presented by Schaffelhofer and Scherberger suggest that grasping movements are generated from visual information about the object via AIP and F5 neurons communicating with each other. The strong links between the premotor and motor cortex also suggest that a common network related to movement executes and refines the prepared plan of movement. Further investigations are now needed to reveal how such networks process the information they receive.

DOI:http://dx.doi.org/10.7554/eLife.15278.002

Through the forest of motor representations

Following neuroscience, and using different labels, several philosophers have addressed the idea of the presence of a single representational mechanism lying in between (visual) perceptual processes and motor processes involved in different functions and useful for shaping suitable action performances: a motor representation (MR). MRs are the naturalized mental antecedents of action. This paper presents a new, non-monolithic view of MRs, according to which, contrarily to the received view, when looking at in between (visual) perceptual processes and motor processes, we find not only a single representational mechanism with different functions, but an ensemble of different sub-representational phenomena, each of which with a different function. This new view is able to avoid several issues emerging from the literature and to address something the literature is silent about, which however turns out to be crucial for a theory of MRs.

Posterior Parietal Cortex Drives Inferotemporal Activations During Three-Dimensional Object Vision

The primate visual system consists of a ventral stream, specialized for object recognition, and a dorsal visual stream, which is crucial for spatial vision and actions. However, little is known about the interactions and information flow between these two streams. We investigated these interactions within the network processing three-dimensional (3D) object information, comprising both the dorsal and ventral stream. Reversible inactivation of the macaque caudal intraparietal area (CIP) during functional magnetic resonance imaging (fMRI) reduced fMRI activations in posterior parietal cortex in the dorsal stream and, surprisingly, also in the inferotemporal cortex (ITC) in the ventral visual stream. Moreover, CIP inactivation caused a perceptual deficit in a depth-structure categorization task. CIP-microstimulation during fMRI further suggests that CIP projects via posterior parietal areas to the ITC in the ventral stream. To our knowledge, these results provide the first causal evidence for the flow of visual 3D information from the dorsal stream to the ventral stream, and identify CIP as a key area for depth-structure processing. Thus, combining reversible inactivation and electrical microstimulation during fMRI provides a detailed view of the functional interactions between the two visual processing streams.

Receptive field properties of neurons in the macaque anterior intraparietal area

Visual object information is necessary for grasping. In primates, the anterior intraparietal area (AIP) plays an essential role in visually guided grasping. Neurons in AIP encode features of objects, but no study has systematically investigated the receptive field (RF) of AIP neurons. We mapped the RF of posterior AIP (pAIP) neurons in the central visual field, using images of objects and small line fragments that evoked robust responses, together with less effective stimuli. The RF sizes we measured varied between 3°(2)and 90°(2), with the highest response either at the fixation point or at parafoveal positions. A large fraction of pAIP neurons showed nonuniform RFs, with multiple local maxima in both ipsilateral and contralateral hemifields. Moreover, the RF profile could depend strongly on the stimulus used to map the RF. Highly similar results were obtained with the smallest stimulus that evoked reliable responses (line fragments measuring 1-2°). The nonuniformity and dependence of the RF profile on the stimulus in pAIP were comparable to previous observations in the anterior part of the lateral intraparietal area (aLIP), but the average RF of pAIP neurons was located at the fovea whereas the average RF of aLIP neurons was located parafoveally. Thus nonuniformity and stimulus dependence of the RF may represent general RF properties of neurons in the dorsal visual stream involved in object analysis, which contrast markedly with those of neurons in the ventral visual stream.

The body and objects represented in the ventral stream of the parieto-premotor network

The network between the parietal cortex and premotor cortex has a pivotal role in sensory-motor control. Grasping-related neurons in the anterior intraparietal area (AIP) and the ventral premotor cortex (F5) showed complementary properties each other. The object information for grasping is sent from the parietal cortex to the premotor cortex for sensory-motor transformation, and the backward signal from the premotor cortex to parietal cortex can be considered an efference copy/corollary discharge that is used to predict sensory outcome during motor behavior. Mirror neurons that represent both own action and other's action are involved in this system. This system also very well fits with body schema that reflects online state of the body during motor execution. We speculate that the parieto-premotor network, which includes the mirror neuron system, is key for mapping one's own body and the bodies of others. This means that the neuronal substrates that control one's own action and the mirror neuron system are shared with the "who" system, which is related to the recognition of action contribution, i.e., sense of agency. Representation of own and other's body in the parieto-premotor network is key to link between sensory-motor control and higher-order cognitive functions.

Linking Objects to Actions: Encoding of Target Object and Grasping Strategy in Primate Ventral Premotor Cortex

Neural activity in ventral premotor cortex (PMv) has been associated with the process of matching perceived objects with the motor commands needed to grasp them. It remains unclear how PMv networks can flexibly link percepts of objects affording multiple grasp options into a final desired hand action. Here, we use a relational encoding approach to track the functional state of PMv neuronal ensembles in macaque monkeys through the process of passive viewing, grip planning, and grasping movement execution. We used objects affording multiple possible grip strategies. The task included separate instructed delay periods for object presentation and grip instruction. This approach allowed us to distinguish responses elicited by the visual presentation of the objects from those associated with selecting a given motor plan for grasping. We show that PMv continuously incorporates information related to object shape and grip strategy as it becomes available, revealing a transition from a set of ensemble states initially most closely related to objects, to a new set of ensemble patterns reflecting unique object-grip combinations. These results suggest that PMv dynamically combines percepts, gradually navigating toward activity patterns associated with specific volitional actions, rather than directly mapping perceptual object properties onto categorical grip representations. Our results support the idea that PMv is part of a network that dynamically computes motor plans from perceptual information.

SIGNIFICANCE STATEMENT

The present work demonstrates that the activity of groups of neurons in primate ventral premotor cortex reflects information related to visually presented objects, as well as the motor strategy used to grasp them, linking individual objects to multiple possible grips. PMv could provide useful control signals for neuroprosthetic assistive devices designed to interact with objects in a flexible way.

Effects of Microstimulation in the Anterior Intraparietal Area during Three-Dimensional Shape Categorization

The anterior intraparietal area (AIP) of rhesus monkeys is part of the dorsal visual stream and contains neurons whose visual response properties are commensurate with a role in three-dimensional (3D) shape perception. Neuronal responses in AIP signal the depth structure of disparity-defined 3D shapes, reflect the choices of monkeys while they categorize 3D shapes, and mirror the behavioral variability across different stimulus conditions during 3D-shape categorization. However, direct evidence for a role of AIP in 3D-shape perception has been lacking. We trained rhesus monkeys to categorize disparity-defined 3D shapes and examined AIP's contribution to 3D-shape categorization by microstimulating in clusters of 3D-shape selective AIP neurons during task performance. We find that microstimulation effects on choices (monkey M1) and reaction times (monkey M1 and M2) depend on the 3D-shape preference of the stimulated site. Moreover, electrical stimulation of the same cells, during either the 3D-shape-categorization task or a saccade task, could affect behavior differently. Interestingly, in one monkey we observed a strong correlation between the strength of choice-related AIP activity (choice probabilities) and the influence of microstimulation on 3D-shape-categorization behavior (choices and reaction time). These findings propose AIP as part of the network responsible for 3D-shape perception. The results also show that the anterior intraparietal cortex contains cells with different tuning properties, i.e. 3D-shape- or saccade-related, that can be dynamically read out depending on the requirements of the task at hand.

Object Affordance Modulates Visual Responses in the Macaque Medial Posterior Parietal Cortex

Area V6A is a visuomotor area of the dorsomedial visual stream that contains cells modulated by object observation and by grip formation. As different objects have different shapes but also evoke different grips, the response selectivity during object presentation could reflect either the coding of object geometry or object affordances. To clarify this point, we here investigate neural responses of V6A cells when monkeys observed two objects with similar visual features but different contextual information, such as the evoked grip type. We demonstrate that many V6A cells respond to the visual presentation of objects and about 30% of them by the object affordance. Given that area V6A is an early stage in the visuomotor processes underlying grasping, these data suggest that V6A may participate in the computation of object affordances. These results add some elements in the recent literature about the role of the dorsal visual stream areas in object representation and contribute in elucidating the neural correlates of the extraction of action-relevant information from general object properties, in agreement with recent neuroimaging studies on humans showing that vision of graspable objects activates action coding in the dorsomedial visual steam.

Shape representations in the primate dorsal visual stream

The primate visual system extracts object shape information for object recognition in the ventral visual stream. Recent research has demonstrated that object shape is also processed in the dorsal visual stream, which is specialized for spatial vision and the planning of actions. A number of studies have investigated the coding of 2D shape in the anterior intraparietal area (AIP), one of the end-stage areas of the dorsal stream which has been implicated in the extraction of affordances for the purpose of grasping. These findings challenge the current understanding of area AIP as a critical stage in the dorsal stream for the extraction of object affordances. The representation of three-dimensional (3D) shape has been studied in two interconnected areas known to be critical for object grasping: area AIP and area F5a in the ventral premotor cortex (PMv), to which AIP projects. In both areas neurons respond selectively to 3D shape defined by binocular disparity, but the latency of the neural selectivity is approximately 10 ms longer in F5a compared to AIP, consistent with its higher position in the hierarchy of cortical areas. Furthermore, F5a neurons were more sensitive to small amplitudes of 3D curvature and could detect subtle differences in 3D structure more reliably than AIP neurons. In both areas, 3D-shape selective neurons were co-localized with neurons showing motor-related activity during object grasping in the dark, indicating a close convergence of visual and motor information on the same clusters of neurons.

Effective connectivity of depth-structure-selective patches in the lateral bank of the macaque intraparietal sulcus

Extrastriate cortical areas are frequently composed of subpopulations of neurons encoding specific features or stimuli, such as color, disparity, or faces, and patches of neurons encoding similar stimulus properties are typically embedded in interconnected networks, such as the attention or face-processing network. The goal of the current study was to examine the effective connectivity of subsectors of neurons in the same cortical area with highly similar neuronal response properties. We first recorded single- and multi-unit activity to identify two neuronal patches in the anterior part of the macaque intraparietal sulcus (IPS) showing the same depth structure selectivity and then employed electrical microstimulation during functional magnetic resonance imaging in these patches to determine the effective connectivity of these patches. The two IPS subsectors we identified-with the same neuronal response properties and in some cases separated by only 3 mm-were effectively connected to remarkably distinct cortical networks in both dorsal and ventral stream in three macaques. Conversely, the differences in effective connectivity could account for the known visual-to-motor gradient within the anterior IPS. These results clarify the role of the anterior IPS as a pivotal brain region where dorsal and ventral visual stream interact during object analysis. Thus, in addition to the anatomical connectivity of cortical areas and the properties of individual neurons in these areas, the effective connectivity provides novel key insights into the widespread functional networks that support behavior.

Monkey cortex through fMRI glasses

In 1998 several groups reported the feasibility of fMRI experiments in monkeys, with the goal to bridge the gap between invasive nonhuman primate studies and human functional imaging. These studies yielded critical insights in the neuronal underpinnings of the BOLD signal. Furthermore, the technology has been successful in guiding electrophysiological recordings and identifying focal perturbation targets. Finally, invaluable information was obtained concerning human brain evolution. We here provide a comprehensive overview of awake monkey fMRI studies mainly confined to the visual system. We review the latest insights about the topographic organization of monkey visual cortex and discuss the spatial relationships between retinotopy and category- and feature-selective clusters. We briefly discuss the functional layout of parietal and frontal cortex and continue with a summary of some fascinating functional and effective connectivity studies. Finally, we review recent comparative fMRI experiments and speculate about the future of nonhuman primate imaging.

Modeling the shape hierarchy for visually guided grasping

The monkey anterior intraparietal area (AIP) encodes visual information about three-dimensional object shape that is used to shape the hand for grasping. We modeled shape tuning in visual AIP neurons and its relationship with curvature and gradient information from the caudal intraparietal area (CIP). The main goal was to gain insight into the kinds of shape parameterizations that can account for AIP tuning and that are consistent with both the inputs to AIP and the role of AIP in grasping. We first experimented with superquadric shape parameters. We considered superquadrics because they occupy a role in robotics that is similar to AIP, in that superquadric fits are derived from visual input and used for grasp planning. We also experimented with an alternative shape parameterization that was based on an Isomap dimension reduction of spatial derivatives of depth (i.e., distance from the observer to the object surface). We considered an Isomap-based model because its parameters lacked discontinuities between similar shapes. When we matched the dimension of the Isomap to the number of superquadric parameters, the superquadric model fit the AIP data somewhat more closely. However, higher-dimensional Isomaps provided excellent fits. Also, we found that the Isomap parameters could be approximated much more accurately than superquadric parameters by feedforward neural networks with CIP-like inputs. We conclude that Isomaps, or perhaps alternative dimension reductions of visual inputs to AIP, provide a promising model of AIP electrophysiology data. Further work is needed to test whether such shape parameterizations actually provide an effective basis for grasp control.

Grasping Execution and Grasping Observation Activity of Single Neurons in the Macaque Anterior Intraparietal Area

Primates use vision to guide their actions in everyday life. Visually guided object grasping is known to rely on a network of cortical areas located in the parietal and premotor cortex. We recorded in the anterior intraparietal area (AIP), an area in the dorsal visual stream that is critical for object grasping and densely connected with the premotor cortex, while monkeys were grasping objects under visual guidance and during passive fixation of videos of grasping actions from the first-person perspective. All AIP neurons in this study responded during grasping execution in the light, that is, became more active after the hand had started to move toward the object and during grasping in the dark. More than half of these AIP neurons responded during the observation of a video of the same grasping actions on a display. Furthermore, these AIP neurons responded as strongly during passive fixation of movements of a hand on a scrambled background and to a lesser extent to a shape appearing within the visual field near the object. Therefore, AIP neurons responding during grasping execution also respond during passive observation of grasping actions and most of them even during passive observation of movements of a simple shape in the visual field.

Visual field preferences of object analysis for grasping with one hand

When we grasp an object using one hand, the opposite hemisphere predominantly guides the motor control of grasp movements (Davare et al., 2007; Rice et al., 2007). However, it is unclear whether visual object analysis for grasp control relies more on inputs (a) from the contralateral than the ipsilateral visual field, (b) from one dominant visual field regardless of the grasping hand, or (c) from both visual fields equally. For bimanual grasping of a single object we have recently demonstrated a visual field preference for the left visual field (Le and Niemeier, 2013a,b), consistent with a general right-hemisphere dominance for sensorimotor control of bimanual grasps (Le et al., 2014). But visual field differences have never been tested for unimanual grasping. Therefore, here we asked right-handed participants to fixate to the left or right of an object and then grasp the object either with their right or left hand using a precision grip. We found that participants grasping with their right hand performed better with objects in the right visual field: maximum grip apertures (MGAs) were more closely matched to the object width and were smaller than for objects in the left visual field. In contrast, when people grasped with their left hand, preferences switched to the left visual field. What is more, MGA scaling with the left hand showed greater visual field differences compared to right-hand grasping. Our data suggest that, visual object analysis for unimanual grasping shows a preference for visual information from the ipsilateral visual field, and that the left hemisphere is better equipped to control grasps in both visual fields.

Coding of shape features in the macaque anterior intraparietal area

The exquisite ability of primates to grasp and manipulate objects relies on the transformation of visual information into motor commands. To this end, the visual system extracts object affordances that can be used to program and execute the appropriate grip. The macaque anterior intraparietal (AIP) area has been implicated in the extraction of affordances for the purpose of grasping. Neurons in the AIP area respond during visually guided grasping and to the visual presentation of objects. A subset of AIP neurons is also activated by two-dimensional images of objects and even by outline contours defining the object shape, but it is unknown how AIP neurons actually represent object shape. In this study, we used a stimulus reduction approach to determine the minimum effective shape feature evoking AIP responses. AIP neurons responding to outline shapes also responded selectively to very small fragment stimuli measuring only 1-2°. This fragment selectivity could not be explained by differences in eye movements or simple orientation selectivity, but proved to be highly dependent on the relative position of the stimulus in the receptive field. Our findings challenge the current understanding of the AIP area as a critical stage in the dorsal stream for the extraction of object affordances.

Human fMRI reveals that delayed action re-recruits visual perception

Behavioral and neuropsychological research suggests that delayed actions rely on different neural substrates than immediate actions; however, the specific brain areas implicated in the two types of actions remain unknown. We used functional magnetic resonance imaging (fMRI) to measure human brain activation during delayed grasping and reaching. Specifically, we examined activation during visual stimulation and action execution separated by a 18-s delay interval in which subjects had to remember an intended action toward the remembered object. The long delay interval enabled us to unambiguously distinguish visual, memory-related, and action responses. Most strikingly, we observed reactivation of the lateral occipital complex (LOC), a ventral-stream area implicated in visual object recognition, and early visual cortex (EVC) at the time of action. Importantly this reactivation was observed even though participants remained in complete darkness with no visual stimulation at the time of the action. Moreover, within EVC, higher activation was observed for grasping than reaching during both vision and action execution. Areas in the dorsal visual stream were activated during action execution as expected and, for some, also during vision. Several areas, including the anterior intraparietal sulcus (aIPS), dorsal premotor cortex (PMd), primary motor cortex (M1) and the supplementary motor area (SMA), showed sustained activation during the delay phase. We propose that during delayed actions, dorsal-stream areas plan and maintain coarse action goals; however, at the time of execution, motor programming requires re-recruitment of detailed visual information about the object through reactivation of (1) ventral-stream areas involved in object perception and (2) early visual areas that contain richly detailed visual representations, particularly for grasping.

Left visual field preference for a bimanual grasping task with ecologically valid object sizes

Grasping using two forelimbs in opposition to one another is evolutionary older than the hand with an opposable thumb (Whishaw and Coles in Behav Brain Res 77:135-148, 1996); yet, the mechanisms for bimanual grasps remain unclear. Similar to unimanual grasping, the localization of matching stable grasp points on an object is computationally expensive and so it makes sense for the signals to converge in a single cortical hemisphere. Indeed, bimanual grasps are faster and more accurate in the left visual field, and are disrupted if there is transcranial stimulation of the right hemisphere (Le and Niemeier in Exp Brain Res 224:263-273, 2013; Le et al. in Cereb Cortex. doi: 10.1093/cercor/bht115, 2013). However, research so far has tested the right hemisphere dominance based on small objects only, which are usually grasped with one hand, whereas bimanual grasping is more commonly used for objects that are too big for a single hand. Because grasping large objects might involve different neural circuits than grasping small objects (Grol et al. in J Neurosci 27:11877-11887, 2007), here we tested whether a left visual field/right hemisphere dominance for bimanual grasping exists with large and thus more ecologically valid objects or whether the right hemisphere dominance is a function of object size. We asked participants to fixate to the left or right of an object and to grasp the object with the index and middle fingers of both hands. Consistent with previous observations, we found that for objects in the left visual field, the maximum grip apertures were scaled closer to the object width and were smaller and less variable, than for objects in the right visual field. Our results demonstrate that bimanual grasping is predominantly controlled by the right hemisphere, even in the context of grasping larger objects.