Anterior regions of monkey parietal cortex process visual 3D shape

Cortico-spinal modularity in the parieto-frontal system: A new perspective on action control

Classical neurophysiology suggests that the motor cortex (MI) has a unique role in action control. In contrast, this review presents evidence for multiple parieto-frontal spinal command modules that can bypass MI. Five observations support this modular perspective: (i) the statistics of cortical connectivity demonstrate functionally-related clusters of cortical areas, defining functional modules in the premotor, cingulate, and parietal cortices; (ii) different corticospinal pathways originate from the above areas, each with a distinct range of conduction velocities; (iii) the activation time of each module varies depending on task, and different modules can be activated simultaneously; (iv) a modular architecture with direct motor output is faster and less metabolically expensive than an architecture that relies on MI, given the slow connections between MI and other cortical areas; (v) lesions of the areas composing parieto-frontal modules have different effects from lesions of MI. Here we provide examples of six cortico-spinal modules and functions they subserve: module 1) arm reaching, tool use and object construction; module 2) spatial navigation and locomotion; module 3) grasping and observation of hand and mouth actions; module 4) action initiation, motor sequences, time encoding; module 5) conditional motor association and learning, action plan switching and action inhibition; module 6) planning defensive actions. These modules can serve as a library of tools to be recombined when faced with novel tasks, and MI might serve as a recombinatory hub. In conclusion, the availability of locally-stored information and multiple outflow paths supports the physiological plausibility of the proposed modular perspective.

No Evidence for Cross-Modal fMRI Adaptation in Macaque Parieto-Premotor Mirror Neuron Regions

To probe the presence of mirror neurons in the human brain, cross-modal fMRI adaptation has been suggested as a suitable technique. The rationale behind this suggestion is that this technique allows making more accurate inferences about neural response properties underlying fMRI voxel activations, beyond merely showing shared voxels that are active during both action observation and execution. However, the validity of using cross-modal fMRI adaptation to demonstrate the presence of mirror neurons in parietal and premotor brain regions has been questioned given the inconsistent and weak results obtained in human studies. A better understanding of cross-modal fMRI adaptation effects in the macaque brain is required as the rationale for using this approach is based on several assumptions related to macaque mirror neuron response properties that still need validation. Here, we conducted a cross-modal fMRI adaptation study in macaque monkeys, using the same action execution and action observation tasks that successfully yielded mirror neuron region cross-modal action decoding in a previous monkey MVPA study. We scanned two male rhesus monkeys while they first executed a sequence of either reach-and-grasp or reach-and-touch hand actions and then observed a video of a human actor performing these motor acts. Both whole-brain and region-of-interest analyses failed to demonstrate cross-modal fMRI adaptation effects in parietal and premotor mirror neuron regions. Our results, in line with previous findings in non-human primates, show that cross-modal motor-to-visual fMRI adaptation is not easily detected in monkey brain regions known to house mirror neurons. Thus, our results advocate caution in using cross-modal fMRI adaptation as a method to infer whether mirror neurons can be found in the primate brain.

Active Vision in Binocular Depth Estimation: A Top-Down Perspective

Depth estimation is an ill-posed problem; objects of different shapes or dimensions, even if at different distances, may project to the same image on the retina. Our brain uses several cues for depth estimation, including monocular cues such as motion parallax and binocular cues such as diplopia. However, it remains unclear how the computations required for depth estimation are implemented in biologically plausible ways. State-of-the-art approaches to depth estimation based on deep neural networks implicitly describe the brain as a hierarchical feature detector. Instead, in this paper we propose an alternative approach that casts depth estimation as a problem of active inference. We show that depth can be inferred by inverting a hierarchical generative model that simultaneously predicts the eyes' projections from a 2D belief over an object. Model inversion consists of a series of biologically plausible homogeneous transformations based on Predictive Coding principles. Under the plausible assumption of a nonuniform fovea resolution, depth estimation favors an active vision strategy that fixates the object with the eyes, rendering the depth belief more accurate. This strategy is not realized by first fixating on a target and then estimating the depth; instead, it combines the two processes through action-perception cycles, with a similar mechanism of the saccades during object recognition. The proposed approach requires only local (top-down and bottom-up) message passing, which can be implemented in biologically plausible neural circuits.

Neuronal Representations Supporting Three-Dimensional Vision in Nonhuman Primates

The visual system must reconstruct the dynamic, three-dimensional (3D) world from ambiguous two-dimensional (2D) retinal images. In this review, we synthesize current literature on how the visual system of nonhuman primates performs this transformation through multiple channels within the classically defined dorsal (where) and ventral (what) pathways. Each of these channels is specialized for processing different 3D features (e.g., the shape, orientation, or motion of objects, or the larger scene structure). Despite the common goal of 3D reconstruction, neurocomputational differences between the channels impose distinct information-limiting constraints on perception. Convergent evidence further points to the little-studied area V3A as a potential branchpoint from which multiple 3D-fugal processing channels diverge. We speculate that the expansion of V3A in humans may have supported the emergence of advanced 3D spatial reasoning skills. Lastly, we discuss future directions for exploring 3D information transmission across brain areas and experimental approaches that can further advance the understanding of 3D vision.

Prehension kinematics in humans and macaques

Non-human primates, especially rhesus macaques, have been a dominant model to study sensorimotor control of the upper limbs. Indeed, human and macaques have similar hands and homologous neural circuits to mediate manual behavior. However, few studies have systematically and quantitatively compared the manual behaviors of the two species. Such comparison is critical for assessing the validity of using the macaque sensorimotor system as a model of its human counterpart. In this study, we systematically compared the prehensile behaviors of humans and rhesus macaques using an identical experimental setup. We found human and macaque prehension kinematics to be generally similar with a few subtle differences. While the structure of the pre-shaping hand postures is similar in humans and macaques, human postures are more object-specific and human joints are less intercorrelated. Conversely, monkeys demonstrate more stereotypical pre-shaping behaviors that are common across all objects and more variability in their postures across repeated presentations of the same object. Despite these subtle differences in manual behavior between humans and monkeys, our results bolster the use of the macaque model to understand the neural mechanisms of manual dexterity in humans.

Two-monkey fMRI setup for investigating multifaceted aspects of social cognition and behavior involving a real-live conspecific

While brain research over the past decades has shed light on the neural correlates of social cognition and behavior in human and non-human primates, most of this research has been performed in virtual settings requiring subjects to observe pictures or recorded videos instead of observing or interacting with another real-live individual. Here we present a two-monkey fMRI setup, allowing examining whole brain responses in macaque monkeys while they observe or interact face-to-face with another real-live conspecific. We tested this setup by comparing overall brain responses during observation of conspecific hand actions in a virtual (observation of recorded videos of actions) or live context (observation of a real-live conspecific performing actions). This dyadic monkey fMRI setup allows examining brain-wide responses in macaque monkeys during different aspects of social behavior, including observation of real-live actions and sensations, social facilitation, joint-attention and social interactions.

The neural mechanisms of manual dexterity

The hand endows us with unparalleled precision and versatility in our interactions with objects, from mundane activities such as grasping to extraordinary ones such as virtuoso pianism. The complex anatomy of the human hand combined with expansive and specialized neuronal control circuits allows a wide range of precise manual behaviours. To support these behaviours, an exquisite sensory apparatus, spanning the modalities of touch and proprioception, conveys detailed and timely information about our interactions with objects and about the objects themselves. The study of manual dexterity provides a unique lens into the sensorimotor mechanisms that endow the nervous system with the ability to flexibly generate complex behaviour.

Common functional localizers to enhance NHP & cross-species neuroscience imaging research

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Modulations of depth responses in the human brain by object context: Does biological relevance matter?

Depth sensitivity has been shown to be modulated by object context (plausibility). It is possible that it is behavioural relevance rather than object plausibility per se which drives this effect. Here, we manipulated the biological relevance of objects (face or a non-face) and tested whether object relevance affects behavioural sensitivity and neural responses to depth-position. In a first experiment, we presented human observers with disparity-defined faces and non-faces, and observers were asked to judge the depth position of the target under signal-noise and clear (fine) task conditions. In the second experiment, we concurrently measured behavioural and fMRI responses to depth. We found that behavioural performance varied across stimulus conditions such that they were significantly worse for the upright face than the inverted face and the random shape in the SNR task, but worse for the random shape than the upright face in the feature task. Pattern analysis of fMRI responses revealed that activity of FFA was distinctly different during depth judgments of the upright face versus the other two stimuli, with its responses (and to a stronger extent, those of V3) appearing functionally-relevant to behavioural performance. We speculate that FFA is not only involved in object analysis, but exerts considerable influence on stereoscopic mechanisms as early as in V3 based on a broader appreciation of the stimulus' behavioural relevance.Significance StatementWe asked how disparity sensitivity is modulated by object (biological) relevance using behavioural and neuroimaging paradigms. We show that behavioural sensitivity to depth-position changes in biological (face) vs non-biological (random surface) contexts, and that these changes are task-dependent. Imaging results highlight a potentially key role of the fusiform region for governing the modulation of stereo encoding by object relevance. These findings highlight powerful interactions between object recognition mechanisms and stereoencoding, such that the utility of disparity information may be up/down weighed depending on the biological relevance of the object.

Unique Neural Activity Patterns Among Lower Order Cortices and Shared Patterns Among Higher Order Cortices During Processing of Similar Shapes With Different Stimulus Types

We investigated the neural mechanism of the processing of three-dimensional (3D) shapes defined by disparity and perspective. We measured blood oxygenation level-dependent signals as participants viewed and classified 3D images of convex-concave shapes. According to the cue (disparity or perspective) and element type (random dots or black and white dotted lines), three types of stimuli were used: random dot stereogram, black and white dotted lines with perspective, and black and white dotted lines with binocular disparity. The blood oxygenation level-dependent images were then classified by multivoxel pattern analysis. To identify areas selective to shape, we assessed convex-concave classification accuracy with classifiers trained and tested using signals evoked by the same stimulus type (same cue and element type). To identify cortical regions with similar neural activity patterns regardless of stimulus type, we assessed the convex-concave classification accuracy of transfer classification in which classifiers were trained and tested using different stimulus types (different cues or element types). Classification accuracy using the same stimulus type was high in the early visual areas and subregions of the intraparietal sulcus (IPS), whereas transfer classification accuracy was high in the dorsal subregions of the IPS. These results indicate that the early visual areas process the specific features of stimuli, whereas the IPS regions perform more generalized processing of 3D shapes, independent of a specific stimulus type.

2D or not 2D? An fMRI study of how dogs visually process objects

Given humans' habitual use of screens, they rarely consider potential differences when viewing two-dimensional (2D) stimuli and real-world versions of dimensional stimuli. Dogs also have access to many forms of screens and touchpads, with owners even subscribing to dog-directed content. Humans understand that 2D stimuli are representations of real-world objects, but do dogs? In canine cognition studies, 2D stimuli are almost always used to study what is normally 3D, like faces, and may assume that both 2D and 3D stimuli are represented in the brain the same way. Here, we used awake fMRI in 15 dogs to examine the neural mechanisms underlying dogs' perception of two- and three-dimensional objects after the dogs were trained on either two- or three-dimensional versions of the objects. Activation within reward processing regions and parietal cortex of the dog brain to 2D and 3D versions of objects was determined by their training experience, as dogs trained on one dimensionality showed greater differential activation within the dimension on which they were trained. These results show that dogs do not automatically generalize between two- and three-dimensional versions of object stimuli and suggest that future research consider the implicit assumptions when using pictures or videos.

Examining cross-modal fMRI adaptation for observed and executed actions in the monkey brain

While mirror neurons have been found in several monkey brain regions, their existence in the human brain is still largely inferred from indirect non-invasive measurements like functional MRI. It has been proposed that, beyond showing overlapping brain responses during action observation and execution tasks, candidate mirror neuron regions should demonstrate cross-modal action specificity, in line with a defining physiological characteristic of these neurons in the monkey brain. Although cross-modal fMRI adaptation has been put forward as a suited method to test this key feature of cross-modal action specificity in the human brain, so far, the overall usefulness of this technique to demonstrate mirror neuron activity remains unclear. To date, it has never been tested to what extent monkey brain regions known to house mirror neurons, would yield uni- and/or cross-modal fMRI adaptation effects. We therefore performed an fMRI adaptation experiment while male rhesus macaques either performed or observed two different goal-directed hand actions. Executing grasp/lift or touch/press actions in the dark, as well as observing videos of these monkey hand actions, yielded robust responses throughout the brain, including overlapping fMRI responses in parietal and premotor mirror neuron regions. Uni-modal adaptation effects were mostly restricted to the visual modality and the early visual cortices. Both frequentist and Bayesian statistical analyses however suggested no evidence for cross-modal fMRI adaptation effects in monkey parietal and premotor mirror neuron regions. Overall, these findings suggest monkey mirror neuron activity does not readily translate into cross-modal repetition suppression effects that can be detected by fMRI.

Sensorimotor Representation Learning for an “Active Self” in Robots: A Model Survey

Safe human-robot interactions require robots to be able to learn how to behave appropriately in spaces populated by people and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots with bio-inspired capacities, in this paper, we first review the developmental processes of underlying mechanisms of these abilities: The sensory representations of body schema, peripersonal space, and the active self in humans. Second, we provide a survey of robotics models of these sensory representations and robotics models of the self; and we compare these models with the human counterparts. Finally, we analyze what is missing from these robotics models and propose a theoretical computational framework, which aims to allow the emergence of the sense of self in artificial agents by developing sensory representations through self-exploration.

Spatial Inhibition of Return Affected by Self-Prioritization Effect in Three-Dimensional Space

Spatial inhibition of return (IOR) being affected by the self-prioritization effect (SPE) in a two-dimensional plane has been well documented. However, it remains unknown how the spatial IOR interacts with the SPE in three-dimensional (3D) space. By constructing a virtual 3D environment, Posner's classically two-dimensional cue-target paradigm was applied to a 3D space. Participants first associated labels for themselves, their best friends, and strangers with geometric shapes in a shape-label matching task, then performed Experiment 1 (referential information appeared as the cue label) and Experiment 2 (referential information appeared as the target label) to investigate whether the IOR effect could be influenced by the SPE in 3D space. This study showed that when the cue was temporarily established with a self-referential shape and appeared in far space, the IOR effect was the smallest. When the target was temporarily established with a self-referential shape and appeared in near space, the IOR effect disappeared. This study suggests that the IOR effect was affected by the SPE when attention was oriented or reoriented in 3D space and that the IOR effect disappeared or decreased when affected by the SPE in 3D space.

Whole brain mapping of somatosensory responses in awake marmosets investigated with ultra-high-field fMRI

The common marmoset (Callithrix jacchus) is a small-bodied New World primate that is becoming an important model to study brain functions. Despite several studies exploring the somatosensory system of marmosets, all results have come from anesthetized animals using invasive techniques and postmortem analyses. Here, we demonstrate the feasibility for getting high-quality and reproducible somatosensory mapping in awake marmosets with functional magnetic resonance imaging (fMRI). We acquired fMRI sequences in four animals, while they received tactile stimulation (via air-puffs), delivered to the face, arm, or leg. We found a topographic body representation with the leg representation in the most medial part, the face representation in the most lateral part, and the arm representation between leg and face representation within areas 3a, 3b, and 1/2. A similar sequence from leg to face from caudal to rostral sites was identified in areas S2 and PV. By generating functional connectivity maps of seeds defined in the primary and second somatosensory regions, we identified two clusters of tactile representation within the posterior and midcingulate cortex. However, unlike humans and macaques, no clear somatotopic maps were observed. At the subcortical level, we found a somatotopic body representation in the thalamus and, for the first time in marmosets, in the putamen. These maps have similar organizations, as those previously found in Old World macaque monkeys and humans, suggesting that these subcortical somatotopic organizations were already established before Old and New World primates diverged. Our results show the first whole brain mapping of somatosensory responses acquired in a noninvasive way in awake marmosets.NEW & NOTEWORTHY We used somatosensory stimulation combined with functional MRI (fMRI) in awake marmosets to reveal the topographic body representation in areas S1, S2, thalamus, and putamen. We showed the existence of a body representation organization within the thalamus and the cingulate cortex by computing functional connectivity maps from seeds defined in S1/S2, using resting-state fMRI data. This noninvasive approach will be essential for chronic studies by guiding invasive recording and manipulation techniques.

Wide-field retinotopy reveals a new visuotopic cluster in macaque posterior parietal cortex

We investigated the visuotopic organization of macaque posterior parietal cortex (PPC) by combining functional imaging (fMRI) and wide-field retinotopic mapping in two macaque monkeys. Whole brain blood-oxygen-level-dependent (BOLD) signal was recorded while monkeys maintained central fixation during the presentation of large rotating wedges and expending/contracting annulus of a "shaking" fruit basket, designed to maximize the recruitment of PPC neurons. Results of the surface-based population receptive field (pRF) analysis reveal a new cluster of four visuotopic areas at the confluence of the parieto-occipital and intra-parietal sulci, in a location previously defined histologically and anatomically as the posterior intra-parietal (PIP) region. This PIP cluster groups together two recently described areas (CIP1/2) laterally and two newly identified ones (PIP1/2) medially, whose foveal representations merge in the fundus of the intra-parietal sulcus. The cluster shares borders with other visuotopic areas: V3d posteriorly, V3A/DP laterally, V6/V6A medially and LIP anteriorly. Together, these results show that monkey PPC is endowed with a dense set of visuotopic areas, as its human counterpart. The fact that fMRI and wide-field stimulation allows a functional parsing of monkey PPC offers a new framework for studying functional homologies with human PPC.

Effective Connectivity Reveals an Interconnected Inferotemporal Network for Three-Dimensional Structure Processing

Disparity-defined 3D shape is processed in both the ventral and the dorsal visual stream. The network of cortical areas that is activated during the processing of disparity-defined 3D shape includes, in addition to parietal and premotor areas, three clearly distinct regions in inferotemporal cortex (ITC). To investigate the connectivity of the latter regions, we combined electrical stimulation with fMRI in male macaque monkeys. Electrical stimulation of each of the 3D-structure nodes in ITC mainly elicited increased fMRI activations in the other 3D-structure nodes and more variably in other parts of ventral visual cortex. Importantly, no increased activation was found in parietal areas, nor in PFC, whereas microstimulation in posterior parietal cortex did activate the ITC. Our results indicate that 3D-structure nodes in ITC form a strongly interconnected network, receiving input from parietal areas implicated in 3D-structure processing.SIGNIFICANCE STATEMENT Previous studies combining electrical microstimulation with functional imaging showed an interconnected set of regions in the ventral stream processing faces or bodies, but is has been unclear whether the same is true for other visual categories. Here the authors show that there is a connected system of stereo-selective regions in inferotemporal cortex, receiving input from parietal areas in the dorsal stream.

Motor resonance in monkey parietal and premotor cortex during action observation: Influence of viewing perspective and effector identity

Observing others performing motor acts like grasping has been shown to elicit neural responses in the observer`s parieto-frontal motor network, which typically becomes active when the observer would perform these actions him/herself. While some human studies suggested strongest motor resonance during observation of first person or egocentric perspectives compared to third person or allocentric perspectives, other research either report the opposite or did not find any viewpoint-related preferences in parieto-premotor cortices. Furthermore, it has been suggested that these motor resonance effects are lateralized in the parietal cortex depending on the viewpoint and identity of the observed effector (left vs right hand). Other studies, however, do not find such straightforward hand identity dependent motor resonance effects. In addition to these conflicting findings in human studies, to date, little is known about the modulatory role of viewing perspective and effector identity (left or right hand) on motor resonance effects in monkey parieto-premotor cortices. Here, we investigated the extent to which different viewpoints of observed conspecific hand actions yield motor resonance in rhesus monkeys using fMRI. Observing first person, lateral and third person viewpoints of conspecific hand actions yielded significant activations throughout the so-called action observation network, including STS, parietal and frontal cortices. Although region-of-interest analysis of parietal and premotor motor/mirror neuron regions AIP, PFG and F5, showed robust responses in these regions during action observation in general, a clear preference for egocentric or allocentric perspectives was not evident. Moreover, except for lateralized effects due to visual field biases, motor resonance in the monkey brain during grasping observation did not reflect hand identity dependent coding.

Fast Compensatory Functional Network Changes Caused by Reversible Inactivation of Monkey Parietal Cortex

The brain has a remarkable capacity to recover after lesions. However, little is known about compensatory neural adaptations at the systems level. We addressed this question by investigating behavioral and (correlated) functional changes throughout the cortex that are induced by focal, reversible inactivations. Specifically, monkeys performed a demanding covert spatial attention task while the lateral intraparietal area (LIP) was inactivated with muscimol and whole-brain fMRI activity was recorded. The inactivation caused LIP-specific decreases in task-related fMRI activity. In addition, these local effects triggered large-scale network changes. Unlike most studies in which animals were mainly passive relative to the stimuli, we observed heterogeneous effects with more profound muscimol-induced increases of task-related fMRI activity in areas connected to LIP, especially FEF. Furthermore, in areas such as FEF and V4, muscimol-induced changes in fMRI activity correlated with changes in behavioral performance. Notably, the activity changes in remote areas did not correlate with the decreased activity at the site of the inactivation, suggesting that such changes arise via neuronal mechanisms lying in the intact portion of the functional task network, with FEF a likely key player. The excitation-inhibition dynamics unmasking existing excitatory connections across the functional network might initiate these rapid adaptive changes.

Anterior Intraparietal Area: A Hub in the Observed Manipulative Action Network

Current knowledge regarding the processing of observed manipulative actions (OMAs) (e.g., grasping, dragging, or dropping) is limited to grasping and underlying neural circuitry remains controversial. Here, we addressed these issues by combining chronic neuronal recordings along the anteroposterior extent of monkeys’ anterior intraparietal (AIP) area with tracer injections into the recorded sites. We found robust neural selectivity for 7 distinct OMAs, particularly in the posterior part of AIP (pAIP), where it was associated with motor coding of grip type and own-hand visual feedback. This cluster of functional properties appears to be specifically grounded in stronger direct connections of pAIP with the temporal regions of the ventral visual stream and the prefrontal cortex, as connections with skeletomotor related areas and regions of the dorsal visual stream exhibited opposite or no rostrocaudal gradients. Temporal and prefrontal areas may provide visual and contextual information relevant for manipulative action processing. These results revise existing models of the action observation network, suggesting that pAIP constitutes a parietal hub for routing information about OMA identity to the other nodes of the network.

Stereomotion Processing in the Nonhuman Primate Brain

The cortical areas that process disparity-defined motion-in-depth (i.e., cyclopean stereomotion [CSM]) were characterized with functional magnetic resonance imaging (fMRI) in two awake, behaving macaques. The experimental protocol was similar to previous human neuroimaging studies. We contrasted the responses to dynamic random-dot patterns that continuously changed their binocular disparity over time with those to a control condition that shared the same properties, except that the temporal frames were shuffled. A whole-brain voxel-wise analysis revealed that in all four cortical hemispheres, three areas showed consistent sensitivity to CSM. Two of them were localized respectively in the lower bank of the superior temporal sulcus (CSMSTS) and on the neighboring infero-temporal gyrus (CSMITG). The third area was situated in the posterior parietal cortex (CSMPPC). Additional regions of interest-based analyses within retinotopic areas defined in both animals indicated weaker but significant responses to CSM within the MT cluster (most notably in areas MSTv and FST). Altogether, our results are in agreement with previous findings in both human and macaque and suggest that the cortical areas that process CSM are relatively well preserved between the two primate species.

Single-cell selectivity and functional architecture of human lateral occipital complex

The human lateral occipital complex (LOC) is more strongly activated by images of objects compared to scrambled controls, but detailed information at the neuronal level is currently lacking. We recorded with microelectrode arrays in the LOC of 2 patients and obtained highly selective single-unit, multi-unit, and high-gamma responses to images of objects. Contrary to predictions derived from functional imaging studies, all neuronal properties indicated that the posterior subsector of LOC we recorded from occupies an unexpectedly high position in the hierarchy of visual areas. Notably, the response latencies of LOC neurons were long, the shape selectivity was spatially clustered, LOC receptive fields (RFs) were large and bilateral, and a number of LOC neurons exhibited three-dimensional (3D)-structure selectivity (a preference for convex or concave stimuli), which are all properties typical of end-stage ventral stream areas. Thus, our results challenge prevailing ideas about the position of the more posterior subsector of LOC in the hierarchy of visual areas.

Representation of shape, space, and attention in monkey cortex

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both 'what' and 'where' cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.

Identifying Cortical Substrates Underlying the Phenomenology of Stereopsis and Realness: A Pilot fMRI Study

Viewing a real scene or a stereoscopic image (e.g., 3D movies) with both eyes yields a vivid subjective impression of object solidity, tangibility, immersive negative space and sense of realness; something that is not experienced when viewing single pictures of 3D scenes normally with both eyes. This phenomenology, sometimes referred to as stereopsis, is conventionally ascribed to the derivation of depth from the differences in the two eye's images (binocular disparity). Here we report on a pilot study designed to explore if dissociable neural activity associated with the phenomenology of realness can be localized in the cortex. In order to dissociate subjective impression from disparity processing, we capitalized on the finding that the impression of realness associated with stereoscopic viewing can also be generated when viewing a single picture of a 3D scene with one eye through an aperture. Under a blocked fMRI design, subjects viewed intact and scrambled images of natural 3-D objects, and scenes under three viewing conditions: (1) single pictures viewed normally with both eyes (binocular); (2) single pictures viewed with one eye through an aperture (monocular-aperture); and (3) stereoscopic anaglyph images of the same scenes viewed with both eyes (binocular stereopsis). Fixed-effects GLM contrasts aimed at isolating the phenomenology of stereopsis demonstrated a selective recruitment of similar posterior parietal regions for both monocular and binocular stereopsis conditions. Our findings provide preliminary evidence that the cortical processing underlying the subjective impression of realness may be dissociable and distinct from the derivation of depth from disparity.

Decoding Grasping Movements from the Parieto-Frontal Reaching Circuit in the Nonhuman Primate

Prehension movements typically include a reaching phase, guiding the hand toward the object, and a grip phase, shaping the hand around it. The dominant view posits that these components rely upon largely independent parieto-frontal circuits: a dorso-medial circuit involved in reaching and a dorso-lateral circuit involved in grasping. However, mounting evidence suggests a more complex arrangement, with dorso-medial areas contributing to both reaching and grasping. To investigate the role of the dorso-medial reaching circuit in grasping, we trained monkeys to reach-and-grasp different objects in the dark and determined if hand configurations could be decoded from functional magnetic resonance imaging (MRI) responses obtained from the reaching and grasping circuits. Indicative of their established role in grasping, object-specific grasp decoding was found in anterior intraparietal (AIP) area, inferior parietal lobule area PFG and ventral premotor region F5 of the lateral grasping circuit, and primary motor cortex. Importantly, the medial reaching circuit also conveyed robust grasp-specific information, as evidenced by significant decoding in parietal reach regions (particular V6A) and dorsal premotor region F2. These data support the proposed role of dorso-medial "reach" regions in controlling aspects of grasping and demonstrate the value of complementing univariate with more sensitive multivariate analyses of functional MRI (fMRI) data in uncovering information coding in the brain.

Dissociating neural activity associated with the subjective phenomenology of monocular stereopsis: An EEG study

The subjective phenomenology associated with stereopsis, of solid tangible objects separated by a palpable negative space, is conventionally thought to be a by-product of the derivation of depth from binocular disparity. However, the same qualitative impression has been reported in the absence of disparity, e.g., when viewing pictorial images monocularly through an aperture. Here we aimed to explore if we could identify dissociable neural activity associated with the qualitative impression of stereopsis in the absence of the processing of binocular disparities. We measured EEG activity while subjects viewed pictorial (non-stereoscopic) images of 2D and 3D geometric forms under four different viewing conditions (binocular, monocular, binocular aperture, monocular aperture). EEG activity was analysed by oscillatory source localization (beamformer technique) to examine power change in occipital and parietal regions across viewing and stimulus conditions in targeted frequency bands (alpha: 8-13 Hz & gamma: 60-90 Hz). We observed expected event-related gamma synchronization and alpha desynchronization in occipital cortex and predominant gamma synchronization in parietal cortex across viewing and stimulus conditions. However, only the viewing condition predicted to generate the strongest impression of stereopsis (monocular aperture) revealed significantly elevated gamma synchronization within the parietal cortex for the critical contrasts (3D vs. 2D form). These findings suggest dissociable neural processes specific to the qualitative impression of stereopsis as distinguished from disparity processing.

Areal differences in depth cue integration between monkey and human

Electrophysiological evidence suggested primarily the involvement of the middle temporal (MT) area in depth cue integration in macaques, as opposed to human imaging data pinpointing area V3B/kinetic occipital area (V3B/KO). To clarify this conundrum, we decoded monkey functional MRI (fMRI) responses evoked by stimuli signaling near or far depths defined by binocular disparity, relative motion, and their combination, and we compared results with those from an identical experiment previously performed in humans. Responses in macaque area MT are more discriminable when two cues concurrently signal depth, and information provided by one cue is diagnostic of depth indicated by the other. This suggests that monkey area MT computes fusion of disparity and motion depth signals, exactly as shown for human area V3B/KO. Hence, these data reconcile previously reported discrepancies between depth processing in human and monkey by showing the involvement of the dorsal stream in depth cue integration using the same technique, despite the engagement of different regions.

In everyday life, we interact with a three-dimensional world that we perceive via our two-dimensional retinas. Our brain can reconstruct the third dimension from these flat retinal images using multiple sources of visual information, or cues. The horizontal displacement of the two retinal images, known as binocular disparity, and the relative motion between different objects are two important depth cues. However, to make the most of the information provided by each cue, our brains must efficiently integrate across them. To examine this process, we used neuroimaging in monkeys to record brain responses evoked by stimuli signaling depths defined by either binocular disparity or relative motion in isolation, and also when the two cues are combined congruently or incongruently. We found that cortical area MT in monkeys is involved in the fusion of these two particular depth cues, in contrast to previous human imaging data that pinpoint a more posterior cortical area, V3B/KO. Our findings support the existence of depth cue integration mechanisms in primates; however, this fusion appears to be computed in slightly different areas in humans and monkeys.

Choice-Related Activity during Visual Slant Discrimination in Macaque CIP But Not V3A

Creating three-dimensional (3D) representations of the world from two-dimensional retinal images is fundamental to visually guided behaviors including reaching and grasping. A critical component of this process is determining the 3D orientation of objects. Previous studies have shown that neurons in the caudal intraparietal area (CIP) of the macaque monkey represent 3D planar surface orientation (i.e., slant and tilt). Here we compare the responses of neurons in areas V3A (which is implicated in 3D visual processing and precedes CIP in the visual hierarchy) and CIP to 3D-oriented planar surfaces. We then examine whether activity in these areas correlates with perception during a fine slant discrimination task in which the monkeys report if the top of a surface is slanted toward or away from them. Although we find that V3A and CIP neurons show similar sensitivity to planar surface orientation, significant choice-related activity during the slant discrimination task is rare in V3A but prominent in CIP. These results implicate both V3A and CIP in the representation of 3D surface orientation, and suggest a functional dissociation between the areas based on slant-related choice signals.

Adaptation reveals multi-stage coding of visual duration

In conflict with historically dominant models of time perception, recent evidence suggests that the encoding of our environment's temporal properties may not require a separate class of neurons whose raison d'être is the dedicated processing of temporal information. If true, it follows that temporal processing should be imbued with the known selectivity found within non-temporal neurons. In the current study, we tested this hypothesis for the processing of a poorly understood stimulus parameter: visual event duration. We used sensory adaptation techniques to generate duration aftereffects: bidirectional distortions of perceived duration. Presenting adapting and test durations to the same vs different eyes utilises the visual system's anatomical progression from monocular, pre-cortical neurons to their binocular, cortical counterparts. Duration aftereffects exhibited robust inter-ocular transfer alongside a small but significant contribution from monocular mechanisms. We then used novel stimuli which provided duration information that was invisible to monocular neurons. These stimuli generated robust duration aftereffects which showed partial selectivity for adapt-test changes in retinal disparity. Our findings reveal distinct duration encoding mechanisms at monocular, depth-selective and depth-invariant stages of the visual hierarchy.

Disparity level identification using the voxel-wise Gabor model of fMRI data

Perceiving disparities is the intuitive basis for our understanding of the physical world. Although many electrophysiology studies have revealed the disparity-tuning characteristics of the neurons in the visual areas of the macaque brain, neuron population responses to disparity processing have seldom been investigated. Many disparity studies using functional magnetic resonance imaging (fMRI) have revealed the disparity-selective visual areas in the human brain. However, it is unclear how to characterize neuron population disparity-tuning responses using fMRI technique. In the present study, we constructed three voxel-wise encoding Gabor models to predict the voxel responses to novel disparity levels and used a decoding method to identify the new disparity levels from population responses in the cortex. Among the three encoding models, the fine-coarse model (FCM) that used fine/coarse disparities to fit the voxel responses to disparities outperformed the single model and uncrossed-crossed model. Moreover, the FCM demonstrated high accuracy in predicting voxel responses in V3A complex and high accuracy in identifying novel disparities from responses in V3A complex. Our results suggest that the FCM can better characterize the voxel responses to disparities than the other two models and V3A complex is a critical visual area for representing disparity information.

Functional specialization of macaque premotor F5 subfields with respect to hand and mouth movements: A comparison of task and resting-state fMRI

Based on architectonic, tract-tracing or functional criteria, the rostral portion of ventral premotor cortex in the macaque monkey, also termed area F5, has been divided into several subfields. Cytoarchitectonical investigations suggest the existence of three subfields, F5c (convexity), F5p (posterior) and F5a (anterior). Electrophysiological investigations have suggested a gradual dorso-ventral transition from hand- to mouth-dominated motor fields, with F5p and ventral F5c strictly related to hand movements and mouth movements, respectively. The involvement of F5a in this respect, however, has received much less attention. Recently, data-driven resting-state fMRI approaches have also been used to examine the presence of distinct functional fields in macaque ventral premotor cortex. Although these studies have suggested several functional clusters in/near macaque F5, so far the parcellation schemes derived from these clustering methods do not completely retrieve the same level of F5 specialization as suggested by aforementioned invasive techniques. Here, using seed-based resting-state fMRI analyses, we examined the functional connectivity of different F5 seeds with key regions of the hand and face/mouth parieto-frontal-insular motor networks. In addition, we trained monkeys to perform either hand grasping or ingestive mouth movements in the scanner in order to compare resting-state with task-derived functional hand and mouth motor networks. In line with previous single-cell investigations, task-fMRI suggests involvement of F5p, dorsal F5c and F5a in the execution of hand grasping movements, while non-communicative mouth movements yielded particularly pronounced responses in ventral F5c. Corroborating with anatomical tracing data of macaque F5 subfields, seed-based resting-state fMRI suggests a transition from predominant functional correlations with the hand-motor network in F5p to mostly mouth-motor network functional correlations in ventral F5c. Dorsal F5c yielded robust functional correlations with both hand- and mouth-motor networks. In addition, the deepest part of the fundus of the inferior arcuate, corresponding to area 44, displayed a strikingly different functional connectivity profile compared to neighboring F5a, suggesting a different functional specialization for these two neighboring regions.

Brain mechanisms of visual long-term memory retrieval in primates

Memorizing events or objects and retrieving them from memory are essential for daily life. Historically, memory processing was studied in neuropsychology, in which patients provided us with insights into the brain mechanisms underlying memory. Psychological hypotheses about memory processing have been further investigated using neuroscience techniques, such as functional imaging and electrophysiology. In this article, I briefly summarize recent findings on multi-scale neural circuitry for memory at the scale of single neurons and cortical layers as well as inter-area and whole-brain interactions. The key idea which connects multi-scale neural circuits is how neuronal assemblies utilize the frequency of communication between neurons, cortical layers, and brain areas. Using findings and ideas from other cognitive function studies, I discuss the plausible communication between neurons involved in memory.

Investigating common coding of observed and executed actions in the monkey brain using cross-modal multi-variate fMRI classification

Mirror neurons are generally described as a neural substrate hosting shared representations of actions, by simulating or 'mirroring' the actions of others onto the observer's own motor system. Since single neuron recordings are rarely feasible in humans, it has been argued that cross-modal multi-variate pattern analysis (MVPA) of non-invasive fMRI data is a suitable technique to investigate common coding of observed and executed actions, allowing researchers to infer the presence of mirror neurons in the human brain. In an effort to close the gap between monkey electrophysiology and human fMRI data with respect to the mirror neuron system, here we tested this proposal for the first time in the monkey. Rhesus monkeys either performed reach-and-grasp or reach-and-touch motor acts with their right hand in the dark or observed videos of human actors performing similar motor acts. Unimodal decoding showed that both executed or observed motor acts could be decoded from numerous brain regions. Specific portions of rostral parietal, premotor and motor cortices, previously shown to house mirror neurons, in addition to somatosensory regions, yielded significant asymmetric action-specific cross-modal decoding. These results validate the use of cross-modal multi-variate fMRI analyses to probe the representations of own and others' actions in the primate brain and support the proposed mapping of others' actions onto the observer's own motor cortices.

Towards a unified perspective of object shape and motion processing in human dorsal cortex

Although object-related areas were discovered in human parietal cortex a decade ago, surprisingly little is known about the nature and purpose of these representations, and how they differ from those in the ventral processing stream. In this article, we review evidence for the unique contribution of object areas of dorsal cortex to three-dimensional (3-D) shape representation, the localization of objects in space, and in guiding reaching and grasping actions. We also highlight the role of dorsal cortex in form-motion interaction and spatiotemporal integration, possible functional relationships between 3-D shape and motion processing, and how these processes operate together in the service of supporting goal-directed actions with objects. Fundamental differences between the nature of object representations in the dorsal versus ventral processing streams are considered, with an emphasis on how and why dorsal cortex supports veridical (rather than invariant) representations of objects to guide goal-directed hand actions in dynamic visual environments.

More than Action: The Dorsal Pathway Contributes to the Perception of 3-D Structure

An evolving view in cognitive neuroscience is that the dorsal visual pathway not only plays a key role in visuomotor behavior but that it also contributes functionally to the recognition of objects. To characterize the nature of the object representations derived by the dorsal pathway, we assessed perceptual performance in the context of the continuous flash suppression paradigm, which suppresses object processing in the ventral pathway while sparing computation in the dorsal pathway. In a series of experiments, prime stimuli, which were rendered imperceptible by the continuous flash suppression, still contributed to perceptual decisions related to the subsequent perceptible target stimuli. However, the contribution of the prime to perception was contingent on the prime's structural coherence, in that a perceptual advantage was observed only for targets primed by objects with legitimate 3-D structure. Finally, we obtained additional evidence to demonstrate that the processing of the suppressed objects was contingent on the magnocellular, rather than the parvocellular, system, further linking the processing of the suppressed stimuli to the dorsal pathway. Together, these results provide novel evidence that the dorsal pathway does not only support visuomotor control but, rather, that it also derives the structural description of 3-D objects and contributes to shape perception.

Thresholds for sine-wave corrugations defined by binocular disparity in random dot stereograms: Factor analysis of individual differences reveals two stereoscopic mechanisms tuned for spatial frequency

Threshold functions for sinusoidal depth corrugations typically reach their minimum (highest sensitivity) at spatial frequencies of 0.2-0.4 cycles/degree (cpd), with lower thresholds for horizontal than vertical corrugations at low spatial frequencies. To elucidate spatial frequency and orientation tuning of stereoscopic mechanisms, we measured the disparity sensitivity functions, and used factor analytic techniques to estimate the existence of independent underlying stereo channels. The data set (N = 30 individuals) was for horizontal and vertical corrugations of spatial frequencies ranging from 0.1 to 1.6 cpd. A principal component analysis of disparity sensitivities (log-arcsec) revealed that two significant factors accounted for 70% of the variability. Following Varimax rotation to approximate "simple structure", one factor clearly loaded onto low spatial frequencies (≤0.4 cpd), and a second was tuned to higher spatial frequencies (≥0.8 cpd). Each factor had nearly identical tuning (loadings) for horizontal and vertical patterns. The finding of separate factors for low and high spatial frequencies is consistent with previous studies. The failure to find separate factors for horizontal and vertical corrugations is somewhat surprising because the neuronal mechanisms are believed to be different. Following an oblique rotation (Direct Oblimin), the two factors correlated significantly, suggesting some interdependence rather than full independence between the two factors.

The large-scale organization of shape processing in the ventral and dorsal pathways

Although shape perception is considered a function of the ventral visual pathway, evidence suggests that the dorsal pathway also derives shape-based representations. In two psychophysics and neuroimaging experiments, we characterized the response properties, topographical organization and perceptual relevance of these representations. In both pathways, shape sensitivity increased from early visual cortex to extrastriate cortex but then decreased in anterior regions. Moreover, the lateral aspect of the ventral pathway and posterior regions of the dorsal pathway were sensitive to the availability of fundamental shape properties, even for unrecognizable images. This apparent representational similarity between the posterior-dorsal and lateral-ventral regions was corroborated by a multivariate analysis. Finally, as with ventral pathway, the activation profile of posterior dorsal regions was correlated with recognition performance, suggesting a possible contribution to perception. These findings challenge a strict functional dichotomy between the pathways and suggest a more distributed model of shape processing.

We rely on our sense of vision to perceive the world around us and the objects within it. We also use vision to guide our interactions with objects. One of the most influential theories in cognitive neuroscience is the idea that separate pathways within the brain support these two processes. The ventral pathway is in charge of vision-for-perception. It analyses the features that help us recognize objects, such as their color, size or shape, enabling us to identify the hammer in a toolbox, for example. The dorsal pathway is responsible for vision-for-action. It processes features that help us interact with objects, such as their movement and location, enabling us to use the hammer to strike a nail.

However, recent studies have suggested that the ventral and dorsal pathways may not be as independent as originally thought. Freud et al. now test this idea by examining if the dorsal vision-for-action pathway can also perceive and process objects.

Healthy volunteers viewed pictures of objects while lying inside a brain scanner. Some of the objects in the pictures were intact, whereas others had been distorted. If a brain region shows greater activation when viewing intact objects than distorted ones, it implies that that region is sensitive to the normal shapes of objects. Freud et al. found that both the ventral and dorsal pathways were sensitive to shape, with some areas in the two pathways showing highly similar responses. Furthermore, the shape sensitivity of certain regions within the dorsal pathway correlated with the volunteers’ ability to recognize the objects. This suggests that regions distributed across both pathways – and not just the ventral one – may contribute to object recognition.

The two-pathways hypothesis has governed our understanding of vision and of other sensory systems including hearing for several decades. By challenging the binary distinction between the two pathways, the results of Freud et al. suggest that models of sensory processing may require updating. This improved understanding may ultimately improve diagnosis and treatment of perceptual disorders such as agnosia, in which patients struggle to recognize objects.

Connectome-scale functional intrinsic connectivity networks in macaques

There have been extensive studies of intrinsic connectivity networks (ICNs) in the human brains using resting-state functional magnetic resonance imaging (fMRI) in the literature. However, the functional organization of ICNs in macaque brains has been less explored so far, despite growing interests in the field. In this work, we propose a computational framework to identify connectome-scale group-wise consistent ICNs in macaques via sparse representation of whole-brain resting-state fMRI data. Experimental results demonstrate that 70 group-wise consistent ICNs are successfully identified in macaque brains via the proposed framework. These 70 ICNs are interpreted based on two publicly available parcellation maps of macaque brains and our work significantly expand currently known macaque ICNs already reported in the literature. In general, this set of connectome-scale group-wise consistent ICNs can potentially benefit a variety of studies in the neuroscience and brain-mapping fields, and they provide a foundation to better understand brain evolution in the future.

Parkinson's patients can rely on perspective cues to perceive 3D space

3D perception, which is necessary for an optimal navigation in our environment, relies on 2 complementary kinds of cues; binocular cues allowing precise depth localization near the point of visual interest and monocular ones that are necessary for correct global perception of visual space. Recent studies described deficient binocular 3D vision in PD patients; here we tested 3D vision in PD patients when based on monocular cues (m3D). Sixteen PD patients and 16 controls had to categorize visual stimuli as perceived in 2D (flat) or 3D (with depth). Both performance and response times were measured. EEGs were recorded to extract Visual Evoked Potentials. Effects of PD were tested by comparing psychometric and electrophysiological data obtained in controls and PD patients evaluated without dopaminergic treatment. Effects of Levodopa were tested by comparing data in PD patients with and without dopaminergic treatment. We didn't find statistical differences between PD patients and controls' performance. Severity of PD (UPDRS III) in OFF condition is positively correlated with P1 amplitudes and latencies for both 2D and m3D stimuli. Levodopa administration didn't modify either PD patients' performances although it increases principal visual components latencies for both 2D and m3D stimuli. Unlike binocular 3D vision, monocular 3D vision does not seem to get affected by PD. However given the correlation between severity of PD and VEPs' modifications, alteration of visual cortical processing might have nonetheless begun. PD patients reporting trouble in perceiving space must rely more on m3D cues present in the environment.

A dataset of stereoscopic images and ground-truth disparity mimicking human fixations in peripersonal space

Binocular stereopsis is the ability of a visual system, belonging to a live being or a machine, to interpret the different visual information deriving from two eyes/cameras for depth perception. From this perspective, the ground-truth information about three-dimensional visual space, which is hardly available, is an ideal tool both for evaluating human performance and for benchmarking machine vision algorithms. In the present work, we implemented a rendering methodology in which the camera pose mimics realistic eye pose for a fixating observer, thus including convergent eye geometry and cyclotorsion. The virtual environment we developed relies on highly accurate 3D virtual models, and its full controllability allows us to obtain the stereoscopic pairs together with the ground-truth depth and camera pose information. We thus created a stereoscopic dataset: GENUA PESTO-GENoa hUman Active fixation database: PEripersonal space STereoscopic images and grOund truth disparity. The dataset aims to provide a unified framework useful for a number of problems relevant to human and computer vision, from scene exploration and eye movement studies to 3D scene reconstruction.

Functional interactions between the macaque dorsal and ventral visual pathways during three-dimensional object vision

The division of labor between the dorsal and the ventral visual stream in the primate brain has inspired numerous studies on the visual system in humans and in nonhuman primates. However, how and under which circumstances the two visual streams interact is still poorly understood. Here we review evidence from anatomy, modelling, electrophysiology, electrical microstimulation (EM), reversible inactivation and functional imaging in the macaque monkey aimed at clarifying at which levels in the hierarchy of visual areas the two streams interact, and what type of information might be exchanged between the two streams during three-dimensional (3D) object viewing. Neurons in both streams encode 3D structure from binocular disparity, synchronized activity between parietal and inferotemporal areas is present during 3D structure categorization, and clusters of 3D structure-selective neurons in parietal cortex are anatomically connected to ventral stream areas. In addition, caudal intraparietal cortex exerts a causal influence on 3D-structure related activations in more anterior parietal cortex and in inferotemporal cortex. Thus, both anatomical and functional evidence indicates that the dorsal and the ventral visual stream interact during 3D object viewing.

A brief comparative review of primate posterior parietal cortex: A novel hypothesis on the human toolmaker

The primate visual system contains two major cortical pathways: a ventral-temporal pathway that has been associated with object processing and recognition, and a dorsal-parietal pathway that has been associated with spatial processing and action guidance. Our understanding of the role of the dorsal pathway, in particular, has greatly evolved within the framework of the two-pathway hypothesis since its original conception. Here, we present a comparative review of the primate dorsal pathway in humans and monkeys based on electrophysiological, neuroimaging, neuropsychological, and neuroanatomical studies. We consider similarities and differences across species in terms of the topographic representation of visual space; specificity for eye, reaching, or grasping movements; multi-modal response properties; and the representation of objects and tools. We also review the relative anatomical location of functionally- and topographically-defined regions of the posterior parietal cortex. An emerging theme from this comparative analysis is that non-spatial information is represented to a greater degree, and with increased complexity, in the human dorsal visual system. We propose that non-spatial information in the primate parietal cortex contributes to the perception-to-action system aimed at manipulating objects in peripersonal space. In humans, this network has expanded in multiple ways, including the development of a dorsal object vision system mirroring the complexity of the ventral stream, the integration of object information with parietal working memory systems, and the emergence of tool-specific object representations in the anterior intraparietal sulcus and regions of the inferior parietal lobe. We propose that these evolutionary changes have enabled the emergence of human-specific behaviors, such as the sophisticated use of tools.

Two Visual Pathways in Primates Based on Sampling of Space: Exploitation and Exploration of Visual Information

Evidence is strong that the visual pathway is segregated into two distinct streams—ventral and dorsal. Two proposals theorize that the pathways are segregated in function: The ventral stream processes information about object identity, whereas the dorsal stream, according to one model, processes information about either object location, and according to another, is responsible in executing movements under visual control. The models are influential; however recent experimental evidence challenges them, e.g., the ventral stream is not solely responsible for object recognition; conversely, its function is not strictly limited to object vision; the dorsal stream is not responsible by itself for spatial vision or visuomotor control; conversely, its function extends beyond vision or visuomotor control. In their place, we suggest a robust dichotomy consisting of a ventral stream selectively sampling high-resolution/focal spaces, and a dorsal stream sampling nearly all of space with reduced foveal bias. The proposal hews closely to the theme of embodied cognition: Function arises as a consequence of an extant sensory underpinning. A continuous, not sharp, segregation based on function emerges, and carries with it an undercurrent of an exploitation-exploration dichotomy. Under this interpretation, cells of the ventral stream, which individually have more punctate receptive fields that generally include the fovea or parafovea, provide detailed information about object shapes and features and lead to the systematic exploitation of said information; cells of the dorsal stream, which individually have large receptive fields, contribute to visuospatial perception, provide information about the presence/absence of salient objects and their locations for novel exploration and subsequent exploitation by the ventral stream or, under certain conditions, the dorsal stream. We leverage the dichotomy to unify neuropsychological cases under a common umbrella, account for the increased prevalence of multisensory integration in the dorsal stream under a Bayesian framework, predict conditions under which object recognition utilizes the ventral or dorsal stream, and explain why cells of the dorsal stream drive sensorimotor control and motion processing and have poorer feature selectivity. Finally, the model speculates on a dynamic interaction between the two streams that underscores a unified, seamless perception. Existing theories are subsumed under our proposal.

Stereopsis after anterior temporal lobectomy

Brain areas critical for stereopsis have been investigated in non-human primates but are largely unknown in the human brain. Microelectrode recordings and functional MRI (fMRI) studies in monkeys have shown that in monkeys the inferior temporal cortex is critically involved in 3D shape categorization. Furthermore, some human fMRI studies similarly suggest an involvement of visual areas in the temporal lobe in depth perception. We aimed to investigate the role of the human anterior temporal neocortex in stereopsis by assessing stereoscopic depth perception before and after anterior temporal lobectomy. Eighteen epilepsy surgery patients were tested, pre- and postoperatively, in 3 different depth discrimination tasks. Sensitivity for local and global disparity was tested in a near-far discrimination task and sensitivity for 3D curvature was assessed in a convex-concave discrimination task, where 3D shapes were presented at different positions in depth. We found no evidence that temporal lobe epilepsy surgery has a significant effect on stereopsis. In contrast with earlier findings, we conclude that local as well as global stereopsis is maintained after unilateral resection of the temporal pole in epilepsy surgery patients. Our findings, together with previous studies, suggest that in humans more posterior visual regions underlie depth perception.

Lesions to right posterior parietal cortex impair visual depth perception from disparity but not motion cues

The posterior parietal cortex (PPC) is understood to be active when observers perceive three-dimensional (3D) structure. However, it is not clear how central this activity is in the construction of 3D spatial representations. Here, we examine whether PPC is essential for two aspects of visual depth perception by testing patients with lesions affecting this region. First, we measured subjects' ability to discriminate depth structure in various 3D surfaces and objects using binocular disparity. Patients with lesions to right PPC (N = 3) exhibited marked perceptual deficits on these tasks, whereas those with left hemisphere lesions (N = 2) were able to reliably discriminate depth as accurately as control subjects. Second, we presented an ambiguous 3D stimulus defined by structure from motion to determine whether PPC lesions influence the rate of bistable perceptual alternations. Patients' percept durations for the 3D stimulus were generally within a normal range, although the two patients with bilateral PPC lesions showed the fastest perceptual alternation rates in our sample. Intermittent stimulus presentation reduced the reversal rate similarly across subjects. Together, the results suggest that PPC plays a causal role in both inferring and maintaining the perception of 3D structure with stereopsis supported primarily by the right hemisphere, but do not lend support to the view that PPC is a critical contributor to bistable perceptual alternations.This article is part of the themed issue 'Vision in our three-dimensional world'.