Extracting 3D from motion: differences in human and monkey intraparietal cortex

Differential processing of the direction and focus of expansion of optic flow stimuli in areas MST and V3A of the human visual cortex

Human neuropsychological and neuroimaging studies have raised the possibility that different attributes of optic flow stimuli, namely radial direction and the position of the focus of expansion (FOE), are processed within separate cortical areas. In the human brain, visual areas V5/MT+ and V3A have been proposed as integral to the analysis of these different attributes of optic flow stimuli. To establish direct causal relationships between neural activity in human (h)V5/MT+ and V3A and the perception of radial motion direction and FOE position, we used transcranial magnetic stimulation (TMS) to disrupt cortical activity in these areas while participants performed behavioral tasks dependent on these different aspects of optic flow stimuli. The cortical regions of interest were identified in seven human participants using standard functional MRI retinotopic mapping techniques and functional localizers. TMS to area V3A was found to disrupt FOE positional judgments but not radial direction discrimination, whereas the application of TMS to an anterior subdivision of hV5/MT+, MST/TO-2 produced the reverse effects, disrupting radial direction discrimination but eliciting no effect on the FOE positional judgment task. This double dissociation demonstrates that FOE position and radial direction of optic flow stimuli are signaled independently by neural activity in areas hV5/MT+ and V3A.NEW & NOTEWORTHY Optic flow constitutes a biologically relevant visual cue as we move through any environment. With the use of neuroimaging and brain-stimulation techniques, this study demonstrates that separate human brain areas are involved in the analysis of the direction of radial motion and the focus of expansion in optic flow. This dissociation reveals the existence of separate processing pathways for the analysis of different attributes of optic flow that are important for the guidance of self-locomotion and object avoidance.

Computational Architecture of the Parieto-Frontal Network Underlying Cognitive-Motor Control in Monkeys

The statistical structure of intrinsic parietal and parieto-frontal connectivity in monkeys was studied through hierarchical cluster analysis. Based on their inputs, parietal and frontal areas were grouped into different clusters, including a variable number of areas that in most instances occupied contiguous architectonic fields. Connectivity tended to be stronger locally: that is, within areas of the same cluster. Distant frontal and parietal areas were targeted through connections that in most instances were reciprocal and often of different strength. These connections linked parietal and frontal clusters formed by areas sharing basic functional properties. This led to five different medio-laterally oriented pillar domains spanning the entire extent of the parieto-frontal system, in the posterior parietal, anterior parietal, cingulate, frontal, and prefrontal cortex. Different information processing streams could be identified thanks to inter-domain connectivity. These streams encode fast hand reaching and its control, complex visuomotor action spaces, hand grasping, action/intention recognition, oculomotor intention and visual attention, behavioral goals and strategies, and reward and decision value outcome. Most of these streams converge on the cingulate domain, the main hub of the system. All of them are embedded within a larger eye–hand coordination network, from which they can be selectively set in motion by task demands.

The dorsal visual stream revisited: Stable circuits or dynamic pathways?

In both macaque and human brain, information regarding visual motion flows from the extrastriate area V6 along two different paths: a dorsolateral one towards areas MT/V5, MST, V3A, and a dorsomedial one towards the visuomotor areas of the superior parietal lobule (V6A, MIP, VIP). The dorsolateral visual stream is involved in many aspects of visual motion analysis, including the recognition of object motion and self motion. The dorsomedial stream uses visual motion information to continuously monitor the spatial location of objects while we are looking and/or moving around, to allow skilled reaching for and grasping of the objects in structured, dynamically changing environments. Grasping activity is present in two areas of the dorsal stream, AIP and V6A. Area AIP is more involved than V6A in object recognition, V6A in encoding vision for action. We suggest that V6A is involved in the fast control of prehension and plays a critical role in biomechanically selecting appropriate postures during reach to grasp behaviors. In everyday life, numerous functional networks, often involving the same cortical areas, are continuously in action in the dorsal visual stream, with each network dynamically activated or inhibited according to the context. The dorsolateral and dorsomedial streams represent only two examples of these networks. Many others streams have been described in the literature, but it is worthwhile noting that the same cortical area, and even the same neurons within an area, are not specific for just one functional property, being part of networks that encode multiple functional aspects. Our proposal is to conceive the cortical streams not as fixed series of interconnected cortical areas in which each area belongs univocally to one stream and is strictly involved in only one function, but as interconnected neuronal networks, often involving the same neurons, that are involved in a number of functional processes and whose activation changes dynamically according to the context.

'What' Is Happening in the Dorsal Visual Pathway

The cortical visual system is almost universally thought to be segregated into two anatomically and functionally distinct pathways: a ventral occipitotemporal pathway that subserves object perception, and a dorsal occipitoparietal pathway that subserves object localization and visually guided action. Accumulating evidence from both human and non-human primate studies, however, challenges this binary distinction and suggests that regions in the dorsal pathway contain object representations that are independent of those in ventral cortex and that play a functional role in object perception. We review here the evidence implicating dorsal object representations, and we propose an account of the anatomical organization, functional contributions, and origins of these representations in the service of perception.

Visual motion serves but is not under the purview of the dorsal pathway

Visual motion processing is often attributed to the dorsal visual pathway despite visual motion's involvement in almost all visual functions. Furthermore, some visual motion tasks critically depend on the structural integrity of regions outside the dorsal pathway. Here, based on numerous studies, I propose that visual motion signals are swiftly transmitted via multiple non-hierarchical routes to primary motion-dedicated processing regions (MT/V5 and MST) that are not part of the dorsal pathway, and then propagated to a multiplicity of brain areas according to task demands, reaching these regions earlier than the dorsal/ventral hierarchical flow. This not only places MT/V5 at the same or even earlier visual processing stage as that of V1, but can also elucidate many findings with implications to visual awareness. While the integrity of the non-hierarchical motion pathway is necessary for all visual motion perception, it is insufficient on its own, and the transfer of visual motion signals to additional brain areas is crucial to allow the different motion perception tasks (e.g. optic flow, visuo-vestibular balance, movement observation, dynamic form detection and perception, and even reading). I argue that this lateral visual motion pathway can be distinguished from the dorsal pathway not only based on faster response latencies and distinct anatomical connections, but also based on its full field representation. I also distinguish between this primary lateral visual motion pathway sensitive to all motion in the visual field, and a much less investigated optic flow sensitive medial processing pathway (from V1 to V6 and V6A) that appears to be part of the dorsal pathway. Multiple additional predictions are provided that allow testing this proposal and distinguishing between the visual pathways.

Lesions to right posterior parietal cortex impair visual depth perception from disparity but not motion cues

The posterior parietal cortex (PPC) is understood to be active when observers perceive three-dimensional (3D) structure. However, it is not clear how central this activity is in the construction of 3D spatial representations. Here, we examine whether PPC is essential for two aspects of visual depth perception by testing patients with lesions affecting this region. First, we measured subjects' ability to discriminate depth structure in various 3D surfaces and objects using binocular disparity. Patients with lesions to right PPC (N = 3) exhibited marked perceptual deficits on these tasks, whereas those with left hemisphere lesions (N = 2) were able to reliably discriminate depth as accurately as control subjects. Second, we presented an ambiguous 3D stimulus defined by structure from motion to determine whether PPC lesions influence the rate of bistable perceptual alternations. Patients' percept durations for the 3D stimulus were generally within a normal range, although the two patients with bilateral PPC lesions showed the fastest perceptual alternation rates in our sample. Intermittent stimulus presentation reduced the reversal rate similarly across subjects. Together, the results suggest that PPC plays a causal role in both inferring and maintaining the perception of 3D structure with stereopsis supported primarily by the right hemisphere, but do not lend support to the view that PPC is a critical contributor to bistable perceptual alternations.This article is part of the themed issue 'Vision in our three-dimensional world'.

Dorsal and ventral stream contributions to form-from-motion perception in a patient with form-from motion deficit: a case report

The main model of visual processing in primates proposes an anatomo-functional distinction between the dorsal stream, specialized in spatio-temporal information, and the ventral stream, processing essentially form information. However, these two pathways also communicate to share much visual information. These dorso-ventral interactions have been studied using form-from-motion (FfM) stimuli, revealing that FfM perception first activates dorsal regions (e.g., MT+/V5), followed by successive activations of ventral regions (e.g., LOC). However, relatively little is known about the implications of focal brain damage of visual areas on these dorso-ventral interactions. In the present case report, we investigated the dynamics of dorsal and ventral activations related to FfM perception (using topographical ERP analysis and electrical source imaging) in a patient suffering from a deficit in FfM perception due to right extrastriate brain damage in the ventral stream. Despite the patient's FfM impairment, both successful (observed for the highest level of FfM signal) and absent/failed FfM perception evoked the same temporal sequence of three processing states observed previously in healthy subjects. During the first period, brain source localization revealed cortical activations along the dorsal stream, currently associated with preserved elementary motion processing. During the latter two periods, the patterns of activity differed from normal subjects: activations were observed in the ventral stream (as reported for normal subjects), but also in the dorsal pathway, with the strongest and most sustained activity localized in the parieto-occipital regions. On the other hand, absent/failed FfM perception was characterized by weaker brain activity, restricted to the more lateral regions. This study shows that in the present case report, successful FfM perception, while following the same temporal sequence of processing steps as in normal subjects, evoked different patterns of brain activity. By revealing a brain circuit involving the most rostral part of the dorsal pathway, this study provides further support for neuro-imaging studies and brain lesion investigations that have suggested the existence of different brain circuits associated with different profiles of interaction between the dorsal and the ventral streams.

Functional definitions of parietal areas in human and non-human primates

Establishing homologies between cortical areas in animal models and humans lies at the heart of translational neuroscience, as it demonstrates how knowledge obtained from these models can be applied to the human brain. Here, we review progress in using parallel functional imaging to ascertain homologies between parietal areas of human and non-human primates, species sharing similar behavioural repertoires. The human homologues of several areas along monkey IPS involved in action planning and observation, such as AIP, LIP and CIP, as well as those of opercular areas (SII complex), have been defined. In addition, uniquely human areas, such as the tool-use area in left anterior supramarginal gyrus, have also been identified.

The neural representation of objects formed through the spatiotemporal integration of visual transients

Oftentimes, objects are only partially and transiently visible as parts of them become occluded during observer or object motion. The visual system can integrate such object fragments across space and time into perceptual wholes or spatiotemporal objects. This integrative and dynamic process may involve both ventral and dorsal visual processing pathways, along which shape and spatial representations are thought to arise. We measured fMRI BOLD response to spatiotemporal objects and used multi-voxel pattern analysis (MVPA) to decode shape information across 20 topographic regions of visual cortex. Object identity could be decoded throughout visual cortex, including intermediate (V3A, V3B, hV4, LO1-2,) and dorsal (TO1-2, and IPS0-1) visual areas. Shape-specific information, therefore, may not be limited to early and ventral visual areas, particularly when it is dynamic and must be integrated. Contrary to the classic view that the representation of objects is the purview of the ventral stream, intermediate and dorsal areas may play a distinct and critical role in the construction of object representations across space and time.

Differential processing of binocular and monocular gloss cues in human visual cortex

The visual impression of an object's surface reflectance (“gloss”) relies on a range of visual cues, both monocular and binocular. Whereas previous imaging work has identified processing within ventral visual areas as important for monocular cues, little is known about cortical areas involved in processing binocular cues. Here, we used human functional MRI (fMRI) to test for brain areas selectively involved in the processing of binocular cues. We manipulated stereoscopic information to create four conditions that differed in their disparity structure and in the impression of surface gloss that they evoked. We performed multivoxel pattern analysis to find areas whose fMRI responses allow classes of stimuli to be distinguished based on their depth structure vs. material appearance. We show that higher dorsal areas play a role in processing binocular gloss information, in addition to known ventral areas involved in material processing, with ventral area lateral occipital responding to both object shape and surface material properties. Moreover, we tested for similarities between the representation of gloss from binocular cues and monocular cues. Specifically, we tested for transfer in the decoding performance of an algorithm trained on glossy vs. matte objects defined by either binocular or by monocular cues. We found transfer effects from monocular to binocular cues in dorsal visual area V3B/kinetic occipital (KO), suggesting a shared representation of the two cues in this area. These results indicate the involvement of mid- to high-level visual circuitry in the estimation of surface material properties, with V3B/KO potentially playing a role in integrating monocular and binocular cues.

A ventral salience network in the macaque brain

Successful navigation of the environment requires attending and responding efficiently to objects and conspecifics with the potential to benefit or harm (i.e., that have value). In humans, this function is subserved by a distributed large-scale neural network called the "salience network". We have recently demonstrated that there are two anatomically and functionally dissociable salience networks anchored in the dorsal and ventral portions of the human anterior insula (Touroutoglou et al., 2012). In this paper, we test the hypothesis that these two subnetworks exist in rhesus macaques (Macaca mulatta). We provide evidence that a homologous ventral salience network exists in macaques, but that the connectivity of the dorsal anterior insula in macaques is not sufficiently developed as a dorsal salience network. The evolutionary implications of these finding are considered.

3D Shape Perception in Posterior Cortical Atrophy: A Visual Neuroscience Perspective

Posterior cortical atrophy (PCA) is a rare focal neurodegenerative syndrome characterized by progressive visuoperceptual and visuospatial deficits, most often due to atypical Alzheimer's disease (AD). We applied insights from basic visual neuroscience to analyze 3D shape perception in humans affected by PCA. Thirteen PCA patients and 30 matched healthy controls participated, together with two patient control groups with diffuse Lewy body dementia (DLBD) and an amnestic-dominant phenotype of AD, respectively. The hierarchical study design consisted of 3D shape processing for 4 cues (shading, motion, texture, and binocular disparity) with corresponding 2D and elementary feature extraction control conditions. PCA and DLBD exhibited severe 3D shape-processing deficits and AD to a lesser degree. In PCA, deficient 3D shape-from-shading was associated with volume loss in the right posterior inferior temporal cortex. This region coincided with a region of functional activation during 3D shape-from-shading in healthy controls. In PCA patients who performed the same fMRI paradigm, response amplitude during 3D shape-from-shading was reduced in this region. Gray matter volume in this region also correlated with 3D shape-from-shading in AD. 3D shape-from-disparity in PCA was associated with volume loss slightly more anteriorly in posterior inferior temporal cortex as well as in ventral premotor cortex. The findings in right posterior inferior temporal cortex and right premotor cortex are consistent with neurophysiologically based models of the functional anatomy of 3D shape processing. However, in DLBD, 3D shape deficits rely on mechanisms distinct from inferior temporal structural integrity.

SIGNIFICANCE STATEMENT

Posterior cortical atrophy (PCA) is a neurodegenerative syndrome characterized by progressive visuoperceptual dysfunction and most often an atypical presentation of Alzheimer's disease (AD) affecting the ventral and dorsal visual streams rather than the medial temporal system. We applied insights from fundamental visual neuroscience to analyze 3D shape perception in PCA. 3D shape-processing deficits were affected beyond what could be accounted for by lower-order processing deficits. For shading and disparity, this was related to volume loss in regions previously implicated in 3D shape processing in the intact human and nonhuman primate brain. Typical amnestic-dominant AD patients also exhibited 3D shape deficits. Advanced visual neuroscience provides insight into the pathogenesis of PCA that also bears relevance for vision in typical AD.

The Visual Priming of Motion-Defined 3D Objects

The perception of a stimulus can be influenced by previous perceptual experience, a phenomenon known as perceptual priming. However, there has been limited investigation on perceptual priming of shape perception of three-dimensional object structures defined by moving dots. Here we examined the perceptual priming of a 3D object shape defined purely by motion-in-depth cues (i.e., Shape-From-Motion, SFM) using a classic prime-target paradigm. The results from the first two experiments revealed a significant increase in accuracy when a “cloudy” SFM stimulus (whose object structure was difficult to recognize due to the presence of strong noise) was preceded by an unambiguous SFM that clearly defined the same transparent 3D shape. In contrast, results from Experiment 3 revealed no change in accuracy when a “cloudy” SFM stimulus was preceded by a static shape or a semantic word that defined the same object shape. Instead, there was a significant decrease in accuracy when preceded by a static shape or a semantic word that defined a different object shape. These results suggested that the perception of a noisy SFM stimulus can be facilitated by a preceding unambiguous SFM stimulus—but not a static image or a semantic stimulus—that defined the same shape. The potential neural and computational mechanisms underlying the difference in priming are discussed.

fMRI Activity in Posterior Parietal Cortex Relates to the Perceptual Use of Binocular Disparity for Both Signal-In-Noise and Feature Difference Tasks

Visually guided action and interaction depends on the brain’s ability to (a) extract and (b) discriminate meaningful targets from complex retinal inputs. Binocular disparity is known to facilitate this process, and it is an open question how activity in different parts of the visual cortex relates to these fundamental visual abilities. Here we examined fMRI responses related to performance on two different tasks (signal-in-noise “coarse” and feature difference “fine” tasks) that have been widely used in previous work, and are believed to differentially target the visual processes of signal extraction and feature discrimination. We used multi-voxel pattern analysis to decode depth positions (near vs. far) from the fMRI activity evoked while participants were engaged in these tasks. To look for similarities between perceptual judgments and brain activity, we constructed ‘fMR-metric’ functions that described decoding performance as a function of signal magnitude. Thereafter we compared fMR-metric and psychometric functions, and report an association between judged depth and fMRI responses in the posterior parietal cortex during performance on both tasks. This highlights common stages of processing during perceptual performance on these tasks.

Mapping the macaque superior temporal sulcus: functional delineation of vergence and version eye-movement-related activity

It is currently thought that the primate oculomotor system has evolved distinct but interrelated subsystems to generate different types of visually guided eye movements (e.g., saccades/smooth pursuit/vergence). Although progress has been made in elucidating the neural basis of these movement types, no study to date has investigated all three movement types on a large scale and within the same animals. Here, we used fMRI in rhesus macaque monkeys to map the superior temporal sulcus (STS) for BOLD modulation associated with visually guided eye movements. Further, we ascertained whether modulation in a given area was movement type specific and, if not, the modulation each movement type elicited relative to the others (i.e., dominance). Our results show that multiple areas within STS modulate during all movement types studied, including the middle temporal, medial superior temporal, fundus of the superior temporal, lower superior temporal, and dorsal posterior inferotemporal areas. Our results also reveal an area in dorsomedial STS that is modulated almost exclusively by vergence movements. In contrast, we found that ventrolateral STS is driven preferentially during versional movements. These results illuminate an STS network involved in processes associated with multiple eye movement types, illustrate unique patterns of modulation within said network as a function of movement type, and provide evidence for a vergence-specific area within dorsomedial STS. We conclude that producing categorically different eye movement types requires access to a common STS network and that individual network nodes are recruited differentially based upon the type of movement generated.

The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study

The present fMRI study examined whether upper-limb action classes differing in their motor goal are encoded by different PPC sectors. Action observation was used as a proxy for action execution. Subjects viewed actors performing object-related (e.g., grasping), skin-displacing (e.g., rubbing the skin), and interpersonal upper limb actions (e.g., pushing someone). Observation of the three action classes activated a three-level network including occipito-temporal, parietal, and premotor cortex. The parietal region common to observing all three action classes was located dorsally to the left intraparietal sulcus (DIPSM/DIPSA border). Regions specific for observing an action class were obtained by combining the interaction between observing action classes and stimulus types with exclusive masking for observing the other classes, while for regions considered preferentially active for a class the interaction was exclusively masked with the regions common to all observed actions. Left putative human anterior intraparietal was specific for observing manipulative actions, and left parietal operculum including putative human SII region, specific for observing skin-displacing actions. Control experiments demonstrated that this latter activation depended on seeing the skin being moved and not simply on seeing touch. Psychophysiological interactions showed that the two specific parietal regions had similar connectivities. Finally, observing interpersonal actions preferentially activated a dorsal sector of left DIPSA, possibly the homologue of ventral intraparietal coding the impingement of the target person's body into the peripersonal space of the actor. These results support the importance of segregation according to the action class as principle of posterior parietal cortex organization for action observation and by implication for action execution.

Organization and evolution of parieto-frontal processing streams in macaque monkeys and humans

The functional organization of the parieto-frontal system is crucial for understanding cognitive-motor behavior and provides the basis for interpreting the consequences of parietal lesions in humans from a neurobiological perspective. The parieto-frontal connectivity defines some main information streams that, rather than being devoted to restricted functions, underlie a rich behavioral repertoire. Surprisingly, from macaque to humans, evolution has added only a few, new functional streams, increasing however their complexity and encoding power. In fact, the characterization of the conduction times of parietal and frontal areas to different target structures has recently opened a new window on cortical dynamics, suggesting that evolution has amplified the probability of dynamic interactions between the nodes of the network, thanks to communication patterns based on temporally-dispersed conduction delays. This might allow the representation of sensory-motor signals within multiple neural assemblies and reference frames, as to optimize sensory-motor remapping within an action space characterized by different and more complex demands across evolution.

Effective connectivity of depth-structure-selective patches in the lateral bank of the macaque intraparietal sulcus

Extrastriate cortical areas are frequently composed of subpopulations of neurons encoding specific features or stimuli, such as color, disparity, or faces, and patches of neurons encoding similar stimulus properties are typically embedded in interconnected networks, such as the attention or face-processing network. The goal of the current study was to examine the effective connectivity of subsectors of neurons in the same cortical area with highly similar neuronal response properties. We first recorded single- and multi-unit activity to identify two neuronal patches in the anterior part of the macaque intraparietal sulcus (IPS) showing the same depth structure selectivity and then employed electrical microstimulation during functional magnetic resonance imaging in these patches to determine the effective connectivity of these patches. The two IPS subsectors we identified-with the same neuronal response properties and in some cases separated by only 3 mm-were effectively connected to remarkably distinct cortical networks in both dorsal and ventral stream in three macaques. Conversely, the differences in effective connectivity could account for the known visual-to-motor gradient within the anterior IPS. These results clarify the role of the anterior IPS as a pivotal brain region where dorsal and ventral visual stream interact during object analysis. Thus, in addition to the anatomical connectivity of cortical areas and the properties of individual neurons in these areas, the effective connectivity provides novel key insights into the widespread functional networks that support behavior.

Monkey cortex through fMRI glasses

In 1998 several groups reported the feasibility of fMRI experiments in monkeys, with the goal to bridge the gap between invasive nonhuman primate studies and human functional imaging. These studies yielded critical insights in the neuronal underpinnings of the BOLD signal. Furthermore, the technology has been successful in guiding electrophysiological recordings and identifying focal perturbation targets. Finally, invaluable information was obtained concerning human brain evolution. We here provide a comprehensive overview of awake monkey fMRI studies mainly confined to the visual system. We review the latest insights about the topographic organization of monkey visual cortex and discuss the spatial relationships between retinotopy and category- and feature-selective clusters. We briefly discuss the functional layout of parietal and frontal cortex and continue with a summary of some fascinating functional and effective connectivity studies. Finally, we review recent comparative fMRI experiments and speculate about the future of nonhuman primate imaging.

fMRI evidence for areas that process surface gloss in the human visual cortex

•

Glossiness information is mainly processed along ventral visual pathway.

•

The posterior fusiform sulcus (pFs) is especially selective to surface gloss.

•

V3B/KO responds to gloss, but differentially from the pFs.

Surface gloss is an important cue to the material properties of objects. Recent progress in the study of macaque’s brain has increased our understating of the areas involved in processing information about gloss, however the homologies with the human brain are not yet fully understood. Here we used human functional magnetic resonance imaging (fMRI) measurements to localize brain areas preferentially responding to glossy objects. We measured cortical activity for thirty-two rendered three-dimensional objects that had either Lambertian or specular surface properties. To control for differences in image structure, we overlaid a grid on the images and scrambled its cells. We found activations related to gloss in the posterior fusiform sulcus (pFs) and in area V3B/KO. Subsequent analysis with Granger causality mapping indicated that V3B/KO processes gloss information differently than pFs. Our results identify a small network of mid-level visual areas whose activity may be important in supporting the perception of surface gloss.

The retinotopic organization of macaque occipitotemporal cortex anterior to V4 and caudoventral to the middle temporal (MT) cluster

The retinotopic organization of macaque occipitotemporal cortex rostral to area V4 and caudorostral to the recently described middle temporal (MT) cluster of the monkey (Kolster et al., 2009) is not well established. The proposed number of areas within this region varies from one to four, underscoring the ambiguity concerning the functional organization in this region of extrastriate cortex. We used phase-encoded retinotopic functional MRI mapping methods to reveal the functional topography of this cortical domain. Polar-angle maps showed one complete hemifield representation bordering area V4 anteriorly, split into dorsal and ventral counterparts corresponding to the lower and upper visual field quadrants, respectively. The location of this hemifield representation corresponds to area V4A. More rostroventrally, we identified three other complete hemifield representations. Two of these correspond to the dorsal and the ventral posterior inferotemporal areas (PITd and PITv, respectively) as identified in the Felleman and Van Essen (1991) scheme. The third representation has been tentatively named dorsal occipitotemporal area (OTd). Areas V4A, PITd, PITv, and OTd share a central visual field representation, similar to the areas constituting the MT cluster. Furthermore, they vary widely in size and represent the complete contralateral visual field. Functionally, these four areas show little motion sensitivity, unlike those of the MT cluster, and two of them, OTd and PITd, displayed pronounced two-dimensional shape sensitivity. In general, these results suggest that retinotopically organized tissue extends farther into rostral occipitotemporal cortex of the monkey than generally assumed.

Human area MT+ shows load-dependent activation during working memory maintenance with continuously morphing stimulation

Background

Initially, human area MT⁺ was considered a visual area solely processing motion information but further research has shown that it is also involved in various different cognitive operations, such as working memory tasks requiring motion-related information to be maintained or cognitive tasks with implied or expected motion.

In the present fMRI study in humans, we focused on MT⁺ modulation during working memory maintenance using a dynamic shape-tracking working memory task with no motion-related working memory content. Working memory load was systematically varied using complex and simple stimulus material and parametrically increasing retention periods. Activation patterns for the difference between retention of complex and simple memorized stimuli were examined in order to preclude that the reported effects are caused by differences in retrieval.

Results

Conjunction analysis over all delay durations for the maintenance of complex versus simple stimuli demonstrated a wide-spread activation pattern. Percent signal change (PSC) in area MT⁺ revealed a pattern with higher values for the maintenance of complex shapes compared to the retention of a simple circle and with higher values for increasing delay durations.

Conclusions

The present data extend previous knowledge by demonstrating that visual area MT⁺ presents a brain activity pattern usually found in brain regions that are actively involved in working memory maintenance.

The neural basis of human tool use

In this review, we propose that the neural basis for the spontaneous, diversified human tool use is an area devoted to the execution and observation of tool actions, located in the left anterior supramarginal gyrus (aSMG). The aSMG activation elicited by observing tool use is typical of human subjects, as macaques show no similar activation, even after an extensive training to use tools. The execution of tool actions, as well as their observation, requires the convergence upon aSMG of inputs from different parts of the dorsal and ventral visual streams. Non-semantic features of the target object may be provided by the posterior parietal cortex (PPC) for tool-object interaction, paralleling the well-known PPC input to anterior intraparietal (AIP) for hand-object interaction. Semantic information regarding tool identity, and knowledge of the typical manner of handling the tool, could be provided by inferior and middle regions of the temporal lobe. Somatosensory feedback and technical reasoning, as well as motor and intentional constraints also play roles during the planning of tool actions and consequently their signals likewise converge upon aSMG. We further propose that aSMG may have arisen though duplication of monkey AIP and invasion of the duplicate area by afferents from PPC providing distinct signals depending on the kinematics of the manipulative action. This duplication may have occurred when Homo Habilis or Homo Erectus emerged, generating the Oldowan or Acheulean Industrial complexes respectively. Hence tool use may have emerged during hominid evolution between bipedalism and language. We conclude that humans have two parietal systems involved in tool behavior: a biological circuit for grasping objects, including tools, and an artifactual system devoted specifically to tool use. Only the latter allows humans to understand the causal relationship between tool use and obtaining the goal, and is likely to be the basis of all technological developments.

Differential reliance of chimpanzees and humans on automatic and deliberate control of motor actions

Humans are often unaware of how they control their limb motor movements. People pay attention to their own motor movements only when their usual motor routines encounter errors. Yet little is known about the extent to which voluntary actions rely on automatic control and when automatic control shifts to deliberate control in nonhuman primates. In this study, we demonstrate that chimpanzees and humans showed similar limb motor adjustment in response to feedback error during reaching actions, whereas attentional allocation inferred from gaze behavior differed. We found that humans shifted attention to their own motor kinematics as errors were induced in motor trajectory feedback regardless of whether the errors actually disrupted their reaching their action goals. In contrast, chimpanzees shifted attention to motor execution only when errors actually interfered with their achieving a planned action goal. These results indicate that the species differed in their criteria for shifting from automatic to deliberate control of motor actions. It is widely accepted that sophisticated motor repertoires have evolved in humans. Our results suggest that the deliberate monitoring of one's own motor kinematics may have evolved in the human lineage.

Structure-from-motion: dissociating perception, neural persistence, and sensory memory of illusory depth and illusory rotation

In the structure-from-motion paradigm, physical motion on a screen produces the vivid illusion of an object rotating in depth. Here, we show how to dissociate illusory depth and illusory rotation in a structure-from-motion stimulus using a rotationally asymmetric shape and reversals of physical motion. Reversals of physical motion create a conflict between the original illusory states and the new physical motion: Either illusory depth remains constant and illusory rotation reverses, or illusory rotation stays the same and illusory depth reverses. When physical motion reverses after the interruption in presentation, we find that illusory rotation tends to remain constant for long blank durations (T (blank) ≥ 0.5 s), but illusory depth is stabilized if interruptions are short (T (blank) ≤ 0.1 s). The stability of illusory depth over brief interruptions is consistent with the effect of neural persistence. When this is curtailed using a mask, stability of ambiguous vision (for either illusory depth or illusory rotation) is disrupted. We also examined the selectivity of the neural persistence of illusory depth. We found that it relies on a static representation of an interpolated illusory object, since changes to low-level display properties had little detrimental effect. We discuss our findings with respect to other types of history dependence in multistable displays (sensory stabilization memory, neural fatigue, etc.). Our results suggest that when brief interruptions are used during the presentation of multistable displays, switches in perception are likely to rely on the same neural mechanisms as spontaneous switches, rather than switches due to the initial percept choice at the stimulus onset.

Are non-human primates capable of rhythmic entrainment? Evidence for the gradual audiomotor evolution hypothesis

We propose a decomposition of the neurocognitive mechanisms that might underlie interval-based timing and rhythmic entrainment. Next to reviewing the concepts central to the definition of rhythmic entrainment, we discuss recent studies that suggest rhythmic entrainment to be specific to humans and a selected group of bird species, but, surprisingly, is not obvious in non-human primates. On the basis of these studies we propose the gradual audiomotor evolution hypothesis that suggests that humans fully share interval-based timing with other primates, but only partially share the ability of rhythmic entrainment (or beat-based timing). This hypothesis accommodates the fact that non-human primates (i.e., macaques) performance is comparable to humans in single interval tasks (such as interval reproduction, categorization, and interception), but show differences in multiple interval tasks (such as rhythmic entrainment, synchronization, and continuation). Furthermore, it is in line with the observation that macaques can, apparently, synchronize in the visual domain, but show less sensitivity in the auditory domain. And finally, while macaques are sensitive to interval-based timing and rhythmic grouping, the absence of a strong coupling between the auditory and motor system of non-human primates might be the reason why macaques cannot rhythmically entrain in the way humans do.

Action and language mechanisms in the brain: data, models and neuroinformatics

We assess the challenges of studying action and language mechanisms in the brain, both singly and in relation to each other to provide a novel perspective on neuroinformatics, integrating the development of databases for encoding – separately or together – neurocomputational models and empirical data that serve systems and cognitive neuroscience.

Differences between chimpanzees and humans in visual temporal integration

Humans have a superior ability to integrate spatially separate visual information into an entire image. In contrast, comparative cognitive studies have demonstrated that nonhuman primates and avian species are superior in processing relatively local features; however, animals in these studies were required to ignore local shape when they perceived the global configuration, and no studies have directly examined the ability to integrate temporally separate events. In this study, we compared the spatio-temporal visual integration of chimpanzees and humans by exploring dynamic shape perception under a slit-viewing condition. The findings suggest that humans exhibit greater temporal integration accuracy than do chimpanzees. The results show that the ability to integrate local visual information into a global whole is among the unique characteristics of humans.

Differences in neural activation for object-directed grasping in chimpanzees and humans

The human faculty for object-mediated action, including tool use and imitation, exceeds that of even our closest primate relatives and is a key foundation of human cognitive and cultural uniqueness. In humans and macaques, observing object-directed grasping actions activates a network of frontal, parietal, and occipitotemporal brain regions, but differences in human and macaque activation suggest that this system has been a focus of selection in the primate lineage. To study the evolution of this system, we performed functional neuroimaging in humans' closest living relatives, chimpanzees. We compare activations during performance of an object-directed manual grasping action, observation of the same action, and observation of a mimed version of the action that consisted of only movements without results. Performance and observation of the same action activated a distributed frontoparietal network similar to that reported in macaques and humans. Like humans and unlike macaques, these regions were also activated by observing movements without results. However, in a direct chimpanzee/human comparison, we also identified unique aspects of human neural responses to observed grasping. Chimpanzee activation showed a prefrontal bias, including significantly more activity in ventrolateral prefrontal cortex, whereas human activation was more evenly distributed across more posterior regions, including significantly more activation in ventral premotor cortex, inferior parietal cortex, and inferotemporal cortex. This indicates a more "bottom-up" representation of observed action in the human brain and suggests that the evolution of tool use, social learning, and cumulative culture may have involved modifications of frontoparietal interactions.

A motion direction preference map in monkey V4

In the primate visual system, area V4 is located in the ventral pathway and is traditionally thought to be involved in processing color and form information. However, little is known about its functional role in processing motion information. Using intrinsic signal optical imaging over large fields of view in V1, V2, and V4, we mapped the direction of motion responses in anesthetized macaques. We found that V4 contains direction-preferring domains that are preferentially activated by stimuli moving in one direction. These direction-preferring domains normally occupy several restricted regions of V4 and tend to overlap with orientation- and color-preferring domains. Single-cell recordings targeting these direction-preferring domains also showed a clustering, as well as a columnar organization of V4 direction-selective neurons. These data suggest that, in contrast to the classical view, motion information is also processed in ventral pathway regions such as area V4.

Dopaminergic reward signals selectively decrease fMRI activity in primate visual cortex

Stimulus-reward coupling without attention can induce highly specific perceptual learning effects, suggesting that reward triggers selective plasticity within visual cortex. Additionally, dopamine-releasing events-temporally surrounding stimulus-reward associations-selectively enhance memory. These forms of plasticity may be evoked by selective modulation of stimulus representations during dopamine-inducing events. However, it remains to be shown whether dopaminergic signals can selectively modulate visual cortical activity. We measured fMRI activity in monkey visual cortex during reward-only trials apart from intermixed cue-reward trials. Reward without visual stimulation selectively decreased fMRI activity within the cue representations that had been paired with reward during other trials. Behavioral tests indicated that these same uncued reward trials strengthened cue-reward associations. Furthermore, such spatially-specific activity modulations depended on prediction error, as shown by manipulations of reward magnitude, cue-reward probability, cue-reward familiarity, and dopamine signaling. This cue-selective negative reward signal offers a mechanism for selectively gating sensory cortical plasticity.

Evolutionarily novel functional networks in the human brain?

Primate evolution has been accompanied by complex reorganizations in brain anatomy and function. Little is known, however, about the relationship between anatomical and functional changes induced through primate evolution. Using functional magnetic resonance imaging, we assessed spatial and temporal correspondences of cortical networks in humans and monkeys. We provided evidence for topologically and functionally correspondent networks in sensory-motor and attention regions. More specifically, we revealed a possible monkey equivalent of the human ventral attention network. For other human networks, such as the language and the default-mode networks, we detected topological correspondent networks in the monkey, but with different functional signatures. Furthermore, we observed two lateralized human frontoparietal networks in the cortical regions displaying the greatest evolutionary expansion, having neither topological nor functional monkey correspondents. This finding may indicate that these two human networks are evolutionarily novel. Thus, our findings confirm the existence of networks where evolution has conserved both topology and function but also suggest that functions of structurally preserved networks can diverge over time and that novel, hence human-specific networks, have emerged during human evolution.

Asymmetries of the parietal operculum in chimpanzees (Pan troglodytes) in relation to handedness for tool use

A left larger than right planum temporale (PT) is a neuroanatomical asymmetry common to both humans and chimpanzees. A similar asymmetry was observed in the human parietal operculum (PO), and the convergence of PT and PO asymmetries is strongly associated with right-handedness. Here, we assessed whether this combination also exists in common chimpanzees. Magnetic resonance scans were obtained in 83 captive subjects. PT was quantified following procedures previously employed and PO was defined as the maximal linear distance between the end point of the sylvian fissure and the central sulcus. Handedness was assessed using 2 tasks that were designed to simulate termite fishing of wild chimpanzees and to elicit bimanual coordination without tool use. Chimpanzees showed population-level leftward asymmetries for both PT and PO. As in humans, these leftward asymmetries were not correlated. Handedness for tool use but not for nontool use motor actions mediated the expression of asymmetries in PT and PO, with right-handed apes showing more pronounced leftward asymmetries. Consistent PT and PO asymmetry combinations were observed in chimpanzees. The proportions of individuals showing these combinations were comparable in humans and chimpanzees; however, interaction between handedness and patterns of combined PO and PT asymmetries differed between the 2 species.

The functional role of the medial motion area V6

In macaque, several visual areas are devoted to analyze motion in the visual field, and V6 is one of these areas. In macaque, area V6 occupies the ventral part of the anterior bank of the parieto-occipital sulcus (POs), is retinotopically-organized and contains a point-to-point representation of the retinal surface. V6 is a motion sensitive area that largely represents the peripheral part of the visual field and whose cells are very sensitive to translational motion. Based on the fact that macaque V6 contains many real-motion cells, it has been suggested that V6 is involved in object-motion recognition. Recently, area V6 has been recognized also in the human brain by neuroimaging and electrophysiological methods. Like macaque V6, human V6 is located in the POs, is retinotopically organized, and represents the entire contralateral hemifield up to the far periphery. Human V6, like macaque V6, is a motion area that responds to unidirectional motion. It has a strong preference for coherent motion and a recent combined VEPs/fMRI work has shown that area V6 is even one of the most early stations coding the motion coherence. Human V6 is highly sensitive to flow field and is also able to distinguish between different 3D flow fields being selective to translational egomotion. This suggests that this area processes visual egomotion signals to extract information about the relative distance of objects, likely in order to act on them, or to avoid them. The view that V6 is involved in the estimation of egomotion has been tested also in other recent fMRI studies. Thus, taken together, human and macaque data suggest that V6 is involved in both object and self-motion recognition. Specifically, V6 could be involved in "subtracting out" self-motion signals across the whole visual field and in providing information about moving objects, particularly during self-motion in a complex and dynamically unstable environment.

High-resolution fMRI reveals laminar differences in neurovascular coupling between positive and negative BOLD responses

The six cortical layers have distinct anatomical and physiological properties, like different energy use and different feedforward and feedback connectivity. It is not known if and how layer-specific neural processes are reflected in the fMRI signal. To address this question we used high-resolution fMRI to measure BOLD, CBV, and CBF responses to stimuli that elicit positive and negative BOLD signals in macaque primary visual cortex. We found that regions with positive BOLD responses had parallel increases in CBV and CBF, whereas areas with negative BOLD responses showed a decrease in CBF but an increase in CBV. For positive BOLD responses, CBF and CBV increased in the center of the cortex, but for negative BOLD responses, CBF decreased superficially while CBV increased in the center. Our findings suggest different mechanisms for neurovascular coupling for BOLD increases and decreases, as well as laminar differences in neurovascular coupling.

Cytoarchitectonic mapping of the human dorsal extrastriate cortex

The dorsal visual stream consists of several functionally specialized areas, but most of their cytoarchitectonic correlates have not yet been identified in the human brain. The cortex adjacent to Brodmann area 18/V2 was therefore analyzed in serial sections of ten human post-mortem brains using morphometrical and multivariate statistical analyses for the definition of areal borders. Two previously unknown cytoarchitectonic areas (hOc3d, hOc4d) were detected. They occupy the medial and, to a smaller extent, lateral surface of the occipital lobe. The larger area, hOc3d, is located dorso-lateral to area V2 in the region of superior and transverse occipital, as well as parieto-occipital sulci. Area hOc4d was identified rostral to hOc3d; it differed from the latter by larger pyramidal cells in lower layer III, thinner layers V and VI, and a sharp cortex-white-matter borderline. The delineated areas were superimposed in the anatomical MNI space, and probabilistic maps were calculated. They show a relatively high intersubject variability in volume and position. Based on their location and neighborhood relationship, areas hOc3d and hOc4d are putative anatomical substrates of functionally defined areas V3d and V3a, a hypothesis that can now be tested by comparing probabilistic cytoarchitectonic maps and activation studies of the living human brain.

Perceptual experience modulates cortical circuits involved in visual awareness

Successful interactions with the environment entail interpreting ambiguous sensory information. To address this challenge it has been suggested that the brain optimizes performance through experience. Here we used functional magnetic resonance imaging (fMRI) to investigate whether perceptual experience modulates the cortical circuits involved in visual awareness. Using ambiguous visual stimuli (binocular rivalry or ambiguous structure-from-motion) we were able to disentangle the co-occurring influences of stimulus repetition and perceptual repetition. For both types of ambiguous stimuli we observed that the mere repetition of the stimulus evoked an entirely different pattern of activity modulations than the repetition of a particular perceptual interpretation of the stimulus. Regarding stimulus repetition, decreased fMRI responses were evident during binocular rivalry but weaker during 3-D motion rivalry. Perceptual repetition, on the other hand, entailed increased activity in stimulus-specific visual brain regions - for binocular rivalry in the early visual regions and for ambiguous structure-from-motion in both early as well as higher visual regions. This indicates that the repeated activation of a visual network mediating a particular percept facilitated its later reactivation. Perceptual repetition was also associated with a response change in the parietal cortex that was similar for the two types of ambiguous stimuli, possibly relating to the temporal integration of perceptual information. We suggest that perceptual repetition is associated with a facilitation of neural activity within and between percept-specific visual networks and parietal networks involved in the temporal integration of perceptual information, thereby enhancing the stability of previously experienced percepts.

Parallel motion signals to the medial and lateral motion areas V6 and MT+

MT+ and V6 are key motion areas of the dorsal visual stream in both macaque and human brains. In the present study, we combined electrophysiological and neuroimaging methods (including retinotopic brain mapping) to find the electrophysiological correlates of V6 and to define its temporal relationship with the activity observed in MT+. We also determined the spatio-temporal profile of the motion coherency effect on visual evoked potentials (VEPs), and localized its neural generators. We found that area V6 participates in the very early phase of the coherent motion processing and that its electroencephalographic activity is almost simultaneous with that of MT+. We also found a late second activity in V6 that we interpret as a re-entrant feedback from extrastriate visual areas (e.g. area V3A). Three main cortical sources were differently modulated by the motion coherence: while V6 and MT+ showed a preference for the coherent motion, area V3A preferred the random condition. The response timing of these cortical sources indicates that motion signals flow in parallel from the occipital pole to the medial and lateral motion areas V6 and MT+, suggesting the view of a differential functional role.

A brain network processing the age of faces

Age is one of the most salient aspects in faces and of fundamental cognitive and social relevance. Although face processing has been studied extensively, brain regions responsive to age have yet to be localized. Using evocative face morphs and fMRI, we segregate two areas extending beyond the previously established face-sensitive core network, centered on the inferior temporal sulci and angular gyri bilaterally, both of which process changes of facial age. By means of probabilistic tractography, we compare their patterns of functional activation and structural connectivity. The ventral portion of Wernicke's understudied perpendicular association fasciculus is shown to interconnect the two areas, and activation within these clusters is related to the probability of fiber connectivity between them. In addition, post-hoc age-rating competence is found to be associated with high response magnitudes in the left angular gyrus. Our results provide the first evidence that facial age has a distinct representation pattern in the posterior human brain. We propose that particular face-sensitive nodes interact with additional object-unselective quantification modules to obtain individual estimates of facial age. This brain network processing the age of faces differs from the cortical areas that have previously been linked to less developmental but instantly changeable face aspects. Our probabilistic method of associating activations with connectivity patterns reveals an exemplary link that can be used to further study, assess and quantify structure-function relationships.