Retinotopy versus face selectivity in macaque visual cortex

Rapid event-related, BOLD fMRI, non-human primates (NHP): choose two out of three

Human functional magnetic resonance imaging (fMRI) typically employs the blood-oxygen-level-dependent (BOLD) contrast mechanism. In non-human primates (NHP), contrast enhancement is possible using monocrystalline iron-oxide nanoparticles (MION) contrast agent, which has a more temporally extended response function. However, using BOLD fMRI in NHP is desirable for interspecies comparison, and the BOLD signal’s faster response function promises to be beneficial for rapid event-related (rER) designs. Here, we used rER BOLD fMRI in macaque monkeys while viewing real-world images, and found visual responses and category selectivity consistent with previous studies. However, activity estimates were very noisy, suggesting that the lower contrast-to-noise ratio of BOLD, suboptimal behavioural performance, and motion artefacts, in combination, render rER BOLD fMRI challenging in NHP. Previous studies have shown that rER fMRI is possible in macaques with MION, despite MION’s prolonged response function. To understand this, we conducted simulations of the BOLD and MION response during rER, and found that no matter how fast the design, the greater amplitude of the MION response outweighs the contrast loss caused by greater temporal smoothing. We conclude that although any two of the three elements (rER, BOLD, NHP) have been shown to work well, the combination of all three is particularly challenging.

Universal Mechanisms and the Development of the Face Network: What You See Is What You Get

Our assignment was to review the development of the face-processing network, an assignment that carries the presupposition that a face-specific developmental program exists. We hope to cast some doubt on this assumption and instead argue that the development of face processing is guided by the same ubiquitous rules that guide the development of cortex in general.

Idiosyncratic, Retinotopic Bias in Face Identification Modulated by Familiarity

The perception of gender and age of unfamiliar faces is reported to vary idiosyncratically across retinal locations such that, for example, the same androgynous face may appear to be male at one location but female at another. Here, we test spatial heterogeneity for the recognition of the identity of personally familiar faces in human participants. We found idiosyncratic biases that were stable within participants and that varied more across locations for low as compared to high familiar faces. These data suggest that like face gender and age, face identity is processed, in part, by independent populations of neurons monitoring restricted spatial regions and that the recognition responses vary for the same face across these different locations. Moreover, repeated and varied social interactions appear to lead to adjustments of these independent face recognition neurons so that the same familiar face is eventually more likely to elicit the same recognition response across widely separated visual field locations. We provide a mechanistic account of this reduced retinotopic bias based on computational simulations.

Mid-level visual features underlie the high-level categorical organization of the ventral stream

Human object-selective cortex shows a large-scale organization characterized by the high-level properties of both animacy and object size. To what extent are these neural responses explained by primitive perceptual features that distinguish animals from objects and big objects from small objects? To address this question, we used a texture synthesis algorithm to create a class of stimuli-texforms-which preserve some mid-level texture and form information from objects while rendering them unrecognizable. We found that unrecognizable texforms were sufficient to elicit the large-scale organizations of object-selective cortex along the entire ventral pathway. Further, the structure in the neural patterns elicited by texforms was well predicted by curvature features and by intermediate layers of a deep convolutional neural network, supporting the mid-level nature of the representations. These results provide clear evidence that a substantial portion of ventral stream organization can be accounted for by coarse texture and form information without requiring explicit recognition of intact objects.

The functional neuroanatomy of face perception: from brain measurements to deep neural networks

A central goal in neuroscience is to understand how processing within the ventral visual stream enables rapid and robust perception and recognition. Recent neuroscientific discoveries have significantly advanced understanding of the function, structure and computations along the ventral visual stream that serve as the infrastructure supporting this behaviour. In parallel, significant advances in computational models, such as hierarchical deep neural networks (DNNs), have brought machine performance to a level that is commensurate with human performance. Here, we propose a new framework using the ventral face network as a model system to illustrate how increasing the neural accuracy of present DNNs may allow researchers to test the computational benefits of the functional architecture of the human brain. Thus, the review (i) considers specific neural implementational features of the ventral face network, (ii) describes similarities and differences between the functional architecture of the brain and DNNs, and (iii) provides a hypothesis for the computational value of implementational features within the brain that may improve DNN performance. Importantly, this new framework promotes the incorporation of neuroscientific findings into DNNs in order to test the computational benefits of fundamental organizational features of the visual system.

The Organization and Operation of Inferior Temporal Cortex

Inferior temporal cortex (IT) is a key part of the ventral visual pathway implicated in object, face, and scene perception. But how does IT work? Here, I describe an organizational scheme that marries form and function and provides a framework for future research. The scheme consists of a series of stages arranged along the posterior-anterior axis of IT, defined by anatomical connections and functional responses. Each stage comprises a complement of subregions that have a systematic spatial relationship. The organization of each stage is governed by an eccentricity template, and corresponding eccentricity representations across stages are interconnected. Foveal representations take on a role in high-acuity object vision (including face recognition); intermediate representations compute other aspects of object vision such as behavioral valence (using color and surface cues); and peripheral representations encode information about scenes. This multistage, parallel-processing model invokes an innately determined organization refined by visual experience that is consistent with principles of cortical development. The model is also consistent with principles of evolution, which suggest that visual cortex expanded through replication of retinotopic areas. Finally, the model predicts that the most extensively studied network within IT-the face patches-is not unique but rather one manifestation of a canonical set of operations that reveal general principles of how IT works.

Face Processing Systems: From Neurons to Real-World Social Perception

Primate face processing depends on a distributed network of interlinked face-selective areas composed of face-selective neurons. In both humans and macaques, the network is divided into a ventral stream and a dorsal stream, and the functional similarities of the areas in humans and macaques indicate they are homologous. Neural correlates for face detection, holistic processing, face space, and other key properties of human face processing have been identified at the single neuron level, and studies providing causal evidence have established firmly that face-selective brain areas are central to face processing. These mechanisms give rise to our highly accurate familiar face recognition but also to our error-prone performance with unfamiliar faces. This limitation of the face system has important implications for consequential situations such as eyewitness identification and policing.

A hierarchical, retinotopic proto-organization of the primate visual system at birth

The adult primate visual system comprises a series of hierarchically organized areas. Each cortical area contains a topographic map of visual space, with different areas extracting different kinds of information from the retinal input. Here we asked to what extent the newborn visual system resembles the adult organization. We find that hierarchical, topographic organization is present at birth and therefore constitutes a proto-organization for the entire primate visual system. Even within inferior temporal cortex, this proto-organization was already present, prior to the emergence of category selectivity (e.g., faces or scenes). We propose that this topographic organization provides the scaffolding for the subsequent development of visual cortex that commences at the onset of visual experience.

A causal relationship between face-patch activity and face-detection behavior

The primate brain contains distinct areas densely populated by face-selective neurons. One of these, face-patch ML, contains neurons selective for contrast relationships between face parts. Such contrast-relationships can serve as powerful heuristics for face detection. However, it is unknown whether neurons with such selectivity actually support face-detection behavior. Here, we devised a naturalistic face-detection task and combined it with fMRI-guided pharmacological inactivation of ML to test whether ML is of critical importance for real-world face detection. We found that inactivation of ML impairs face detection. The effect was anatomically specific, as inactivation of areas outside ML did not affect face detection, and it was categorically specific, as inactivation of ML impaired face detection while sparing body and object detection. These results establish that ML function is crucial for detection of faces in natural scenes, performing a critical first step on which other face processing operations can build.

DOI:http://dx.doi.org/10.7554/eLife.18558.001

Characterizing object- and position-dependent response profiles to uni- and bilateral stimulus configurations in human higher visual cortex: a 7T fMRI study

Visual scenes are initially processed via segregated neural pathways dedicated to either of the two visual hemifields. Although higher-order visual areas are generally believed to utilize invariant object representations (abstracted away from features such as stimulus position), recent findings suggest they retain more spatial information than previously thought. Here, we assessed the nature of such higher-order object representations in human cortex using high-resolution fMRI at 7T, supported by corroborative 3T data. We show that multi-voxel activation patterns in both the contra- and ipsilateral hemisphere can be exploited to successfully classify the object category of unilaterally presented stimuli. Moreover, robustly identified rank order-based response profiles demonstrated a strong contralateral bias which frequently outweighed object category preferences. Finally, we contrasted different combinatorial operations to predict the responses during bilateral stimulation conditions based on responses to their constituent unilateral elements. Results favored a max operation predominantly reflecting the contralateral stimuli. The current findings extend previous work by showing that configuration-dependent modulations in higher-order visual cortex responses as observed in single unit activity have a counterpart in human neural population coding. They furthermore corroborate the emerging view that position coding is a fundamental functional characteristic of ventral visual stream processing.

Two Visual Pathways in Primates Based on Sampling of Space: Exploitation and Exploration of Visual Information

Evidence is strong that the visual pathway is segregated into two distinct streams—ventral and dorsal. Two proposals theorize that the pathways are segregated in function: The ventral stream processes information about object identity, whereas the dorsal stream, according to one model, processes information about either object location, and according to another, is responsible in executing movements under visual control. The models are influential; however recent experimental evidence challenges them, e.g., the ventral stream is not solely responsible for object recognition; conversely, its function is not strictly limited to object vision; the dorsal stream is not responsible by itself for spatial vision or visuomotor control; conversely, its function extends beyond vision or visuomotor control. In their place, we suggest a robust dichotomy consisting of a ventral stream selectively sampling high-resolution/focal spaces, and a dorsal stream sampling nearly all of space with reduced foveal bias. The proposal hews closely to the theme of embodied cognition: Function arises as a consequence of an extant sensory underpinning. A continuous, not sharp, segregation based on function emerges, and carries with it an undercurrent of an exploitation-exploration dichotomy. Under this interpretation, cells of the ventral stream, which individually have more punctate receptive fields that generally include the fovea or parafovea, provide detailed information about object shapes and features and lead to the systematic exploitation of said information; cells of the dorsal stream, which individually have large receptive fields, contribute to visuospatial perception, provide information about the presence/absence of salient objects and their locations for novel exploration and subsequent exploitation by the ventral stream or, under certain conditions, the dorsal stream. We leverage the dichotomy to unify neuropsychological cases under a common umbrella, account for the increased prevalence of multisensory integration in the dorsal stream under a Bayesian framework, predict conditions under which object recognition utilizes the ventral or dorsal stream, and explain why cells of the dorsal stream drive sensorimotor control and motion processing and have poorer feature selectivity. Finally, the model speculates on a dynamic interaction between the two streams that underscores a unified, seamless perception. Existing theories are subsumed under our proposal.

Position selectivity in face-sensitive visual cortex to facial and nonfacial stimuli: an fMRI study

BACKGROUND

Evidence for position sensitivity in object-selective visual areas has been building. On one hand, most of the relevant studies have utilized stimuli for which the areas are optimally selective and examine small sections of cortex. On the other hand, visual field maps established with nonspecific stimuli have been found in increasingly large areas of visual cortex, though generally not in areas primarily responsive to faces.

METHODS

fMRI was used to study the position sensitivity of the occipital face area (OFA) and the fusiform face area (FFA) to both standard rotating wedge retinotopic mapping stimuli and quadrant presentations of synthetic facial stimuli. Analysis methods utilized were both typical, that is, mean univariate BOLD signals and multivoxel pattern analysis (MVPA), and novel, that is, distribution of voxels to pattern classifiers and use of responses to nonfacial retinotopic mapping stimuli to classify responses to facial stimuli.

RESULTS

Polar angle sensitivity was exhibited to standard retinotopic mapping stimuli with a stronger contralateral bias in OFA than in FFA, a stronger bias toward the vertical meridian in FFA than in OFA, and a bias across both areas toward the inferior visual field. Contralateral hemispheric lateralization of both areas was again shown using synthetic face stimuli based on univariate BOLD signals, MVPA, and the biased contribution of voxels toward multivariate classifiers discriminating the contralateral visual field. Classifiers based on polar angle responsivity were used to classify the patterns of activation above chance levels to face stimuli in the OFA but not in the FFA.

CONCLUSIONS

Both the OFA and FFA exhibit quadrant sensitivity to face stimuli, though the OFA exhibits greater position responsivity across stimuli than the FFA and includes overlap in the response pattern to the disparate stimulus types. Such biases are consistent with varying position sensitivity along different surfaces of occipito-temporal cortex.

Functional architecture for disparity in macaque inferior temporal cortex and its relationship to the architecture for faces, color, scenes, and visual field

Binocular disparity is a powerful depth cue for object perception. The computations for object vision culminate in inferior temporal cortex (IT), but the functional organization for disparity in IT is unknown. Here we addressed this question by measuring fMRI responses in alert monkeys to stimuli that appeared in front of (near), behind (far), or at the fixation plane. We discovered three regions that showed preferential responses for near and far stimuli, relative to zero-disparity stimuli at the fixation plane. These "near/far" disparity-biased regions were located within dorsal IT, as predicted by microelectrode studies, and on the posterior inferotemporal gyrus. In a second analysis, we instead compared responses to near stimuli with responses to far stimuli and discovered a separate network of "near" disparity-biased regions that extended along the crest of the superior temporal sulcus. We also measured in the same animals fMRI responses to faces, scenes, color, and checkerboard annuli at different visual field eccentricities. Disparity-biased regions defined in either analysis did not show a color bias, suggesting that disparity and color contribute to different computations within IT. Scene-biased regions responded preferentially to near and far stimuli (compared with stimuli without disparity) and had a peripheral visual field bias, whereas face patches had a marked near bias and a central visual field bias. These results support the idea that IT is organized by a coarse eccentricity map, and show that disparity likely contributes to computations associated with both central (face processing) and peripheral (scene processing) visual field biases, but likely does not contribute much to computations within IT that are implicated in processing color.

Faciotopy-A face-feature map with face-like topology in the human occipital face area

The occipital face area (OFA) and fusiform face area (FFA) are brain regions thought to be specialized for face perception. However, their intrinsic functional organization and status as cortical areas with well-defined boundaries remains unclear. Here we test these regions for “faciotopy”, a particular hypothesis about their intrinsic functional organisation. A faciotopic area would contain a face-feature map on the cortical surface, where cortical patches represent face features and neighbouring patches represent features that are physically neighbouring in a face. The faciotopy hypothesis is motivated by the idea that face regions might develop from a retinotopic protomap and acquire their selectivity for face features through natural visual experience. Faces have a prototypical configuration of features, are usually perceived in a canonical upright orientation, and are frequently fixated in particular locations. To test the faciotopy hypothesis, we presented images of isolated face features at fixation to subjects during functional magnetic resonance imaging. The responses in V1 were best explained by low-level image properties of the stimuli. OFA, and to a lesser degree FFA, showed evidence for faciotopic organization. When a single patch of cortex was estimated for each face feature, the cortical distances between the feature patches reflected the physical distance between the features in a face. Faciotopy would be the first example, to our knowledge, of a cortical map reflecting the topology, not of a part of the organism itself (its retina in retinotopy, its body in somatotopy), but of an external object of particular perceptual significance.

Single-unit recordings in the macaque face patch system reveal limitations of fMRI MVPA

Multivariate pattern analysis (MVPA) of fMRI data has become an important technique for cognitive neuroscientists in recent years; however, the relationship between fMRI MVPA and the underlying neural population activity remains unexamined. Here, we performed MVPA of fMRI data and single-unit data in the same species, the macaque monkey. Facial recognition in the macaque is subserved by a well characterized system of cortical patches, which provided the test bed for our comparison. We showed that neural population information about face viewpoint was readily accessible with fMRI MVPA from all face patches, in agreement with single-unit data. Information about face identity, although it was very strongly represented in the populations of units of the anterior face patches, could not be retrieved from the same data. The discrepancy was especially striking in patch AL, where neurons encode both the identity and viewpoint of human faces. From an analysis of the characteristics of the neural representations for viewpoint and identity, we conclude that fMRI MVPA cannot decode information contained in the weakly clustered neuronal responses responsible for coding the identity of human faces in the macaque brain. Although further studies are needed to elucidate the relationship between information decodable from fMRI multivoxel patterns versus single-unit populations for other variables in other brain regions, our result has important implications for the interpretation of negative findings in fMRI multivoxel pattern analyses.

Monkey cortex through fMRI glasses

In 1998 several groups reported the feasibility of fMRI experiments in monkeys, with the goal to bridge the gap between invasive nonhuman primate studies and human functional imaging. These studies yielded critical insights in the neuronal underpinnings of the BOLD signal. Furthermore, the technology has been successful in guiding electrophysiological recordings and identifying focal perturbation targets. Finally, invaluable information was obtained concerning human brain evolution. We here provide a comprehensive overview of awake monkey fMRI studies mainly confined to the visual system. We review the latest insights about the topographic organization of monkey visual cortex and discuss the spatial relationships between retinotopy and category- and feature-selective clusters. We briefly discuss the functional layout of parietal and frontal cortex and continue with a summary of some fascinating functional and effective connectivity studies. Finally, we review recent comparative fMRI experiments and speculate about the future of nonhuman primate imaging.