Representation of Regular and Irregular Shapes in Macaque Inferotemporal Cortex

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis

Object recognition relies on different transformations of the retinal input, carried out by the visual system, that range from local contrast to object shape and category. While some of those transformations are thought to occur at specific stages of the visual hierarchy, the features they represent are correlated (e.g., object shape and identity) and selectivity for the same feature overlaps in many brain regions. This may be explained either by collinearity across representations or may instead reflect the coding of multiple dimensions by the same cortical population. Moreover, orthogonal and shared components may differently impact distinctive stages of the visual hierarchy. We recorded functional MRI activity while participants passively attended to object images and employed a statistical approach that partitioned orthogonal and shared object representations to reveal their relative impact on brain processing. Orthogonal shape representations (silhouette, curvature, and medial axis) independently explained distinct and overlapping clusters of selectivity in the occitotemporal and parietal cortex. Moreover, we show that the relevance of shared representations linearly increases moving from posterior to anterior regions. These results indicate that the visual cortex encodes shared relations between different features in a topographic fashion and that object shape is encoded along different dimensions, each representing orthogonal features.NEW & NOTEWORTHY There are several possible ways of characterizing the shape of an object. Which shape description better describes our brain responses while we passively perceive objects? Here, we employed three competing shape models to explain brain representations when viewing real objects. We found that object shape is encoded in a multidimensional fashion and thus defined by the interaction of multiple features.

Face Representations via Tensorfaces of Various Complexities

Neurons selective for faces exist in humans and monkeys. However, characteristics of face cell receptive fields are poorly understood. In this theoretical study, we explore the effects of complexity, defined as algorithmic information (Kolmogorov complexity) and logical depth, on possible ways that face cells may be organized. We use tensor decompositions to decompose faces into a set of components, called tensorfaces, and their associated weights, which can be interpreted as model face cells and their firing rates. These tensorfaces form a high-dimensional representation space in which each tensorface forms an axis of the space. A distinctive feature of the decomposition algorithm is the ability to specify tensorface complexity. We found that low-complexity tensorfaces have blob-like appearances crudely approximating faces, while high-complexity tensorfaces appear clearly face-like. Low-complexity tensorfaces require a larger population to reach a criterion face reconstruction error than medium- or high-complexity tensorfaces, and thus are inefficient by that criterion. Low-complexity tensorfaces, however, generalize better when representing statistically novel faces, which are faces falling beyond the distribution of face description parameters found in the tensorface training set. The degree to which face representations are parts based or global forms a continuum as a function of tensorface complexity, with low and medium tensorfaces being more parts based. Given the computational load imposed in creating high-complexity face cells (in the form of algorithmic information and logical depth) and in the absence of a compelling advantage to using high-complexity cells, we suggest face representations consist of a mixture of low- and medium-complexity face cells.

Extrinsic and intrinsic representations

We extend the discussion in the target article about distinctions between extrinsic coding (external references to known things, as required by information theory) and the alternative we and the target article both favor, intrinsic coding (internal relationships within sensory and motor signals). Central to our thinking about intrinsic coding is population coding and the concept of high-dimensional neural response spaces.

Representation of shape, space, and attention in monkey cortex

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both 'what' and 'where' cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.

Effective Dimensionality Reduction for Visualizing Neural Dynamics by Laplacian Eigenmaps

With the development of neural recording technology, it has become possible to collect activities from hundreds or even thousands of neurons simultaneously. Visualization of neural population dynamics can help neuroscientists analyze large-scale neural activities efficiently. In this letter, Laplacian eigenmaps is applied to this task for the first time, and the experimental results show that the proposed method significantly outperforms the commonly used methods. This finding was confirmed by the systematic evaluation using nonhuman primate data, which contained the complex dynamics well suited for testing. According to our results, Laplacian eigenmaps is better than the other methods in various ways and can clearly visualize interesting biological phenomena related to neural dynamics.

Surface diagnosticity predicts the high-level representation of regular and irregular object shape in human vision

The human visual system has an extraordinary capacity to compute three-dimensional (3D) shape structure for both geometrically regular and irregular objects. The goal of this study was to shed new light on the underlying representational structures that support this ability. Observers (N = 85) completed two complementary perceptual tasks. Experiment 1 involved whole–part matching of image parts to whole geometrically regular and irregular novel object shapes. Image parts comprised either regions of edge contour, volumetric parts, or surfaces. Performance was better for irregular than for regular objects and interacted with part type: volumes yielded better matching performance than surfaces for regular but not for irregular objects. The basis for this effect was further explored in Experiment 2, which used implicit part–whole repetition priming. Here, we orthogonally manipulated shape regularity and a new factor of surface diagnosticity (how predictive a single surface is of object identity). The results showed that surface diagnosticity, not object shape regularity, determined the differential processing of volumes and surfaces. Regardless of shape regularity, objects with low surface diagnosticity were better primed by volumes than by surfaces. In contrast, objects with high surface diagnosticity showed the opposite pattern. These findings are the first to show that surface diagnosticity plays a fundamental role in object recognition. We propose that surface-based shape primitives—rather than volumetric parts—underlie the derivation of 3D object shape in human vision.

Grasping Neurons in the Ventral Premotor Cortex of Macaques Are Modulated by Social Goals

Although it is established that F5 neurons can distinguish between nonsocial goals such as bringing food to the mouth for eating or placing it in a container, it is not clear whether they discriminate between social and nonsocial goals. Here, we recorded single-unit activity in the ventral premotor cortex of two female macaques and used a simple reach-to-grasp motor task in which a monkey grasped an object with a precision grip in three conditions, which only differed in terms of their final goal, that is, a subsequent motor act that was either social (placing in the experimenter's hand ["Hand" condition]) or nonsocial (placing in a container ["Container" condition] or bringing to the mouth for eating ["Mouth" condition]). We found that, during the execution of the grasping motor act, the response of a sizable proportion of F5 motor neurons was modulated by the final goal of the action, with some having a preference for the social goal condition. Our results reveal that the representation of goal-directed actions in ventral premotor cortex is influenced by contextual information not only extracted from physical cues but also from cues endowed with biological or social value. Our study suggests that the activity of grasping neurons in the premotor cortex is modulated by social context.

Representations of regular and irregular shapes by deep Convolutional Neural Networks, monkey inferotemporal neurons and human judgments

Recent studies suggest that deep Convolutional Neural Network (CNN) models show higher representational similarity, compared to any other existing object recognition models, with macaque inferior temporal (IT) cortical responses, human ventral stream fMRI activations and human object recognition. These studies employed natural images of objects. A long research tradition employed abstract shapes to probe the selectivity of IT neurons. If CNN models provide a realistic model of IT responses, then they should capture the IT selectivity for such shapes. Here, we compare the activations of CNN units to a stimulus set of 2D regular and irregular shapes with the response selectivity of macaque IT neurons and with human similarity judgements. The shape set consisted of regular shapes that differed in nonaccidental properties, and irregular, asymmetrical shapes with curved or straight boundaries. We found that deep CNNs (Alexnet, VGG-16 and VGG-19) that were trained to classify natural images show response modulations to these shapes that were similar to those of IT neurons. Untrained CNNs with the same architecture than trained CNNs, but with random weights, demonstrated a poorer similarity than CNNs trained in classification. The difference between the trained and untrained CNNs emerged at the deep convolutional layers, where the similarity between the shape-related response modulations of IT neurons and the trained CNNs was high. Unlike IT neurons, human similarity judgements of the same shapes correlated best with the last layers of the trained CNNs. In particular, these deepest layers showed an enhanced sensitivity for straight versus curved irregular shapes, similar to that shown in human shape judgments. In conclusion, the representations of abstract shape similarity are highly comparable between macaque IT neurons and deep convolutional layers of CNNs that were trained to classify natural images, while human shape similarity judgments correlate better with the deepest layers.

The primate inferior temporal (IT) cortex is considered to be the final stage of visual processing that allows for object recognition, identification and categorization of objects. Electrophysiology studies suggest that an object’s shape is a strong determinant of the neuronal response patterns in IT. Here we examine whether deep Convolutional Neural Networks (CNNs), that were trained to classify natural images of objects, show response modulations for abstract shapes similar to those of macaque IT neurons. For trained and untrained versions of three state-of-the-art CNNs, we assessed the response modulations for a set of 2D shapes at each of their stages and compared these to those of a population of macaque IT neurons and human shape similarity judgements. We show that an IT-like representation of similarity amongst 2D abstract shapes develops in the deep convolutional CNN layers when these are trained to classify natural images. Our results reveal a high correspondence between the representation of shape similarity of deep trained CNN stages and macaque IT neurons and an analogous correspondence of the last trained CNN stages with shape similarity as judged by humans.

Non-accidental properties, metric invariance, and encoding by neurons in a model of ventral stream visual object recognition, VisNet

When objects transform into different views, some properties are maintained, such as whether the edges are convex or concave, and these non-accidental properties are likely to be important in view-invariant object recognition. The metric properties, such as the degree of curvature, may change with different views, and are less likely to be useful in object recognition. It is shown that in a model of invariant visual object recognition in the ventral visual stream, VisNet, non-accidental properties are encoded much more than metric properties by neurons. Moreover, it is shown how with the temporal trace rule training in VisNet, non-accidental properties of objects become encoded by neurons, and how metric properties are treated invariantly. We also show how VisNet can generalize between different objects if they have the same non-accidental property, because the metric properties are likely to overlap. VisNet is a 4-layer unsupervised model of visual object recognition trained by competitive learning that utilizes a temporal trace learning rule to implement the learning of invariance using views that occur close together in time. A second crucial property of this model of object recognition is, when neurons in the level corresponding to the inferior temporal visual cortex respond selectively to objects, whether neurons in the intermediate layers can respond to combinations of features that may be parts of two or more objects. In an investigation using the four sides of a square presented in every possible combination, it was shown that even though different layer 4 neurons are tuned to encode each feature or feature combination orthogonally, neurons in the intermediate layers can respond to features or feature combinations present is several objects. This property is an important part of the way in which high capacity can be achieved in the four-layer ventral visual cortical pathway. These findings concerning non-accidental properties and the use of neurons in intermediate layers of the hierarchy help to emphasise fundamental underlying principles of the computations that may be implemented in the ventral cortical visual stream used in object recognition.

What Is Actually Affected by the Scrambling of Objects When Localizing the Lateral Occipital Complex?

The lateral occipital complex (LOC), the cortical region critical for shape perception, is localized with fMRI by its greater BOLD activity when viewing intact objects compared with their scrambled versions (resembling texture). Despite hundreds of studies investigating LOC, what the LOC localizer accomplishes—beyond distinguishing shape from texture—has never been resolved. By independently scattering the intact parts of objects, the axis structure defining the relations between parts was no longer defined. This led to a diminished BOLD response, despite the increase in the number of independent entities (the parts) produced by the scattering, thus indicating that LOC specifies interpart relations, in addition to specifying the shape of the parts themselves. LOC's sensitivity to relations is not confined to those between parts but is also readily apparent between objects, rendering it—and not subsequent “place” areas—as the critical region for the representation of scenes. Moreover, that these effects are witnessed with novel as well as familiar intact objects and scenes suggests that the relations are computed on the fly, rather than being retrieved from memory.

Human Amygdala Represents the Complete Spectrum of Subjective Valence

Although the amygdala is a major locus for hedonic processing, how it encodes valence information is poorly understood. Given the hedonic potency of odor stimuli and the amygdala's anatomical proximity to the peripheral olfactory system, we combined high-resolution fMRI with pattern-based multivariate techniques to examine how valence information is encoded in the amygdala. Ten human subjects underwent fMRI scanning while smelling 9 odorants that systematically varied in perceived valence. Representational similarity analyses showed that amygdala codes the entire dimension of valence, ranging from pleasantness to unpleasantness. This unidimensional representation significantly correlated with self-reported valence ratings but not with intensity ratings. Furthermore, within-trial valence representations evolved over time, prioritizing earlier differentiation of unpleasant stimuli. Together, these findings underscore the idea that both spatial and temporal features uniquely encode pleasant and unpleasant odor valence in the amygdala. The availability of a unidimensional valence code in the amygdala, distributed in both space and time, would create greater flexibility in determining the pleasantness or unpleasantness of stimuli, providing a mechanism by which expectation, context, attention, and learning could influence affective boundaries for guiding behavior.

SIGNIFICANCE STATEMENT

Our findings elucidate the mechanisms of affective processing in the amygdala by demonstrating that this brain region represents the entire valence dimension from pleasant to unpleasant. An important implication of this unidimensional valence code is that pleasant and unpleasant valence cannot coexist in the amygdale because overlap of fMRI ensemble patterns for these two valence extremes obscures their unique content. This functional architecture, whereby subjective valence maps onto a pattern continuum between pleasant and unpleasant poles, offers a robust mechanism by which context, expectation, and experience could alter the set-point for valence-based behavior. Finally, identification of spatial and temporal differentiation of valence in amygdala may shed new insights into individual differences in emotional responding, with potential relevance for affective disorders.

Spike train SIMilarity Space (SSIMS): a framework for single neuron and ensemble data analysis

Increased emphasis on circuit level activity in the brain makes it necessary to have methods to visualize and evaluate large-scale ensemble activity beyond that revealed by raster-histograms or pairwise correlations. We present a method to evaluate the relative similarity of neural spiking patterns by combining spike train distance metrics with dimensionality reduction. Spike train distance metrics provide an estimate of similarity between activity patterns at multiple temporal resolutions. Vectors of pair-wise distances are used to represent the intrinsic relationships between multiple activity patterns at the level of single units or neuronal ensembles. Dimensionality reduction is then used to project the data into concise representations suitable for clustering analysis as well as exploratory visualization. Algorithm performance and robustness are evaluated using multielectrode ensemble activity data recorded in behaving primates. We demonstrate how spike train SIMilarity space (SSIMS) analysis captures the relationship between goal directions for an eight-directional reaching task and successfully segregates grasp types in a 3D grasping task in the absence of kinematic information. The algorithm enables exploration of virtually any type of neural spiking (time series) data, providing similarity-based clustering of neural activity states with minimal assumptions about potential information encoding models.

A conceptual framework of computations in mid-level vision

If a picture is worth a thousand words, as an English idiom goes, what should those words-or, rather, descriptors-capture? What format of image representation would be sufficiently rich if we were to reconstruct the essence of images from their descriptors? In this paper, we set out to develop a conceptual framework that would be: (i) biologically plausible in order to provide a better mechanistic understanding of our visual system; (ii) sufficiently robust to apply in practice on realistic images; and (iii) able to tap into underlying structure of our visual world. We bring forward three key ideas. First, we argue that surface-based representations are constructed based on feature inference from the input in the intermediate processing layers of the visual system. Such representations are computed in a largely pre-semantic (prior to categorization) and pre-attentive manner using multiple cues (orientation, color, polarity, variation in orientation, and so on), and explicitly retain configural relations between features. The constructed surfaces may be partially overlapping to compensate for occlusions and are ordered in depth (figure-ground organization). Second, we propose that such intermediate representations could be formed by a hierarchical computation of similarity between features in local image patches and pooling of highly-similar units, and reestimated via recurrent loops according to the task demands. Finally, we suggest to use datasets composed of realistically rendered artificial objects and surfaces in order to better understand a model's behavior and its limitations.

Mechanisms of object recognition: what we have learned from pigeons

Behavioral studies of object recognition in pigeons have been conducted for 50 years, yielding a large body of data. Recent work has been directed toward synthesizing this evidence and understanding the visual, associative, and cognitive mechanisms that are involved. The outcome is that pigeons are likely to be the non-primate species for which the computational mechanisms of object recognition are best understood. Here, we review this research and suggest that a core set of mechanisms for object recognition might be present in all vertebrates, including pigeons and people, making pigeons an excellent candidate model to study the neural mechanisms of object recognition. Behavioral and computational evidence suggests that error-driven learning participates in object category learning by pigeons and people, and recent neuroscientific research suggests that the basal ganglia, which are homologous in these species, may implement error-driven learning of stimulus-response associations. Furthermore, learning of abstract category representations can be observed in pigeons and other vertebrates. Finally, there is evidence that feedforward visual processing, a central mechanism in models of object recognition in the primate ventral stream, plays a role in object recognition by pigeons. We also highlight differences between pigeons and people in object recognition abilities, and propose candidate adaptive specializations which may explain them, such as holistic face processing and rule-based category learning in primates. From a modern comparative perspective, such specializations are to be expected regardless of the model species under study. The fact that we have a good idea of which aspects of object recognition differ in people and pigeons should be seen as an advantage over other animal models. From this perspective, we suggest that there is much to learn about human object recognition from studying the "simple" brains of pigeons.

Fine-grained stimulus representations in body selective areas of human occipito-temporal cortex

Neurophysiological and functional imaging studies have investigated the representation of animate and inanimate stimulus classes in monkey inferior temporal (IT) and human occipito-temporal cortex (OTC). These studies proposed a distributed representation of stimulus categories across IT and OTC and at the same time highlighted category specific modules for the processing of bodies, faces and objects. Here, we investigated whether the stimulus representation within the extrastriate (EBA) and the fusiform (FBA) body areas differed from the representation across OTC. To address this question, we performed an event-related fMRI experiment, evaluating the pattern of activation elicited by 200 individual stimuli that had already been extensively tested in our earlier monkey imaging and single cell studies (Popivanov et al., 2012, 2014). The set contained achromatic images of headless monkey and human bodies, two sets of man-made objects, monkey and human faces, four-legged mammals, birds, fruits, and sculptures. The fMRI response patterns within EBA and FBA primarily distinguished bodies from non-body stimuli, with subtle differences between the areas. However, despite responding on average stronger to bodies than to other categories, classification performance for preferred and non-preferred categories was comparable. OTC primarily distinguished animate from inanimate stimuli. However, cluster analysis revealed a much more fine-grained representation with several homogeneous clusters consisting entirely of stimuli of individual categories. Overall, our data suggest that category representation varies with location within OTC. Nevertheless, body modules contain information to discriminate also non-preferred stimuli and show an increasing specificity in a posterior to anterior gradient.

Curvature-processing network in macaque visual cortex

Our visual environment abounds with curved features. Thus, the goal of understanding visual processing should include the processing of curved features. Using functional magnetic resonance imaging in behaving monkeys, we demonstrated a network of cortical areas selective for the processing of curved features. This network includes three distinct hierarchically organized regions within the ventral visual pathway: a posterior curvature-biased patch (PCP) located in the near-foveal representation of dorsal V4, a middle curvature-biased patch (MCP) located on the ventral lip of the posterior superior temporal sulcus (STS) in area TEO, and an anterior curvature-biased patch (ACP) located just below the STS in anterior area TE. Our results further indicate that the processing of curvature becomes increasingly complex from PCP to ACP. The proximity of the curvature-processing network to the well-known face-processing network suggests a possible functional link between them.

A toolbox for representational similarity analysis

Neuronal population codes are increasingly being investigated with multivariate pattern-information analyses. A key challenge is to use measured brain-activity patterns to test computational models of brain information processing. One approach to this problem is representational similarity analysis (RSA), which characterizes a representation in a brain or computational model by the distance matrix of the response patterns elicited by a set of stimuli. The representational distance matrix encapsulates what distinctions between stimuli are emphasized and what distinctions are de-emphasized in the representation. A model is tested by comparing the representational distance matrix it predicts to that of a measured brain region. RSA also enables us to compare representations between stages of processing within a given brain or model, between brain and behavioral data, and between individuals and species. Here, we introduce a Matlab toolbox for RSA. The toolbox supports an analysis approach that is simultaneously data- and hypothesis-driven. It is designed to help integrate a wide range of computational models into the analysis of multichannel brain-activity measurements as provided by modern functional imaging and neuronal recording techniques. Tools for visualization and inference enable the user to relate sets of models to sets of brain regions and to statistically test and compare the models using nonparametric inference methods. The toolbox supports searchlight-based RSA, to continuously map a measured brain volume in search of a neuronal population code with a specific geometry. Finally, we introduce the linear-discriminant t value as a measure of representational discriminability that bridges the gap between linear decoding analyses and RSA. In order to demonstrate the capabilities of the toolbox, we apply it to both simulated and real fMRI data. The key functions are equally applicable to other modalities of brain-activity measurement. The toolbox is freely available to the community under an open-source license agreement (http://www.mrc-cbu.cam.ac.uk/methods-and-resources/toolboxes/license/).

Recovering stimulus locations using populations of eye-position modulated neurons in dorsal and ventral visual streams of non-human primates

We recorded visual responses while monkeys fixated the same target at different gaze angles, both dorsally (lateral intraparietal cortex, LIP) and ventrally (anterior inferotemporal cortex, AIT). While eye-position modulations occurred in both areas, they were both more frequent and stronger in LIP neurons. We used an intrinsic population decoding technique, multidimensional scaling (MDS), to recover eye positions, equivalent to recovering fixated target locations. We report that eye-position based visual space in LIP was more accurate (i.e., metric). Nevertheless, the AIT spatial representation remained largely topologically correct, perhaps indicative of a categorical spatial representation (i.e., a qualitative description such as "left of" or "above" as opposed to a quantitative, metrically precise description). Additionally, we developed a simple neural model of eye position signals and illustrate that differences in single cell characteristics can influence the ability to recover target position in a population of cells. We demonstrate for the first time that the ventral stream contains sufficient information for constructing an eye-position based spatial representation. Furthermore we demonstrate, in dorsal and ventral streams as well as modeling, that target locations can be extracted directly from eye position signals in cortical visual responses without computing coordinate transforms of visual space.

Population coding and the labeling problem: extrinsic versus intrinsic representations

Current population coding methods, including weighted averaging and Bayesian estimation, are based on extrinsic representations. These require that neurons be labeled with response parameters, such as tuning curve peaks or noise distributions, which are tied to some external, world-based metric scale. Firing rates alone, without this external labeling, are insufficient to represent a variable. However, the extrinsic approach does not explain how such neural labeling is implemented. A radically different and perhaps more physiological approach is based on intrinsic representations, which have access only to firing rates. Because neurons are unlabeled, intrinsic coding represents relative, rather than absolute, values of a variable. We show that intrinsic coding has representational advantages, including invariance, categorization, and discrimination, and in certain situations it may also recover absolute stimulus values.

Human Object-Similarity Judgments Reflect and Transcend the Primate-IT Object Representation

Primate inferior temporal (IT) cortex is thought to contain a high-level representation of objects at the interface between vision and semantics. This suggests that the perceived similarity of real-world objects might be predicted from the IT representation. Here we show that objects that elicit similar activity patterns in human IT (hIT) tend to be judged as similar by humans. The IT representation explained the human judgments better than early visual cortex, other ventral-stream regions, and a range of computational models. Human similarity judgments exhibited category clusters that reflected several categorical divisions that are prevalent in the IT representation of both human and monkey, including the animate/inanimate and the face/body division. Human judgments also reflected the within-category representation of IT. However, the judgments transcended the IT representation in that they introduced additional categorical divisions. In particular, human judgments emphasized human-related additional divisions between human and non-human animals and between man-made and natural objects. hIT was more similar to monkey IT than to human judgments. One interpretation is that IT has evolved visual-feature detectors that distinguish between animates and inanimates and between faces and bodies because these divisions are fundamental to survival and reproduction for all primate species, and that other brain systems serve to more flexibly introduce species-dependent and evolutionarily more recent divisions.

The transfer of object learning across exemplars and their orientation is related to perceptual similarity

Recognition of objects improves after training. The exact characteristics of this visual learning process remain unclear. We examined to which extent object learning depends on the exact exemplar and orientation used during training. Participants were trained to name object pictures at as short a picture presentation time as possible. The required presentation time diminished over training. After training participants were tested with a completely new set of objects as well as with two variants of the trained object set, namely an orientation change and a change of the exact exemplar shown. Both manipulations led to a decrease in performance compared to the original picture set. Nevertheless, performance with the manipulated versions of the trained stimuli was better than performance with the completely new set, at least when only one manipulation was performed. Amount of transfer to new images of an object was related to perceptual similarity, but not to pixel overlap or to measurements of similarity in the different layers of a popular hierarchical object recognition model (HMAX). Thus, object learning generalizes only partially over changes in exemplars and orientation, which is consistent with the tuning properties of neurons in object-selective cortical regions and the role of perceptual similarity in these representations.

The nature of experience determines object representations in the visual system

Visual perceptual learning (PL) and perceptual expertise (PE) traditionally lead to different training effects and recruit different brain areas, but reasons for these differences are largely unknown. Here, we tested how the learning history influences visual object representations. Two groups were trained with tasks typically used in PL or PE studies, with the same novel objects, training duration and parafoveal stimulus presentation. We observed qualitatively different changes in the cortical representations of these objects following PL and PE training, replicating typical training effects in each field. These effects were also modulated by testing tasks, suggesting that experience interacts with attentional set and that the choice of testing tasks critically determines the pattern of training effects one can observe after a short-term visual training. Experience appears sufficient to account for prior differences in the neural locus of learning between PL and PE. The nature of the experience with an object's category can determine its representation in the visual system.

Sensitivity to nonaccidental properties across various shape dimensions

Nonaccidental properties (NAPs) are image properties that are invariant over orientation in depth and are distinguished from metric properties (MPs) that can change continuously with variations over depth orientation. To a large extent NAPs allow facile recognition of objects at novel viewpoints. Two match-to-sample experiments with 2D or 3D appearing geons assessed sensitivity to NAP vs. MP differences. A matching geon was always identical to the sample and the distractor differed from the matching geon in either a NAP or an MP on a single generalized cone dimension. For example, if the sample was a cylinder with a slightly curved axis, the NAP distractor would have a straight axis and the MP distractor would have an axis of greater curvature than the sample. Critically, the NAP and MP differences were scaled so that the MP differences were slightly greater according to pixel energy and Gabor wavelet measures of dissimilarity. Exp. 1 used a staircase procedure to determine the threshold presentation time required to achieve 75% accuracy. Exp. 2 used a constant, brief display presentation time with reaction times and error rates as dependent measures. Both experiments revealed markedly greater sensitivity to NAP over MP differences, and this was generally true for the individual dimensions. The NAP advantage was not reflected in the similarity computations of the C2 stage of HMAX, a widely cited model of later stage cortical ventral stream processing.

Stimulus dependency of object-evoked responses in human visual cortex: an inverse problem for category specificity

Many studies have linked the processing of different object categories to specific event-related potentials (ERPs) such as the face-specific N170. Despite reports showing that object-related ERPs are influenced by visual stimulus features, there is consensus that these components primarily reflect categorical aspects of the stimuli. Here, we re-investigated this idea by systematically measuring the effects of visual feature manipulations on ERP responses elicited by both structure-from-motion (SFM)-defined and luminance-defined object stimuli. SFM objects elicited a novel component at 200-250 ms (N250) over parietal and posterior temporal sites. We found, however, that the N250 amplitude was unaffected by restructuring SFM stimuli into meaningless objects based on identical visual cues. This suggests that this N250 peak was not uniquely linked to categorical aspects of the objects, but is strongly determined by visual stimulus features. We provide strong support for this hypothesis by parametrically manipulating the depth range of both SFM- and luminance-defined object stimuli and showing that the N250 evoked by SFM stimuli as well as the well-known N170 to static faces were sensitive to this manipulation. Importantly, this effect could not be attributed to compromised object categorization in low depth stimuli, confirming a strong impact of visual stimulus features on object-related ERP signals. As ERP components linked with visual categorical object perception are likely determined by multiple stimulus features, this creates an interesting inverse problem when deriving specific perceptual processes from variations in ERP components.

How does the brain solve visual object recognition?

Mounting evidence suggests that 'core object recognition,' the ability to rapidly recognize objects despite substantial appearance variation, is solved in the brain via a cascade of reflexive, largely feedforward computations that culminate in a powerful neuronal representation in the inferior temporal cortex. However, the algorithm that produces this solution remains poorly understood. Here we review evidence ranging from individual neurons and neuronal populations to behavior and computational models. We propose that understanding this algorithm will require using neuronal and psychophysical data to sift through many computational models, each based on building blocks of small, canonical subnetworks with a common functional goal.

What are the Visual Features Underlying Rapid Object Recognition?

Research progress in machine vision has been very significant in recent years. Robust face detection and identification algorithms are already readily available to consumers, and modern computer vision algorithms for generic object recognition are now coping with the richness and complexity of natural visual scenes. Unlike early vision models of object recognition that emphasized the role of figure-ground segmentation and spatial information between parts, recent successful approaches are based on the computation of loose collections of image features without prior segmentation or any explicit encoding of spatial relations. While these models remain simplistic models of visual processing, they suggest that, in principle, bottom-up activation of a loose collection of image features could support the rapid recognition of natural object categories and provide an initial coarse visual representation before more complex visual routines and attentional mechanisms take place. Focusing on biologically plausible computational models of (bottom-up) pre-attentive visual recognition, we review some of the key visual features that have been described in the literature. We discuss the consistency of these feature-based representations with classical theories from visual psychology and test their ability to account for human performance on a rapid object categorization task.

Similarity and categorization: from vision to touch

Even though human perceptual development relies on combining multiple modalities, most categorization studies so far have focused on the visual modality. To better understand the mechanisms underlying multisensory categorization, we analyzed visual and haptic perceptual spaces and compared them with human categorization behavior. As stimuli we used a three-dimensional object space of complex, parametrically-defined objects. First, we gathered similarity ratings for all objects and analyzed the perceptual spaces of both modalities using multidimensional scaling analysis. Next, we performed three different categorization tasks which are representative of every-day learning scenarios: in a fully unconstrained task, objects were freely categorized, in a semi-constrained task, exactly three groups had to be created, whereas in a constrained task, participants received three prototype objects and had to assign all other objects accordingly. We found that the haptic modality was on par with the visual modality both in recovering the topology of the physical space and in solving the categorization tasks. We also found that within-category similarity was consistently higher than across-category similarity for all categorization tasks and thus show how perceptual spaces based on similarity can explain visual and haptic object categorization. Our results suggest that both modalities employ similar processes in forming categories of complex objects.

From single cells to social perception

Research describing the cellular coding of faces in non-human primates often provides the underlying physiological framework for our understanding of face processing in humans. Models of face perception, explanations of perceptual after-effects from viewing particular types of faces, and interpretation of human neuroimaging data rely on monkey neurophysiological data and the assumption that neurophysiological responses of humans are comparable to those recorded in the non-human primate. Here, we review studies that describe cells that preferentially respond to faces, and assess the link between the physiological characteristics of single cells and social perception. Principally, we describe cells recorded from the non-human primate, although a limited number of cells have been recorded in humans, and are included in order to appraise the validity of non-human physiological data for our understanding of human face and social perception.

Encoding of complexity, shape, and curvature by macaque infero-temporal neurons

We recorded responses of macaque infero-temporal (IT) neurons to a stimulus set of Fourier boundary descriptor shapes wherein complexity, general shape, and curvature were systematically varied. We analyzed the response patterns of the neurons to the different stimuli using multidimensional scaling. The resulting neural shape space differed in important ways from the physical, image-based shape space. We found a particular sensitivity for the presence of curved versus straight contours that existed only for the simple but not for the medium and highly complex shapes. Also, IT neurons could linearly separate the simple and the complex shapes within a low-dimensional neural shape space, but no distinction was found between the medium and high levels of complexity. None of these effects could be derived from physical image metrics, either directly or by comparing the neural data with similarities yielded by two models of low-level visual processing (one using wavelet-based filters and one that models position and size invariant object selectivity through four hierarchically organized neural layers). This study highlights the relevance of complexity to IT neural encoding, both as a neurally independently represented shape property and through its influence on curvature detection.

The "parahippocampal place area" responds preferentially to high spatial frequencies in humans and monkeys

A visual brain area that is thought to encode higher-level "place" information responds instead to lower-level "edge" information. A corresponding brain area is demonstrated in non-human species.

Defining the exact mechanisms by which the brain processes visual objects and scenes remains an unresolved challenge. Valuable clues to this process have emerged from the demonstration that clusters of neurons (“modules”) in inferior temporal cortex apparently respond selectively to specific categories of visual stimuli, such as places/scenes. However, the higher-order “category-selective” response could also reflect specific lower-level spatial factors. Here we tested this idea in multiple functional MRI experiments, in humans and macaque monkeys, by systematically manipulating the spatial content of geometrical shapes and natural images. These tests revealed that visual spatial discontinuities (as reflected by an increased response to high spatial frequencies) selectively activate a well-known place-selective region of visual cortex (the “parahippocampal place area”) in humans. In macaques, we demonstrate a homologous cortical area, and show that it also responds selectively to higher spatial frequencies. The parahippocampal place area may use such information for detecting object borders and scene details during spatial perception and navigation.

Many reports suggest that different categories of visual stimuli are processed in correspondingly specific “modules” in the visual cortex. For instance, images of faces are processed in one cortical module (the “fusiform face area”), while images of scenes are processed in an adjacent module (the “parahippocampal place area,” or PPA). How does the PPA encode for such high-level, complex visual scenes? In this study, we show that at least part of the PPA response is due to a lower-level variable, reflected as higher spatial frequencies. These are prominent in the edges and details of scenes, but less prominent in faces and other stimuli. When we altered standard images of faces and places so that they only contained low, medium, or high spatial frequencies, we found that the PPA responded strongly to images containing high spatial frequencies. Importantly, using the same stimuli as for the human studies, we also demonstrated a homolog of human PPA in macaque temporal cortex (“mPPA”). As in humans, mPPA responds selectively to higher spatial frequencies. This demonstration of PPA in macaques paves the way for carrying out further electrophysiological and anatomical studies that may help elucidate the neural mechanisms for place selectivity in the human visual cortex.

Population coding of visual space: comparison of spatial representations in dorsal and ventral pathways

Although the representation of space is as fundamental to visual processing as the representation of shape, it has received relatively little attention from neurophysiological investigations. In this study we characterize representations of space within visual cortex, and examine how they differ in a first direct comparison between dorsal and ventral subdivisions of the visual pathways. Neural activities were recorded in anterior inferotemporal cortex (AIT) and lateral intraparietal cortex (LIP) of awake behaving monkeys, structures associated with the ventral and dorsal visual pathways respectively, as a stimulus was presented at different locations within the visual field. In spatially selective cells, we find greater modulation of cell responses in LIP with changes in stimulus position. Further, using a novel population-based statistical approach (namely, multidimensional scaling), we recover the spatial map implicit within activities of neural populations, allowing us to quantitatively compare the geometry of neural space with physical space. We show that a population of spatially selective LIP neurons, despite having large receptive fields, is able to almost perfectly reconstruct stimulus locations within a low-dimensional representation. In contrast, a population of AIT neurons, despite each cell being spatially selective, provide less accurate low-dimensional reconstructions of stimulus locations. They produce instead only a topologically (categorically) correct rendition of space, which nevertheless might be critical for object and scene recognition. Furthermore, we found that the spatial representation recovered from population activity shows greater translation invariance in LIP than in AIT. We suggest that LIP spatial representations may be dimensionally isomorphic with 3D physical space, while in AIT spatial representations may reflect a more categorical representation of space (e.g., "next to" or "above").

Infants and toddlers show enlarged visual sensitivity to nonaccidental compared with metric shape changes

Some shape changes are more important for object perception than others. We used a habituation paradigm to measure visual sensitivity to a nonaccidental shape change—that is, the transformation of a trapezium into a triangle and vice versa—and a metric shape change—that is, changing the aspect ratio of the shapes. Our data show that an enhanced perceptual sensitivity to nonaccidental changes is already present in infancy and remains stable into toddlerhood. We have thus established an example of how early visual perception deviates from the null hypothesis of representing similarity as a function of physical overlap between shapes, and does so in agreement with more cognitive, categorical demands.

Dynamic norm-based encoding for unfamiliar shapes in human visual cortex

Previous studies have argued that faces and other objects are encoded in terms of their deviation from a class prototype or norm. This prototype is associated with a smaller neural population response compared with nonprototype objects. However, it is still unclear (1) whether a norm-based representation can emerge for unfamiliar or novel object classes through visual experience at the time scale of an experiment and (2) whether the results from previous studies are caused by the prototypicality of a stimulus, by the physical properties of individual stimuli independent from the stimulus distribution, and/or by the trial-to-trial adaptation. Here we show with a combined behavioral and event-related fMRI study in humans that a short amount of visual experience with exemplars from novel object classes determines which stimulus is represented as the norm. Prototypicality effects were observed at the behavioral level by behavioral asymmetries during a stimulus comparison task. The fMRI data revealed that class exemplars closest to the prototypes--the perceived average of each class--were associated with a smaller response in the anterior part of the visual object-selective cortex compared with other class exemplars. By dissociating between the physical characteristics and the prototypicality status of the stimuli and by controlling for trial-to-trial adaptation, we can firmly conclude for the first time that high-level visual areas represent the identity of exemplars using a dynamic, norm-based encoding principle.

Relating Population-Code Representations between Man, Monkey, and Computational Models

Perceptual and cognitive content is thought to be represented in the brain by patterns of activity across populations of neurons. In order to test whether a computational model can explain a given population code and whether corresponding codes in man and monkey convey the same information, we need to quantitatively relate population-code representations. Here I give a brief introduction to representational similarity analysis, a particular approach to this problem. A population code is characterized by a representational dissimilarity matrix (RDM), which contains a dissimilarity for each pair of activity patterns elicited by a given stimulus set. The RDM encapsulates which distinctions the representation emphasizes and which it deemphasizes. By analyzing correlations between RDMs we can test models and compare different species. Moreover, we can study how representations are transformed across stages of processing and how they relate to behavioral measures of object similarity. We use an example from object vision to illustrate the method's potential to bridge major divides that have hampered progress in systems neuroscience.

Representation of shape in individuals from a culture with minimal exposure to regular, simple artifacts: sensitivity to nonaccidental versus metric properties

Many of the phenomena underlying shape recognition can be derived from the greater sensitivity to nonaccidental properties of an image (e.g., whether a contour is straight or curved), which are invariant to orientation in depth, than to the metric properties of an image (e.g., a contour's degree of curvature), which can vary with orientation. What enables this sensitivity? One explanation is that it derives from people's immersion in a manufactured world in which simple, regular shapes distinguished by nonaccidental properties abound (e.g., a can, a brick), and toddlers are encouraged to play with toy shape sorters. This report provides evidence against this explanation. The Himba, a seminomadic people living in a remote region of northwestern Namibia where there is little exposure to regular, simple artifacts, were virtually identical to Western observers in their greater sensitivity to nonaccidental properties than to metric properties of simple shapes.

Representing the forest before the trees: a global advantage effect in monkey inferotemporal cortex

Hierarchical stimuli (large shapes composed of small shapes) have long been used to study how humans perceive the global and the local content of a scene--the forest and the trees. Studies using these stimuli have revealed a global advantage effect: humans consistently report global shape faster than local shape. The neuronal underpinnings of this effect remain unclear. Here we demonstrate a correlate and possible mechanism in monkey inferotemporal cortex (IT). Inferotemporal neurons signal the global content of a hierarchical display approximately 30 ms before they signal its local content. This is a specific expression of a general principle, related to spatial scale or spatial frequency rather than to hierarchical level, whereby the representation of a large shape develops in IT before that of a small shape. These findings provide support for a coarse-to-fine model of visual scene representation.

Odor quality coding and categorization in human posterior piriform cortex

Efficient recognition of odorous objects universally shapes animal behavior and is crucial for survival. To distinguish kin from nonkin, mate from nonmate and food from nonfood, organisms must be able to create meaningful perceptual representations of odor qualities and categories. It is currently unknown where and in what form the brain encodes information about odor quality. By combining functional magnetic resonance imaging (fMRI) with multivariate (pattern-based) techniques, we found that spatially distributed ensemble activity in human posterior piriform cortex (PPC) coincides with perceptual ratings of odor quality, such that odorants with more (or less) similar fMRI patterns were perceived as more (or less) alike. We did not observe these effects in anterior piriform cortex, amygdala or orbitofrontal cortex, indicating that ensemble coding of odor categorical perception is regionally specific for PPC. These findings substantiate theoretical models emphasizing the importance of distributed piriform templates for the perceptual reconstruction of odor object quality.

Matching categorical object representations in inferior temporal cortex of man and monkey

Inferior temporal (IT) object representations have been intensively studied in monkeys and humans, but representations of the same particular objects have never been compared between the species. Moreover, IT's role in categorization is not well understood. Here, we presented monkeys and humans with the same images of real-world objects and measured the IT response pattern elicited by each image. In order to relate the representations between the species and to computational models, we compare response-pattern dissimilarity matrices. IT response patterns form category clusters, which match between man and monkey. The clusters correspond to animate and inanimate objects; within the animate objects, faces and bodies form subclusters. Within each category, IT distinguishes individual exemplars, and the within-category exemplar similarities also match between the species. Our findings suggest that primate IT across species may host a common code, which combines a categorical and a continuous representation of objects.

Delayed shape matching benefits from simplicity and symmetry

The aim of this study was to evaluate the influence of complexity and symmetry on shape recognition, by measuring the recognition of unfamiliar shapes (created using Fourier Boundary Descriptors, FBDs) through a delayed matching task. Between complexity levels the shapes differed in the frequency of the FBDs and within complexity levels in their phase. Shapes were calibrated to be physically equally similar for the different complexity levels. Matching two sequentially presented shapes was slower and less accurate when complexity increased and for asymmetrical compared to symmetrical versions of the shapes. Thus, we show that simplicity in general and symmetry in particular enhance the short-term recognition of unfamiliar shapes.

Multivariate patterns in object-selective cortex dissociate perceptual and physical shape similarity

Prior research has identified the lateral occipital complex (LOC) as a critical cortical region for the representation of object shape in humans. However, little is known about the nature of the representations contained in the LOC and their relationship to the perceptual experience of shape. We used human functional MRI to measure the physical, behavioral, and neural similarity between pairs of novel shapes to ask whether the representations of shape contained in subregions of the LOC more closely reflect the physical stimuli themselves, or the perceptual experience of those stimuli. Perceptual similarity measures for each pair of shapes were obtained from a psychophysical same-different task; physical similarity measures were based on stimulus parameters; and neural similarity measures were obtained from multivoxel pattern analysis methods applied to anterior LOC (pFs) and posterior LOC (LO). We found that the pattern of pairwise shape similarities in LO most closely matched physical shape similarities, whereas shape similarities in pFs most closely matched perceptual shape similarities. Further, shape representations were similar across participants in LO but highly variable across participants in pFs. Together, these findings indicate that activation patterns in subregions of object-selective cortex encode objects according to a hierarchy, with stimulus-based representations in posterior regions and subjective and observer-specific representations in anterior regions.

As early as 1031 a.d., the Arab scholar Ibn al-Haytham suggested that visual experience was not veridical, but inherently subjective. During the last few decades, this observation has given rise to one of the core questions in visual neuroscience: how does the subjective experience of visual stimuli relate to their neural representations in the brain? It is well-known that visual shape is represented in a brain region called lateral occipital complex (LOC). However, do these representations reflect physical or perceptual stimulus characteristics? We presented observers with a set of complex visual stimuli and obtained three measures of similarity for these stimuli: a physical similarity measure based on stimulus parameters; a behavioral similarity measure based on discrimination performance; and finally a neural similarity measure based on multivariate pattern analyses in LOC. We found that in anterior LOC, neural stimulus similarities correlated with subjective perceptual similarities, but not with physical stimulus similarities; the reverse was true in posterior LOC. In addition, neural similarities were consistent across participants in posterior LOC, but highly variable across participants in anterior LOC. Together these findings suggest a two-part answer to the question of how cortical object representations relate to subjective experience: anterior regions appear to contain subjective, individually variable shape representations, whereas posterior regions contain stimulus-based shape representations.

How does the subjective experience of visual shapes relate to the neural representations of these shapes in the brain? Using psychophysics, functional MRI, and multivariate pattern analysis methods, this study shows that activation patterns in anterior, shape-selective brain regions reflect perceptual shape similarities, whereas patterns in posterior regions reflect physical similarities.

Perceived shape similarity among unfamiliar objects and the organization of the human object vision pathway

Humans rely heavily on shape similarity among objects for object categorization and identification. Studies using functional magnetic resonance imaging (fMRI) have shown that a large region in human occipitotemporal cortex processes the shape of meaningful as well as unfamiliar objects. Here, we investigate whether the functional organization of this region as measured with fMRI is related to perceived shape similarity. We found that unfamiliar object classes that are rated as having a similar shape were associated with a very similar response pattern distributed across object-selective cortex, whereas object classes that were rated as being very different in shape were associated with a more different response pattern. Human observers, as well as object-selective cortex, were very sensitive to differences in shape features of the objects such as straight versus curved versus "spiky" edges, more so than to differences in overall shape envelope. Response patterns in retinotopic areas V1, V2, and V4 were not found to be related to perceived shape. The functional organization in area V3 was partially related to perceived shape but without a stronger sensitivity for shape features relative to overall shape envelope. Thus, for unfamiliar objects, the organization of human object-selective cortex is strongly related to perceived shape, and this shape-based organization emerges gradually throughout the object vision pathway.

Mechanisms of face perception

Faces are among the most informative stimuli we ever perceive: Even a split-second glimpse of a person's face tells us his identity, sex, mood, age, race, and direction of attention. The specialness of face processing is acknowledged in the artificial vision community, where contests for face-recognition algorithms abound. Neurological evidence strongly implicates a dedicated machinery for face processing in the human brain to explain the double dissociability of face- and object-recognition deficits. Furthermore, recent evidence shows that macaques too have specialized neural machinery for processing faces. Here we propose a unifying hypothesis, deduced from computational, neurological, fMRI, and single-unit experiments: that what makes face processing special is that it is gated by an obligatory detection process. We clarify this idea in concrete algorithmic terms and show how it can explain a variety of phenomena associated with face processing.

Interpreting fMRI data: maps, modules and dimensions

Neuroimaging research over the past decade has revealed a detailed picture of the functional organization of the human brain. Here we focus on two fundamental questions that are raised by the detailed mapping of sensory and cognitive functions and illustrate these questions with findings from the object-vision pathway. First, are functionally specific regions that are located close together best understood as distinct cortical modules or as parts of a larger-scale cortical map? Second, what functional properties define each cortical map or module? We propose a model in which overlapping continuous maps of simple features give rise to discrete modules that are selective for complex stimuli.

Pigeons and humans are more sensitive to nonaccidental than to metric changes in visual objects

Humans and macaques are more sensitive to differences in nonaccidental image properties, such as straight vs. curved contours, than to differences in metric properties, such as degree of curvature [Biederman, I., Bar, M., 1999. One-shot viewpoint invariance in matching novel objects. Vis. Res. 39, 2885-2899; Kayaert, G., Biederman, I., Vogels, R., 2003. Shape tuning in macaque inferior temporal cortex. J. Neurosci. 23, 3016-3027; Kayaert, G., Biederman, I., Vogels, R., 2005. Representation of regular and irregular shapes in macaque inferotemporal cortex. Cereb. Cortex 15, 1308-1321]. This differential sensitivity allows facile recognition when the object is viewed at an orientation in depth not previously experienced. In Experiment 1, we trained pigeons to discriminate grayscale, shaded images of four shapes. Pigeons made more confusion errors to shapes that shared more nonaccidental properties. Although the images in that experiment were not well controlled for incidental changes in metric properties, the same results were apparent with better controlled stimuli in Experiment 2: pigeons trained to discriminate a target shape from a metrically changed shape and a nonaccidentally changed shape committed more confusion errors to the metrically changed shape, suggesting that they perceived it to be more similar to the target shape. Humans trained with similar stimuli and procedure exhibited the same tendency to make more errors to the metrically changed shape. These results document the greater saliency of nonaccidental differences for shape recognition and discrimination in a non-primate species and suggest that nonaccidental sensitivity may be characteristic of all shape-discriminating species.

A stable topography of selectivity for unfamiliar shape classes in monkey inferior temporal cortex

The inferior temporal (IT) cortex in monkeys plays a central role in visual object recognition and learning. Previous studies have observed patches in IT cortex with strong selectivity for highly familiar object classes (e.g., faces), but the principles behind this functional organization are largely unknown due to the many properties that distinguish different object classes. To unconfound shape from meaning and memory, we scanned monkeys with functional magnetic resonance imaging while they viewed classes of initially novel objects. Our data revealed a topography of selectivity for these novel object classes across IT cortex. We found that this selectivity topography was highly reproducible and remarkably stable across a 3-month interval during which monkeys were extensively trained to discriminate among exemplars within one of the object classes. Furthermore, this selectivity topography was largely unaffected by changes in behavioral task and object retinal position, both of which preserve shape. In contrast, it was strongly influenced by changes in object shape. The topography was partially related to, but not explained by, the previously described pattern of face selectivity. Together, these results suggest that IT cortex contains a large-scale map of shape that is largely independent of meaning, familiarity, and behavioral task.