How position dependent is visual object recognition?

Development of the mirror-image sensitivity for different object categories-Evidence from the mirror costs of object images in children and adults

Object recognition relies on a multitude of factors, including size, orientation, and so on. Mirrored orientation, particularly due to children's mirror confusion in reading, holds special significance among various object orientations. Brain imaging studies suggest that the visual ventral and dorsal streams exhibit distinct orientation sensitivity across diverse object categories. Yet, it remains unclear whether mirror orientation sensitivity also varies among these categories during development at the behavioral level. Here, we explored the mirror sensitivity of children and adults across five distinct categories, which encompass tools that activate both the visual ventral stream for function information and the dorsal stream for manipulation information, and animals and faces that mainly activate the ventral stream. Two types of symbols, letters and Chinese characters, were also included. Mirror sensitivity was assessed through mirror costs-that is, the additional reaction time or error rate in the mirrored versus the same orientation condition when judging the identity of object pairs. The mirror costs in reaction times and error rates consistently revealed that children exhibited null mirror costs for tools, and the mirror costs for tools in adults were minimal, if any, and were smaller than those for letters and characters. The mirror costs reflected in absolute reaction time and error rate were similar across adults and children, but when the overall difference in reaction times was considered, adults showed a larger mirror cost than children. Overall, our investigation unveils categorical distinctions and development in mirror sensitivity of object recognition across the ventral and dorsal streams.

What determines location specificity or generalization of transsaccadic learning?

Humans incorporate knowledge of transsaccadic associations into peripheral object perception. Several studies have shown that learning of new manipulated transsaccadic associations leads to a presaccadic perceptual bias. However, there was still disagreement whether this learning effect was location specific (Herwig, Weiß, & Schneider, 2018) or generalizes to new locations (Valsecchi & Gegenfurtner, 2016). The current study investigated under what conditions location generalization of transsaccadic learning occurs. In all experiments, there were acquisition phases in which the spatial frequency (Experiment 1) or the size (Experiment 2 and 3) of objects was changed transsaccadically. In the test phases, participants judged the respective feature of peripheral objects. These could appear either at the location where learning had taken place or at new locations. All experiments replicated the perceptual bias effect at the old learning locations. In two experiments, transsaccadic learning remained location specific even when learning occurred at multiple locations (Experiment 1) or with the feature of size (Experiment 2) for which a transfer had previously been shown. Only in Experiment 3 was a transfer of the learning effect to new locations observable. Here, learning only took place for one object and not for several objects that had to be discriminated. Therefore, one can conclude that, when specific associations are learned for multiple objects, transsaccadic learning stays location specific and when a transsaccadic association is learned for only one object it allows a generalization to other locations.

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Regressing Image Sub-Population Distributions with Deep Learning

Regressing the distribution of different sub-populations from a batch of images with learning algorithms is not a trivial task, as models tend to make errors that are unequally distributed across the different sub-populations. Obviously, the baseline is forming a histogram from the batch after having characterized each image independently. However, we show that this approach can be strongly improved by making the model aware of the ultimate task thanks to a density loss for both sub-populations related to classes (on three public datasets of image classification) and sub-populations related to size (on two public datasets of object detection in image). For example, class distribution was improved two-fold on the EUROSAT dataset and size distribution was improved by 10% on the PASCAL VOC dataset with both RESNET and VGG backbones. The code is released in the GitHub archive at achanhon/AdversarialModel/tree/master/proportion.

Information redundancy across spatial scales modulates early visual cortical processing

Visual images contain redundant information across spatial scales where low spatial frequency contrast is informative towards the location and likely content of high spatial frequency detail. Previous research suggests that the visual system makes use of those redundancies to facilitate efficient processing. In this framework, a fast, initial analysis of low-spatial frequency (LSF) information guides the slower and later processing of high spatial frequency (HSF) detail. Here, we used multivariate classification as well as time-frequency analysis of MEG responses to the viewing of intact and phase scrambled images of human faces to demonstrate that the availability of redundant LSF information, as found in broadband intact images, correlates with a reduction in HSF representational dominance in both early and higher-level visual areas as well as a reduction of gamma-band power in early visual cortex. Our results indicate that the cross spatial frequency information redundancy that can be found in all natural images might be a driving factor in the efficient integration of fine image details.

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

Visual translation tolerance refers to our capacity to recognize objects over a wide range of different retinal locations. Although translation is perhaps the simplest spatial transform that the visual system needs to cope with, the extent to which the human visual system can identify objects at previously unseen locations is unclear, with some studies reporting near complete invariance over 10 degrees and other reporting zero invariance at 4 degrees of visual angle. Similarly, there is confusion regarding the extent of translation tolerance in computational models of vision, as well as the degree of match between human and model performance. Here, we report a series of eye-tracking studies (total N = 70) demonstrating that novel objects trained at one retinal location can be recognized at high accuracy rates following translations up to 18 degrees. We also show that standard deep convolutional neural networks (DCNNs) support our findings when pretrained to classify another set of stimuli across a range of locations, or when a global average pooling (GAP) layer is added to produce larger receptive fields. Our findings provide a strong constraint for theories of human vision and help explain inconsistent findings previously reported with convolutional neural networks (CNNs).

Examining the Coding Strength of Object Identity and Nonidentity Features in Human Occipito-Temporal Cortex and Convolutional Neural Networks

A visual object is characterized by multiple visual features, including its identity, position and size. Despite the usefulness of identity and nonidentity features in vision and their joint coding throughout the primate ventral visual processing pathway, they have so far been studied relatively independently. Here in both female and male human participants, the coding of identity and nonidentity features was examined together across the human ventral visual pathway. The nonidentity features tested included two Euclidean features (position and size) and two non-Euclidean features (image statistics and spatial frequency (SF) content of an image). Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with identity outweighing the non-Euclidean but not the Euclidean features at higher levels of visual processing. In 14 convolutional neural networks (CNNs) pretrained for object categorization with varying architecture, depth, and with/without recurrent processing, nonidentity feature representation showed an initial large increase from early to mid-stage of processing, followed by a decrease at later stages of processing, different from brain responses. Additionally, from lower to higher levels of visual processing, position became more underrepresented and image statistics and SF became more overrepresented compared with identity in CNNs than in the human brain. Similar results were obtained in a CNN trained with stylized images that emphasized shape representations. Overall, by measuring the coding strength of object identity and nonidentity features together, our approach provides a new tool for characterizing feature coding in the human brain and the correspondence between the brain and CNNs.SIGNIFICANCE STATEMENT This study examined the coding strength of object identity and four types of nonidentity features along the human ventral visual processing pathway and compared brain responses with those of 14 convolutional neural networks (CNNs) pretrained to perform object categorization. Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with some notable differences among the different nonidentity features. CNNs differed from the brain in a number of aspects in their representations of identity and nonidentity features over the course of visual processing. Our approach provides a new tool for characterizing feature coding in the human brain and the correspondence between the brain and CNNs.

What Does Dorsal Cortex Contribute to Perception?

According to the influential "Two Visual Pathways" hypothesis, the cortical visual system is segregated into two pathways, with the ventral, occipitotemporal pathway subserving object perception, and the dorsal, occipitoparietal pathway subserving the visuomotor control of action. However, growing evidence suggests that the dorsal pathway also plays a functional role in object perception. In the current article, we present evidence that the dorsal pathway contributes uniquely to the perception of a range of visuospatial attributes that are not redundant with representations in ventral cortex. We describe how dorsal cortex is recruited automatically during perception, even when no explicit visuomotor response is required. Importantly, we propose that dorsal cortex may selectively process visual attributes that can inform the perception of potential actions on objects and environments, and we consider plausible developmental and cognitive mechanisms that might give rise to these representations. As such, we consider whether naturalistic stimuli, such as real-world solid objects, might engage dorsal cortex more so than simplified or artificial stimuli such as images that do not afford action, and how the use of suboptimal stimuli might limit our understanding of the functional contribution of dorsal cortex to visual perception.

Recent advances in understanding object recognition in the human brain: deep neural networks, temporal dynamics, and context

Object recognition is the ability to identify an object or category based on the combination of visual features observed. It is a remarkable feat of the human brain, given that the patterns of light received by the eye associated with the properties of a given object vary widely with simple changes in viewing angle, ambient lighting, and distance. Furthermore, different exemplars of a specific object category can vary widely in visual appearance, such that successful categorization requires generalization across disparate visual features. In this review, we discuss recent advances in understanding the neural representations underlying object recognition in the human brain. We highlight three current trends in the approach towards this goal within the field of cognitive neuroscience. Firstly, we consider the influence of deep neural networks both as potential models of object vision and in how their representations relate to those in the human brain. Secondly, we review the contribution that time-series neuroimaging methods have made towards understanding the temporal dynamics of object representations beyond their spatial organization within different brain regions. Finally, we argue that an increasing emphasis on the context (both visual and task) within which object recognition occurs has led to a broader conceptualization of what constitutes an object representation for the brain. We conclude by identifying some current challenges facing the experimental pursuit of understanding object recognition and outline some emerging directions that are likely to yield new insight into this complex cognitive process.

Object responses are highly malleable, rather than invariant, with changes in object appearance

Theoretical frameworks of human vision argue that object responses remain stable, or 'invariant', despite changes in viewing conditions that can alter object appearance but not identity. Here, in a major departure from previous approaches that have relied on two-dimensional (2-D) images to study object processing, we demonstrate that changes in an object's appearance, but not its identity, can lead to striking shifts in behavioral responses to objects. We used inverse multidimensional scaling (MDS) to measure the extent to which arrangements of objects in a sorting task were similar or different when the stimuli were displayed as scaled 2-D images, three-dimensional (3-D) augmented reality (AR) projections, or real-world solids. We were especially interested in whether sorting behavior in each display format was based on conceptual (e.g., typical location) versus physical object characteristics. We found that 2-D images of objects were arranged according to conceptual (typical location), but not physical, properties. AR projections, conversely, were arranged primarily according to physical properties such as real-world size, elongation and weight, but not conceptual properties. Real-world solid objects, unlike both 2-D and 3-D images, were arranged using multidimensional criteria that incorporated both conceptual and physical object characteristics. Our results suggest that object responses can be strikingly malleable, rather than invariant, with changes in the visual characteristics of the stimulus. The findings raise important questions about limits of invariance in object processing, and underscore the importance of studying responses to richer stimuli that more closely resemble those we encounter in real-world environments.

A Prototypical Template for Rapid Face Detection Is Embedded in the Monkey Superior Colliculus

Human babies respond preferentially to faces or face-like images. It has been proposed that an innate and rapid face detection system is present at birth before the cortical visual pathway is developed in many species, including primates. However, in primates, the visual area responsible for this process is yet to be unraveled. We hypothesized that the superior colliculus (SC) that receives direct and indirect retinal visual inputs may serve as an innate rapid face-detection system in primates. To test this hypothesis, we examined the responsiveness of monkey SC neurons to first-order information of faces required for face detection (basic spatial layout of facial features including eyes, nose, and mouth), by analyzing neuronal responses to line drawing images of: (1) face-like patterns with contours and properly placed facial features; (2) non-face patterns including face contours only; and (3) nonface random patterns with contours and randomly placed face features. Here, we show that SC neurons respond stronger and faster to upright and inverted face-like patterns compared to the responses to nonface patterns, regardless of contrast polarity and contour shapes. Furthermore, SC neurons with central receptive fields (RFs) were more selective to face-like patterns. In addition, the population activity of SC neurons with central RFs can discriminate face-like patterns from nonface patterns as early as 50 ms after the stimulus onset. Our results provide strong neurophysiological evidence for the involvement of the primate SC in face detection and suggest the existence of a broadly tuned template for face detection in the subcortical visual pathway.

Scale and translation-invariance for novel objects in human vision

Though the range of invariance in recognition of novel objects is a basic aspect of human vision, its characterization has remained surprisingly elusive. Here we report tolerance to scale and position changes in one-shot learning by measuring recognition accuracy of Korean letters presented in a flash to non-Korean subjects who had no previous experience with Korean letters. We found that humans have significant scale-invariance after only a single exposure to a novel object. The range of translation-invariance is limited, depending on the size and position of presented objects. To understand the underlying brain computation associated with the invariance properties, we compared experimental data with computational modeling results. Our results suggest that to explain invariant recognition of objects by humans, neural network models should explicitly incorporate built-in scale-invariance, by encoding different scale channels as well as eccentricity-dependent representations captured by neurons' receptive field sizes and sampling density that change with eccentricity. Our psychophysical experiments and related simulations strongly suggest that the human visual system uses a computational strategy that differs in some key aspects from current deep learning architectures, being more data efficient and relying more critically on eye-movements.

Object Vision in a Structured World

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world.

Embedded word priming elicits enhanced fMRI responses in the visual word form area

Lexical embedding is common in all languages and elicits mutual orthographic interference between an embedded word and its carrier. The neural basis of such interference remains unknown. We employed a novel fMRI prime-target embedded word paradigm to test for involvement of a visual word form area (VWFA) in left ventral occipitotemporal cortex in co-activation of embedded words and their carriers. Based on the results of related fMRI studies we predicted either enhancement or suppression of fMRI responses to embedded words initially viewed as primes, and repeated in the context of target carrier words. Our results clearly showed enhancement of fMRI responses in the VWFA to embedded-carrier word pairs as compared to unrelated prime-target pairs. In contrast to non-visual language-related areas (e.g., left inferior frontal gyrus), enhanced fMRI responses did not occur in the VWFA when embedded-carrier word pairs were restricted to the left visual hemifield. Our finding of fMRI enhancement in the VWFA is novel evidence of its involvement in representational rivalry between orthographically similar words, and the co-activation of embedded words and their carriers.

Idiosyncratic, Retinotopic Bias in Face Identification Modulated by Familiarity

The perception of gender and age of unfamiliar faces is reported to vary idiosyncratically across retinal locations such that, for example, the same androgynous face may appear to be male at one location but female at another. Here, we test spatial heterogeneity for the recognition of the identity of personally familiar faces in human participants. We found idiosyncratic biases that were stable within participants and that varied more across locations for low as compared to high familiar faces. These data suggest that like face gender and age, face identity is processed, in part, by independent populations of neurons monitoring restricted spatial regions and that the recognition responses vary for the same face across these different locations. Moreover, repeated and varied social interactions appear to lead to adjustments of these independent face recognition neurons so that the same familiar face is eventually more likely to elicit the same recognition response across widely separated visual field locations. We provide a mechanistic account of this reduced retinotopic bias based on computational simulations.

Mid-level visual features underlie the high-level categorical organization of the ventral stream

Human object-selective cortex shows a large-scale organization characterized by the high-level properties of both animacy and object size. To what extent are these neural responses explained by primitive perceptual features that distinguish animals from objects and big objects from small objects? To address this question, we used a texture synthesis algorithm to create a class of stimuli-texforms-which preserve some mid-level texture and form information from objects while rendering them unrecognizable. We found that unrecognizable texforms were sufficient to elicit the large-scale organizations of object-selective cortex along the entire ventral pathway. Further, the structure in the neural patterns elicited by texforms was well predicted by curvature features and by intermediate layers of a deep convolutional neural network, supporting the mid-level nature of the representations. These results provide clear evidence that a substantial portion of ventral stream organization can be accounted for by coarse texture and form information without requiring explicit recognition of intact objects.

General Transformations of Object Representations in Human Visual Cortex

The brain actively represents incoming information, but these representations are only useful to the extent that they flexibly reflect changes in the environment. How does the brain transform representations across changes, such as in size or viewing angle? We conducted a fMRI experiment and a magnetoencephalography experiment in humans (both sexes) in which participants viewed objects before and after affine viewpoint changes (rotation, translation, enlargement). We used a novel approach, representational transformation analysis, to derive transformation functions that linked the distributed patterns of brain activity evoked by an object before and after an affine change. Crucially, transformations derived from one object could predict a postchange representation for novel objects. These results provide evidence of general operations in the brain that are distinct from neural representations evoked by particular objects and scenes.SIGNIFICANCE STATEMENT The dominant focus in cognitive neuroscience has been on how the brain represents information, but these representations are only useful to the extent that they flexibly reflect changes in the environment. How does the brain transform representations, such as linking two states of an object, for example, before and after an object undergoes a physical change? We used a novel method to derive transformations between the brain activity evoked by an object before and after an affine viewpoint change. We show that transformations derived from one object undergoing a change generalized to a novel object undergoing the same change. This result shows that there are general perceptual operations that transform object representations from one state to another.

Typical visual-field locations enhance processing in object-selective channels of human occipital cortex

Natural environments consist of multiple objects, many of which repeatedly occupy similar locations within a scene. For example, hats are seen on people’s heads, while shoes are most often seen close to the ground. Such positional regularities bias the distribution of objects across the visual field: hats are more often encountered in the upper visual field, while shoes are more often encountered in the lower visual field. Here we tested the hypothesis that typical visual field locations of objects facilitate cortical processing. We recorded functional MRI while participants viewed images of objects that were associated with upper or lower visual field locations. Using multivariate classification, we show that object information can be more successfully decoded from response patterns in object-selective lateral occipital cortex (LO) when the objects are presented in their typical location (e.g., shoe in the lower visual field) than when they are presented in an atypical location (e.g., shoe in the upper visual field). In a functional connectivity analysis, we relate this benefit to increased coupling between LO and early visual cortex, suggesting that typical object positioning facilitates information propagation across the visual hierarchy. Together these results suggest that object representations in occipital visual cortex are tuned to the structure of natural environments. This tuning may support object perception in spatially structured environments.

NEW & NOTEWORTHY In the real world, objects appear in predictable spatial locations. Hats, commonly appearing on people’s heads, often fall into the upper visual field. Shoes, mostly appearing on people’s feet, often fall into the lower visual field. Here we used functional MRI to demonstrate that such regularities facilitate cortical processing: Objects encountered in their typical locations are coded more efficiently, which may allow us to effortlessly recognize objects in natural environments.

The influence of behavioral relevance on the processing of global scene properties: An ERP study

Recent work studying the temporal dynamics of visual scene processing (Harel et al., 2016) has found that global scene properties (GSPs) modulate the amplitude of early Event-Related Potentials (ERPs). It is still not clear, however, to what extent the processing of these GSPs is influenced by their behavioral relevance, determined by the goals of the observer. To address this question, we investigated how behavioral relevance, operationalized by the task context impacts the electrophysiological responses to GSPs. In a set of two experiments we recorded ERPs while participants viewed images of real-world scenes, varying along two GSPs, naturalness (manmade/natural) and spatial expanse (open/closed). In Experiment 1, very little attention to scene content was required as participants viewed the scenes while performing an orthogonal fixation-cross task. In Experiment 2 participants saw the same scenes but now had to actively categorize them, based either on their naturalness or spatial expense. We found that task context had very little impact on the early ERP responses to the naturalness and spatial expanse of the scenes: P1, N1, and P2 could distinguish between open and closed scenes and between manmade and natural scenes across both experiments. Further, the specific effects of naturalness and spatial expanse on the ERP components were largely unaffected by their relevance for the task. A task effect was found at the N1 and P2 level, but this effect was manifest across all scene dimensions, indicating a general effect rather than an interaction between task context and GSPs. Together, these findings suggest that the extraction of global scene information reflected in the early ERP components is rapid and very little influenced by top-down observer-based goals.

On the partnership between neural representations of object categories and visual features in the ventral visual pathway

A dominant view in the cognitive neuroscience of object vision is that regions of the ventral visual pathway exhibit some degree of category selectivity. However, recent findings obtained with multivariate pattern analyses (MVPA) suggest that apparent category selectivity in these regions is dependent on more basic visual features of stimuli. In which case a rethinking of the function and organization of the ventral pathway may be in order. We suggest that addressing this issue of functional specificity requires clear coding hypotheses, about object category and visual features, which make contrasting predictions about neuroimaging results in ventral pathway regions. One way to differentiate between categorical and featural coding hypotheses is to test for residual categorical effects: effects of category selectivity that cannot be accounted for by visual features of stimuli. A strong method for testing these effects, we argue, is to make object category and target visual features orthogonal in stimulus design. Recent studies that adopt this approach support a feature-based categorical coding hypothesis according to which regions of the ventral stream do indeed code for object category, but in a format at least partially based on the visual features of stimuli.

Motor experience influences object knowledge

An object's perceived readiness-for-action (e.g., its size, the degree of rotation from its canonical position, the user's viewpoint) can influence semantic knowledge retrieval. Yet, the organization of object knowledge may also be affected by body-specific sensorimotor experiences. Here, we investigated whether people's history of performing motor actions with their hands influences the knowledge they store and retrieve about graspable objects. We compared object representations between healthy right- and left-handers (Experiment 1), and between unilateral stroke patients, whose motor experience was changed by impairment of either their right or left hand (Experiment 2). Participants saw pictures of graspable everyday items with the handles oriented toward either the left or right hand, and they generated the type of grasp they would employ (i.e., clench or pinch) when using each object, responding orally. In both experiments, hand dominance and object orientation interacted to predict response times. In Experiment 1, judgments were fastest when objects were oriented toward the right hand in right-handers, but not in left-handers. In Experiment 2, judgments were fastest when objects were oriented toward the left hand in patients who had lost the use of their right hand, even though these patients were right-handed prior to brain injury. Results suggest that at least some aspects of object knowledge are determined by motor experience, and can be changed by new patterns of motor experience. People with different bodily characteristics, who interact with objects in systematically different ways, form correspondingly different neurocognitive representations of the same common objects. (PsycINFO Database Record

Hard-wired feed-forward visual mechanisms of the brain compensate for affine variations in object recognition

Humans perform object recognition effortlessly and accurately. However, it is unknown how the visual system copes with variations in objects' appearance and the environmental conditions. Previous studies have suggested that affine variations such as size and position are compensated for in the feed-forward sweep of visual information processing while feedback signals are needed for precise recognition when encountering non-affine variations such as pose and lighting. Yet, no empirical data exist to support this suggestion. We systematically investigated the impact of the above-mentioned affine and non-affine variations on the categorization performance of the feed-forward mechanisms of the human brain. For that purpose, we designed a backward-masking behavioral categorization paradigm as well as a passive viewing EEG recording experiment. On a set of varying stimuli, we found that the feed-forward visual pathways contributed more dominantly to the compensation of variations in size and position compared to lighting and pose. This was reflected in both the amplitude and the latency of the category separability indices obtained from the EEG signals. Using a feed-forward computational model of the ventral visual stream, we also confirmed a more dominant role for the feed-forward visual mechanisms of the brain in the compensation of affine variations. Taken together, our experimental results support the theory that non-affine variations such as pose and lighting may need top-down feedback information from higher areas such as IT and PFC for precise object recognition.

Functional modulation of contralateral bias in early and object-selective areas after stroke of the occipital ventral cortices

Object agnosia is a rare symptom, occurring mainly after bilateral damage of the ventral visual cortex. Most patients suffering from unilateral ventral lesions are clinically non-agnosic. Here, we studied the effect of unilateral occipito-temporal lesions on object categorization and its underlying neural correlates in visual areas. Thirteen non-agnosic stroke patients and twelve control subjects performed an event-related rapid object categorization task in the fMRI scanner where images were presented either to the left or to the right of a fixed point. Eight patients had intact central visual fields within at least 10° eccentricity while five patients showed an incomplete hemianopia. Patients made more errors than controls for both contra- and ipsilesional presentation, meaning that object categorization was impaired bilaterally in both patient groups. The activity in cortical visual areas is usually higher when a stimulus is presented contralaterally compared to presented ipsilaterally (contralateral bias). A region of interest analysis of early visual (V1-V4) and object-selective areas (lateral occipital complex, LOC; fusiform face area, FFA; and parahippocampal place area, PPA) revealed that the lesioned-hemisphere of patients showed reduced contralateral bias in early visual areas and LOC. In contrast, literally no contralateral bias in FFA and PPA was found. These findings indicate disturbed processing in the lesioned hemisphere, which might be related to the processing of visually presented objects. Thus, unilateral occipito-temporal damage leads to altered contralateral bias in the lesioned hemisphere, which might be the cause of impaired categorization performance in both visual hemifields in clinically non-agnosic patients. We conclude that both hemispheres need to be functionally intact for unimpaired object processing.

Position selectivity in face-sensitive visual cortex to facial and nonfacial stimuli: an fMRI study

BACKGROUND

Evidence for position sensitivity in object-selective visual areas has been building. On one hand, most of the relevant studies have utilized stimuli for which the areas are optimally selective and examine small sections of cortex. On the other hand, visual field maps established with nonspecific stimuli have been found in increasingly large areas of visual cortex, though generally not in areas primarily responsive to faces.

METHODS

fMRI was used to study the position sensitivity of the occipital face area (OFA) and the fusiform face area (FFA) to both standard rotating wedge retinotopic mapping stimuli and quadrant presentations of synthetic facial stimuli. Analysis methods utilized were both typical, that is, mean univariate BOLD signals and multivoxel pattern analysis (MVPA), and novel, that is, distribution of voxels to pattern classifiers and use of responses to nonfacial retinotopic mapping stimuli to classify responses to facial stimuli.

RESULTS

Polar angle sensitivity was exhibited to standard retinotopic mapping stimuli with a stronger contralateral bias in OFA than in FFA, a stronger bias toward the vertical meridian in FFA than in OFA, and a bias across both areas toward the inferior visual field. Contralateral hemispheric lateralization of both areas was again shown using synthetic face stimuli based on univariate BOLD signals, MVPA, and the biased contribution of voxels toward multivariate classifiers discriminating the contralateral visual field. Classifiers based on polar angle responsivity were used to classify the patterns of activation above chance levels to face stimuli in the OFA but not in the FFA.

CONCLUSIONS

Both the OFA and FFA exhibit quadrant sensitivity to face stimuli, though the OFA exhibits greater position responsivity across stimuli than the FFA and includes overlap in the response pattern to the disparate stimulus types. Such biases are consistent with varying position sensitivity along different surfaces of occipito-temporal cortex.

Representational Dynamics of Facial Viewpoint Encoding

Faces provide a wealth of information, including the identity of the seen person and social cues, such as the direction of gaze. Crucially, different aspects of face processing require distinct forms of information encoding. Another person's attentional focus can be derived based on a view-dependent code. In contrast, identification benefits from invariance across all viewpoints. Different cortical areas have been suggested to subserve these distinct functions. However, little is known about the temporal aspects of differential viewpoint encoding in the human brain. Here, we combine EEG with multivariate data analyses to resolve the dynamics of face processing with high temporal resolution. This revealed a distinct sequence of viewpoint encoding. Head orientations were encoded first, starting after around 60 msec of processing. Shortly afterward, peaking around 115 msec after stimulus onset, a different encoding scheme emerged. At this latency, mirror-symmetric viewing angles elicited highly similar cortical responses. Finally, about 280 msec after visual onset, EEG response patterns demonstrated a considerable degree of viewpoint invariance across all viewpoints tested, with the noteworthy exception of the front-facing view. Taken together, our results indicate that the processing of facial viewpoints follows a temporal sequence of encoding schemes, potentially mirroring different levels of computational complexity.

Evaluating the correspondence between face-, scene-, and object-selectivity and retinotopic organization within lateral occipitotemporal cortex

The organization of human lateral occipitotemporal cortex (lOTC) has been characterized largely according to two distinct principles: retinotopy and category-selectivity. Whereas category-selective regions were originally thought to exist beyond retinotopic maps, recent evidence highlights overlap. Here, we combined detailed mapping of retinotopy, using population receptive fields (pRF), and category-selectivity to examine and contrast the retinotopic profiles of scene- (occipital place area, OPA), face- (occipital face area, OFA) and object- (lateral occipital cortex, LO) selective regions of lOTC. We observe striking differences in the relationship each region has to underlying retinotopy. Whereas OPA overlapped multiple retinotopic maps (including V3A, V3B, LO1, and LO2), and LO overlapped two maps (LO1 and LO2), OFA overlapped almost none. There appears no simple consistent relationship between category-selectivity and retinotopic maps, meaning category-selective regions are not constrained spatially to retinotopic map borders consistently. The multiple maps that overlap OPA suggests it is likely not appropriate to conceptualize it as a single scene-selective region, whereas the inconsistency in any systematic map overlapping OFA suggests it may constitute a more uniform area. Beyond their relationship to retinotopy, all three regions evidenced strongly retinotopic voxels, with pRFs exhibiting a significant bias towards the contralateral lower visual field, despite differences in pRF size, contributing to an emerging literature suggesting this bias is present across much of lOTC. Taken together, these results suggest that whereas category-selective regions are not constrained to consistently contain ordered retinotopic maps, they nonetheless likely inherit retinotopic characteristics of the maps from which they draw information.

Perception and Processing of Faces in the Human Brain Is Tuned to Typical Feature Locations

Faces are salient social stimuli whose features attract a stereotypical pattern of fixations. The implications of this gaze behavior for perception and brain activity are largely unknown. Here, we characterize and quantify a retinotopic bias implied by typical gaze behavior toward faces, which leads to eyes and mouth appearing most often in the upper and lower visual field, respectively. We found that the adult human visual system is tuned to these contingencies. In two recognition experiments, recognition performance for isolated face parts was better when they were presented at typical, rather than reversed, visual field locations. The recognition cost of reversed locations was equal to ∼60% of that for whole face inversion in the same sample. Similarly, an fMRI experiment showed that patterns of activity evoked by eye and mouth stimuli in the right inferior occipital gyrus could be separated with significantly higher accuracy when these features were presented at typical, rather than reversed, visual field locations. Our findings demonstrate that human face perception is determined not only by the local position of features within a face context, but by whether features appear at the typical retinotopic location given normal gaze behavior. Such location sensitivity may reflect fine-tuning of category-specific visual processing to retinal input statistics. Our findings further suggest that retinotopic heterogeneity might play a role for face inversion effects and for the understanding of conditions affecting gaze behavior toward faces, such as autism spectrum disorders and congenital prosopagnosia.

SIGNIFICANCE STATEMENT

Faces attract our attention and trigger stereotypical patterns of visual fixations, concentrating on inner features, like eyes and mouth. Here we show that the visual system represents face features better when they are shown at retinal positions where they typically fall during natural vision. When facial features were shown at typical (rather than reversed) visual field locations, they were discriminated better by humans and could be decoded with higher accuracy from brain activity patterns in the right occipital face area. This suggests that brain representations of face features do not cover the visual field uniformly. It may help us understand the well-known face-inversion effect and conditions affecting gaze behavior toward faces, such as prosopagnosia and autism spectrum disorders.

Behavioral Differences in the Upper and Lower Visual Hemifields in Shape and Motion Perception

Perceptual accuracy is known to be influenced by stimuli location within the visual field. In particular, it seems to be enhanced in the lower visual hemifield (VH) for motion and space processing, and in the upper VH for object and face processing. The origins of such asymmetries are attributed to attentional biases across the visual field, and in the functional organization of the visual system. In this article, we tested content-dependent perceptual asymmetries in different regions of the visual field. Twenty-five healthy volunteers participated in this study. They performed three visual tests involving perception of shapes, orientation and motion, in the four quadrants of the visual field. The results of the visual tests showed that perceptual accuracy was better in the lower than in the upper visual field for motion perception, and better in the upper than in the lower visual field for shape perception. Orientation perception did not show any vertical bias. No difference was found when comparing right and left VHs. The functional organization of the visual system seems to indicate that the dorsal and the ventral visual streams, responsible for motion and shape perception, respectively, show a bias for the lower and upper VHs, respectively. Such a bias depends on the content of the visual information.

Resilience to the contralateral visual field bias as a window into object representations

Viewing images of manipulable objects elicits differential blood oxygen level-dependent (BOLD) contrast across parietal and dorsal occipital areas of the human brain that support object-directed reaching, grasping, and complex object manipulation. However, it is unknown which object-selective regions of parietal cortex receive their principal inputs from the ventral object-processing pathway and which receive their inputs from the dorsal object-processing pathway. Parietal areas that receive their inputs from the ventral visual pathway, rather than from the dorsal stream, will have inputs that are already filtered through object categorization and identification processes. This predicts that parietal regions that receive inputs from the ventral visual pathway should exhibit object-selective responses that are resilient to contralateral visual field biases. To test this hypothesis, adult participants viewed images of tools and animals that were presented to the left or right visual fields during functional magnetic resonance imaging (fMRI). We found that the left inferior parietal lobule showed robust tool preferences independently of the visual field in which tool stimuli were presented. In contrast, a region in posterior parietal/dorsal occipital cortex in the right hemisphere exhibited an interaction between visual field and category: tool-preferences were strongest contralateral to the stimulus. These findings suggest that action knowledge accessed in the left inferior parietal lobule operates over inputs that are abstracted from the visual input and is contingent on analysis by the ventral visual pathway, consistent with its putative role in supporting object manipulation knowledge.

Neural representation for object recognition in inferotemporal cortex

We suggest that population representation of objects in inferotemporal cortex lie on a continuum between a purely structural, parts-based description and a purely holistic description. The intrinsic dimensionality of object representation is estimated to be around 100, perhaps with lower dimensionalities for object representations more toward the holistic end of the spectrum. Cognitive knowledge in the form of semantic information and task information feed back to inferotemporal cortex from perirhinal and prefrontal cortex respectively, providing high-level multimodal-based expectations that assist in the interpretation of object stimuli. Integration of object information across eye movements may also contribute to object recognition through a process of active vision.

Promoting rotational-invariance in object recognition despite experience with only a single view

Different processes are assumed to underlie invariant object recognition across affine transformations, such as changes in size, and non-affine transformations, such as rotations in depth. From this perspective, promoting invariant object recognition across rotations in depth requires visual experience with the object from multiple viewpoints. One learning mechanism potentially contributing to invariant recognition is the error-driven learning of associations between relatively view-invariant visual properties and motor responses or object labels. This account uniquely predicts that experience with affine transformations of a single object view may also promote view-invariance, if view-invariant properties are also invariant across such affine transformations. We empirically confirmed this prediction in both people and pigeons, thereby suggesting that: (a) the hypothesized mechanism participates in view-invariance learning, (b) this mechanism is present across distantly-related vertebrates, and (c) the distinction between affine and non-affine transformations may not be fundamental for biological visual systems, as previously assumed.

The conjunction of non-consciously perceived object identity and spatial position can be retained during a visual short-term memory task

Although non-consciously perceived information has previously been assumed to be short-lived (< 500 ms), recent findings show that non-consciously perceived information can be maintained for at least 15 s. Such findings can be explained as working memory without a conscious experience of the information to be retained. However, whether or not working memory can operate on non-consciously perceived information remains controversial, and little is known about the nature of such non-conscious visual short-term memory (VSTM). Here we used continuous flash suppression to render stimuli non-conscious, to investigate the properties of non-consciously perceived representations in delayed match-to-sample (DMS) tasks. In Experiment I we used variable delays (5 or 15 s) and found that performance was significantly better than chance and was unaffected by delay duration, thereby replicating previous findings. In Experiment II the DMS task required participants to combine information of spatial position and object identity on a trial-by-trial basis to successfully solve the task. We found that the conjunction of spatial position and object identity was retained, thereby verifying that non-conscious, trial-specific information can be maintained for prospective use. We conclude that our results are consistent with a working memory interpretation, but that more research is needed to verify this interpretation.

Value-Driven Attentional Capture is Modulated by Spatial Context

When stimuli are associated with reward outcome, their visual features acquire high attentional priority such that stimuli possessing those features involuntarily capture attention. Whether a particular feature is predictive of reward, however, will vary with a number of contextual factors. One such factor is spatial location: for example, red berries are likely to be found in low-lying bushes, whereas yellow bananas are likely to be found on treetops. In the present study, I explore whether the attentional priority afforded to reward-associated features is modulated by such location-based contingencies. The results demonstrate that when a stimulus feature is associated with a reward outcome in one spatial location but not another, attentional capture by that feature is selective to when it appears in the rewarded location. This finding provides insight into how reward learning effectively modulates attention in an environment with complex stimulus-reward contingencies, thereby supporting efficient foraging.

Invariant visual object recognition and shape processing in rats

Invariant visual object recognition is the ability to recognize visual objects despite the vastly different images that each object can project onto the retina during natural vision, depending on its position and size within the visual field, its orientation relative to the viewer, etc. Achieving invariant recognition represents such a formidable computational challenge that is often assumed to be a unique hallmark of primate vision. Historically, this has limited the invasive investigation of its neuronal underpinnings to monkey studies, in spite of the narrow range of experimental approaches that these animal models allow. Meanwhile, rodents have been largely neglected as models of object vision, because of the widespread belief that they are incapable of advanced visual processing. However, the powerful array of experimental tools that have been developed to dissect neuronal circuits in rodents has made these species very attractive to vision scientists too, promoting a new tide of studies that have started to systematically explore visual functions in rats and mice. Rats, in particular, have been the subjects of several behavioral studies, aimed at assessing how advanced object recognition and shape processing is in this species. Here, I review these recent investigations, as well as earlier studies of rat pattern vision, to provide an historical overview and a critical summary of the status of the knowledge about rat object vision. The picture emerging from this survey is very encouraging with regard to the possibility of using rats as complementary models to monkeys in the study of higher-level vision.

Mechanisms of object recognition: what we have learned from pigeons

Behavioral studies of object recognition in pigeons have been conducted for 50 years, yielding a large body of data. Recent work has been directed toward synthesizing this evidence and understanding the visual, associative, and cognitive mechanisms that are involved. The outcome is that pigeons are likely to be the non-primate species for which the computational mechanisms of object recognition are best understood. Here, we review this research and suggest that a core set of mechanisms for object recognition might be present in all vertebrates, including pigeons and people, making pigeons an excellent candidate model to study the neural mechanisms of object recognition. Behavioral and computational evidence suggests that error-driven learning participates in object category learning by pigeons and people, and recent neuroscientific research suggests that the basal ganglia, which are homologous in these species, may implement error-driven learning of stimulus-response associations. Furthermore, learning of abstract category representations can be observed in pigeons and other vertebrates. Finally, there is evidence that feedforward visual processing, a central mechanism in models of object recognition in the primate ventral stream, plays a role in object recognition by pigeons. We also highlight differences between pigeons and people in object recognition abilities, and propose candidate adaptive specializations which may explain them, such as holistic face processing and rule-based category learning in primates. From a modern comparative perspective, such specializations are to be expected regardless of the model species under study. The fact that we have a good idea of which aspects of object recognition differ in people and pigeons should be seen as an advantage over other animal models. From this perspective, we suggest that there is much to learn about human object recognition from studying the "simple" brains of pigeons.

Functional specialization in rat occipital and temporal visual cortex

Recent studies have revealed a surprising degree of functional specialization in rodent visual cortex. Anatomically, suggestions have been made about the existence of hierarchical pathways with similarities to the ventral and dorsal pathways in primates. Here we aimed to characterize some important functional properties in part of the supposed "ventral" pathway in rats. We investigated the functional properties along a progression of five visual areas in awake rats, from primary visual cortex (V1) over lateromedial (LM), latero-intermediate (LI), and laterolateral (LL) areas up to the newly found lateral occipito-temporal cortex (TO). Response latency increased >20 ms from areas V1/LM/LI to areas LL and TO. Orientation and direction selectivity for the used grating patterns increased gradually from V1 to TO. Overall responsiveness and selectivity to shape stimuli decreased from V1 to TO and was increasingly dependent upon shape motion. Neural similarity for shapes could be accounted for by a simple computational model in V1, but not in the other areas. Across areas, we find a gradual change in which stimulus pairs are most discriminable. Finally, tolerance to position changes increased toward TO. These findings provide unique information about possible commonalities and differences between rodents and primates in hierarchical cortical processing.

The neural mechanisms for the recognition of face identity in humans

Every day we encounter dozens of people, and in order to interact with them appropriately we need to recognize their identity. The face is a crucial source of information to recognize a person’s identity. However, recognizing the identity of a face is challenging because it requires distinguishing between very similar images (e.g., the front views of two different faces) while categorizing very different images (e.g., a front view and a profile) as the same person. Neuroimaging has the whole-brain coverage needed to investigate where representations of face identity are encoded, but it is limited in terms of spatial and temporal resolution. In this article, we review recent neuroimaging research that attempted to investigate the representation of face identity, the challenges it faces, and the proposed solutions, to conclude that given the current state of the evidence the right anterior temporal lobe is the most promising candidate region for the representation of face identity.

Retinotopy versus face selectivity in macaque visual cortex

Retinotopic organization is a ubiquitous property of lower-tier visual cortical areas in human and nonhuman primates. In macaque visual cortex, the retinotopic maps extend to higher-order areas in the ventral visual pathway, including area TEO in the inferior temporal (IT) cortex. Distinct regions within IT cortex are also selective to specific object categories such as faces. Here we tested the topographic relationship between retinotopic maps and face-selective patches in macaque visual cortex using high-resolution fMRI and retinotopic face stimuli. Distinct subregions within face-selective patches showed either (1) a coarse retinotopic map of eccentricity and polar angle, (2) a retinotopic bias to a specific location of visual field, or (3) nonretinotopic selectivity. In general, regions along the lateral convexity of IT cortex showed more overlap between retinotopic maps and face selectivity, compared with regions within the STS. Thus, face patches in macaques can be subdivided into smaller patches with distinguishable retinotopic properties.

Processing multiple visual objects is limited by overlap in neural channels

High-level visual categories (e.g., faces, bodies, scenes, and objects) have separable neural representations across the visual cortex. Here, we show that this division of neural resources affects the ability to simultaneously process multiple items. In a behavioral task, we found that performance was superior when items were drawn from different categories (e.g., two faces/two scenes) compared to when items were drawn from one category (e.g., four faces). The magnitude of this mixed-category benefit depended on which stimulus categories were paired together (e.g., faces and scenes showed a greater behavioral benefit than objects and scenes). Using functional neuroimaging (i.e., functional MRI), we showed that the size of the mixed-category benefit was predicted by the amount of separation between neural response patterns, particularly within occipitotemporal cortex. These results suggest that the ability to process multiple items at once is limited by the extent to which those items are represented by separate neural populations.

The emergent executive: a dynamic field theory of the development of executive function

Executive function (EF) is a central aspect of cognition that undergoes significant changes in early childhood. Changes in EF in early childhood are robustly predictive of academic achievement and general quality of life measures later in adulthood. We present a dynamic neural field (DNF) model that provides a process-based account of behavior and developmental change in a key task used to probe the early development of executive function—the Dimensional Change Card Sort (DCCS) task. In the DCCS, children must flexibly switch from sorting cards either by shape or color to sorting by the other dimension. Typically, 3-year-olds, but not 5-year-olds, lack the flexibility to do so and perseverate on the first set of rules when instructed to switch. Using the DNF model, we demonstrate how rule-use and behavioral flexibility come about through a form of dimensional attention. Further, developmental change is captured by increasing the robustness and precision of dimensional attention. Note that although this enables the model to effectively switch tasks, the dimensional attention system does not “know” the details of task-specific performance. Rather, correct performance emerges as a property of system–wide interactions. We show how this captures children’s behavior in quantitative detail across 14 versions of the DCCS task. Moreover, we successfully test a set of novel predictions with 3-year-old children from a version of the task not explained by other theories.

Object concepts in the chemical senses

This paper examines the applicability of the object concept to the chemical senses, by evaluating them against a set of criteria for object-hood. Taste and chemesthesis do not generate objects. Their parts, perceptible from birth, never combine. Orthonasal olfaction (sniffing) presents a strong case for generating objects. Odorants have many parts yet they are perceived as wholes, this process is based on learning, and there is figure-ground segregation. While flavors are multimodal representations bound together by learning, there is no functional need for flavor objects in the mouth. Rather, food identification occurs prior to ingestion using the eye and nose, with the latter retrieving multimodal flavor objects via sniffing (e.g., sweet smelling caramel). While there are differences in object perception between vision, audition, and orthonasal olfaction, the commonalities suggest that the brain has adopted the same basic solution when faced with extracting meaning from complex stimulus arrays.

Can laptops be left inside passenger bags if motion imaging is used in X-ray security screening?

This paper describes a study where a new X-ray machine for security screening featuring motion imaging (i.e., 5 views of a bag are shown as an image sequence) was evaluated and compared to single view imaging available on conventional X-ray screening systems. More specifically, it was investigated whether with this new technology X-ray screening of passenger bags could be enhanced to such an extent that laptops could be left inside passenger bags, without causing a significant impairment in threat detection performance. An X-ray image interpretation test was created in four different versions, manipulating the factors packing condition (laptop and bag separate vs. laptop in bag) and display condition (single vs. motion imaging). There was a highly significant and large main effect of packing condition. When laptops and bags were screened separately, threat item detection was substantially higher. For display condition, a medium effect was observed. Detection could be slightly enhanced through the application of motion imaging. There was no interaction between display and packing condition, implying that the high negative effect of leaving laptops in passenger bags could not be fully compensated by motion imaging. Additional analyses were carried out to examine effects depending on different threat categories (guns, improvised explosive devices, knives, others), the placement of the threat items (in bag vs. in laptop) and viewpoint (easy vs. difficult view). In summary, although motion imaging provides an enhancement, it is not strong enough to allow leaving laptops in bags for security screening.