Neurocomputational bases of object and face recognition

The role of inversion and face masks on simultaneous and delayed face matching tasks

Although it is generally accepted that face recognition relies on holistic processing, it has been suggested that the simultaneous face matching task may depend on a more analytical or featural processing approach. However, empirical evidence supporting this claim is limited. In two experiments, we further explored the role of holistic and featural processing on simultaneous face matching by manipulating holistic processing through inversion and presenting faces with or without face masks. The results from Experiment 1 revealed that both inversion and face masks impaired matching performance. However, while the inversion effect was evident in both full-view and masked faces, the mask effect was only found in upright, but not inverted, faces. These results were replicated in Experiment 2 but, the inversion and mask effects were stronger in delayed face matching than in simultaneous face matching. Our findings suggest that simultaneous face matching relies on holistic processing, but to a smaller extent compared to higher memory-demanding identification tasks.

Why psychologists should embrace rather than abandon DNNs

Deep neural networks (DNNs) are powerful computational models, which generate complex, high-level representations that were missing in previous models of human cognition. By studying these high-level representations, psychologists can now gain new insights into the nature and origin of human high-level vision, which was not possible with traditional handcrafted models. Abandoning DNNs would be a huge oversight for psychological sciences.

Serial dependence in facial identity perception and visual working memory

Serial dependence (SD) refers to the effect in which a person's current perceptual judgment is attracted toward recent stimulus history. Perceptual and memory processes, as well as response and decisional biases, are thought to contribute to SD effects. The current study examined the processing stages of SD facial identity effects in the context of task-related decision processes and how such effects may differ from visual working memory (VWM) interactions. In two experiments, participants were shown a series of two sequentially presented face images. In Experiment 1, the two faces were separated by an interstimulus interval (ISI) of 1, 3, 6, or 10 s, and participants were instructed to reproduce the second face after a varying response delay of 0, 1, 3, 6, or 10 s. Results showed that SD effects occurred most consistently at ISI of 1 s and response delays of 1 and 6 s consistent with early and late stages of processing. In Experiment 2, the ISI was held constant at 1 s, and to separate SD from VWM interactions participants were post-cued to reproduce either the first or the second face. When the second face was the target, SD effects again occurred at response delays of 1 and 6 s, but not when the first face was the target. Together, the results demonstrates that SD facial identity effects occur independently of task-related processes in a distinct temporal fashion and suggest that SD and VWM interactions may rely on separate underlying mechanisms.

Underfunding Basic Psychological Science Because of the Primacy of the Here and Now: A Scientific Conundrum

The psychological sciences are suffering from the primacy of the "here and now" and from a radical utilitarian view of science-this is certainly true in less affluent countries around the world. Portugal is a particular case study, as both psychology departments and funding agencies are largely biased toward applied psychology-but this is a more global trend. The field needs to find a balance between applied and basic psychology to better respond to the challenges of tomorrow.

CNNs reveal the computational implausibility of the expertise hypothesis

Face perception has long served as a classic example of domain specificity of mind and brain. But an alternative "expertise" hypothesis holds that putatively face-specific mechanisms are actually domain-general, and can be recruited for the perception of other objects of expertise (e.g., cars for car experts). Here, we demonstrate the computational implausibility of this hypothesis: Neural network models optimized for generic object categorization provide a better foundation for expert fine-grained discrimination than do models optimized for face recognition.

Contributions of low- and high-level contextual mechanisms to human face perception

Contextual modulations at primary stages of visual processing depend on the strength of local input. Contextual modulations at high-level stages of (face) processing show a similar dependence to local input strength. Namely, the discriminability of a facial feature determines the amount of influence of the face context on that feature. How high-level contextual modulations emerge from primary mechanisms is unclear due to the scarcity of empirical research systematically addressing the functional link between the two. We tested (62) young adults' ability to process local input independent of the context using contrast detection and (upright and inverted) morphed facial feature matching tasks. We first investigated contextual modulation magnitudes across tasks to address their shared variance. A second analysis focused on the profile of performance across contextual conditions. In upright eye matching and contrast detection tasks, contextual modulations only correlated at the level of their profile (averaged Fisher-Z transformed r = 1.18, BF10 > 100), but not magnitude (r = .15, BF10 = .61), suggesting the functional independence but similar working principles of the mechanisms involved. Both the profile (averaged Fisher-Z transformed r = .32, BF10 = 9.7) and magnitude (r = .28, BF10 = 4.58) of the contextual modulations correlated between inverted eye matching and contrast detection tasks. Our results suggest that non-face-specialized high-level contextual mechanisms (inverted faces) work in connection to primary contextual mechanisms, but that the engagement of face-specialized mechanisms for upright faces obscures this connection. Such combined study of low- and high-level contextual modulations sheds new light on the functional relationship between different levels of the visual processing hierarchy, and thus on its functional organization.

Face dissimilarity judgments are predicted by representational distance in morphable and image-computable models

Discerning the subtle differences between individuals’ faces is crucial for social functioning. It requires us not only to solve general challenges of object recognition (e.g., invariant recognition over changes in view or lighting) but also to be attuned to the specific ways in which face structure varies. Three-dimensional morphable models based on principal component analyses of real faces provide descriptions of statistical differences between faces, as well as tools to generate novel faces. We rendered large sets of realistic face pairs from such a model and collected similarity and same/different identity judgments. The statistical model predicted human perception as well as state-of-the-art image-computable neural networks. Results underscore the statistical tuning of face encoding.

Human vision is attuned to the subtle differences between individual faces. Yet we lack a quantitative way of predicting how similar two face images look and whether they appear to show the same person. Principal component–based three-dimensional (3D) morphable models are widely used to generate stimuli in face perception research. These models capture the distribution of real human faces in terms of dimensions of physical shape and texture. How well does a “face space” based on these dimensions capture the similarity relationships humans perceive among faces? To answer this, we designed a behavioral task to collect dissimilarity and same/different identity judgments for 232 pairs of realistic faces. Stimuli sampled geometric relationships in a face space derived from principal components of 3D shape and texture (Basel face model [BFM]). We then compared a wide range of models in their ability to predict the data, including the BFM from which faces were generated, an active appearance model derived from face photographs, and image-computable models of visual perception. Euclidean distance in the BFM explained both dissimilarity and identity judgments surprisingly well. In a comparison against 16 diverse models, BFM distance was competitive with representational distances in state-of-the-art deep neural networks (DNNs), including novel DNNs trained on BFM synthetic identities or BFM latents. Models capturing the distribution of face shape and texture across individuals are not only useful tools for stimulus generation. They also capture important information about how faces are perceived, suggesting that human face representations are tuned to the statistical distribution of faces.

The sizable difficulty in matching unfamiliar faces differing only moderately in orientation in depth is a function of image dissimilarity

Attempting to match unfamiliar, highly similar faces at moderate differences in orientation in depth is surprisingly difficult. No neurocomputational account of these costs that addressed the representation of faces by which a face-similarity metric can be derived has been offered. A metric specifying the similarity of the to-be-distinguished faces is required as the rotation costs will be a function of the difficulty in distinguishing the faces. Consequently, rotation costs have typically been described in terms of angle of disparity, rather than the dissimilarity of the faces produced by the rotation. We assessed the effects of orientation disparity in a match-to-sample paradigm of a simultaneous presentation of a triangular display of three faces. Two lower test faces, a matching face and a foil, were always at the same orientation and differed by 0° to 20° from the sample on top. The similarity of the images was scaled by a model based on simple cell tuning, modeled as Gabor wavelets, that correlates almost perfectly with psychophysical similarity. Two measures of face similarity, with approximately additive effects on reaction times, accounted for matching performance: a) the decrease in similarity between the images of the matching and sample faces produced by increases in their orientation disparity, and b) the similarity between the matching face and the selection of a particular foil. The 20° orientation disparity was sufficient to yield a sizeable 301 msec increase in reaction time. An implication of the results is that the activity in V1 produced by viewing a face is fed forward to areas responsible for the individuation of that face.

Perception of Dynamic Point Light Facial Expression

This study uses point light displays both to investigate the roles of global and local motion analyses in the perception of dynamic facial expressions and to measure the information threshold for reliable recognition of emotions. We videotaped the faces of actors wearing black makeup with white dots while they dynamically produced each of 6 basic Darwin/Ekman emotional expressions. The number of point lights was varied to systematically manipulate amount of information available. For all but one of the expressions, discriminability (d′) increased approximately linearly with number of point lights, with most remaining largely discriminable with as few as only 6 point lights. This finding supports reliance on global motion patterns produced by facial muscles. However, discriminability for the happy expression was notably higher and largely unaffected by number of point lights and thus appears to rely on characteristic local motion, probably the unique upward curvature of the mouth. The findings indicate that recognition of facial expression is not a unitary process and that different expressions may be conveyed by different perceptual information, but in general, basic facial emotional expressions typically remain largely discriminable with as few as 6 dynamic point lights.

Implementation-Independent Representation for Deep Convolutional Neural Networks and Humans in Processing Faces

Deep convolutional neural networks (DCNN) nowadays can match human performance in challenging complex tasks, but it remains unknown whether DCNNs achieve human-like performance through human-like processes. Here we applied a reverse-correlation method to make explicit representations of DCNNs and humans when performing face gender classification. We found that humans and a typical DCNN, VGG-Face, used similar critical information for this task, which mainly resided at low spatial frequencies. Importantly, the prior task experience, which the VGG-Face was pre-trained to process faces at the subordinate level (i.e., identification) as humans do, seemed necessary for such representational similarity, because AlexNet, a DCNN pre-trained to process objects at the basic level (i.e., categorization), succeeded in gender classification but relied on a completely different representation. In sum, although DCNNs and humans rely on different sets of hardware to process faces, they can use a similar and implementation-independent representation to achieve the same computation goal.

The visual and semantic features that predict object memory: Concept property norms for 1,000 object images

Humans have a remarkable fidelity for visual long-term memory, and yet the composition of these memories is a longstanding debate in cognitive psychology. While much of the work on long-term memory has focused on processes associated with successful encoding and retrieval, more recent work on visual object recognition has developed a focus on the memorability of specific visual stimuli. Such work is engendering a view of object representation as a hierarchical movement from low-level visual representations to higher level categorical organization of conceptual representations. However, studies on object recognition often fail to account for how these high- and low-level features interact to promote distinct forms of memory. Here, we use both visual and semantic factors to investigate their relative contributions to two different forms of memory of everyday objects. We first collected normative visual and semantic feature information on 1,000 object images. We then conducted a memory study where we presented these same images during encoding (picture target) on Day 1, and then either a Lexical (lexical cue) or Visual (picture cue) memory test on Day 2. Our findings indicate that: (1) higher level visual factors (via DNNs) and semantic factors (via feature-based statistics) make independent contributions to object memory, (2) semantic information contributes to both true and false memory performance, and (3) factors that predict object memory depend on the type of memory being tested. These findings help to provide a more complete picture of what factors influence object memorability. These data are available online upon publication as a public resource.

FFA and OFA Encode Distinct Types of Face Identity Information

Faces of different people elicit distinct fMRI patterns in several face-selective regions of the human brain. Here we used representational similarity analysis to investigate what type of identity-distinguishing information is encoded in three face-selective regions: fusiform face area (FFA), occipital face area (OFA), and posterior superior temporal sulcus (pSTS). In a sample of 30 human participants (22 females, 8 males), we used fMRI to measure brain activity patterns elicited by naturalistic videos of famous face identities, and compared their representational distances in each region with models of the differences between identities. We built diverse candidate models, ranging from low-level image-computable properties (pixel-wise, GIST, and Gabor-Jet dissimilarities), through higher-level image-computable descriptions (OpenFace deep neural network, trained to cluster faces by identity), to complex human-rated properties (perceived similarity, social traits, and gender). We found marked differences in the information represented by the FFA and OFA. Dissimilarities between face identities in FFA were accounted for by differences in perceived similarity, Social Traits, Gender, and by the OpenFace network. In contrast, representational distances in OFA were mainly driven by differences in low-level image-based properties (pixel-wise and Gabor-Jet dissimilarities). Our results suggest that, although FFA and OFA can both discriminate between identities, the FFA representation is further removed from the image, encoding higher-level perceptual and social face information.SIGNIFICANCE STATEMENT Recent studies using fMRI have shown that several face-responsive brain regions can distinguish between different face identities. It is however unclear whether these different face-responsive regions distinguish between identities in similar or different ways. We used representational similarity analysis to investigate the computations within three brain regions in response to naturalistically varying videos of face identities. Our results revealed that two regions, the fusiform face area and the occipital face area, encode distinct identity information about faces. Although identity can be decoded from both regions, identity representations in fusiform face area primarily contained information about social traits, gender, and high-level visual features, whereas occipital face area primarily represented lower-level image features.

Capturing human categorization of natural images by combining deep networks and cognitive models

Human categorization is one of the most important and successful targets of cognitive modeling, with decades of model development and assessment using simple, low-dimensional artificial stimuli. However, it remains unclear how these findings relate to categorization in more natural settings, involving complex, high-dimensional stimuli. Here, we take a step towards addressing this question by modeling human categorization over a large behavioral dataset, comprising more than 500,000 judgments over 10,000 natural images from ten object categories. We apply a range of machine learning methods to generate candidate representations for these images, and show that combining rich image representations with flexible cognitive models captures human decisions best. We also find that in the high-dimensional representational spaces these methods generate, simple prototype models can perform comparably to the more complex memory-based exemplar models dominant in laboratory settings.

Theories of human categorization have traditionally been evaluated in the context of simple, low-dimensional stimuli. In this work, the authors use a large dataset of human behavior over 10,000 natural images to re-evaluate these theories, revealing interesting differences from previous results.

Does stereopsis improve face identification? A study using a virtual reality display with integrated eye-tracking and pupillometry

Stereopsis is a powerful depth cue for humans, which may also contribute to object recognition. In particular, we surmise that face identification would benefit from the availability of stereoscopic depth cues, since facial perception may be based on three-dimensional (3D) representations. In this study, a virtual reality (VR) headset with integrated eye-tracking was used to present stereoscopic images of faces. As a monoscopic contrast condition, identical images of faces were displayed to the two eyes. We monitored the participants' gaze behavior and pupil diameters while they performed a sample-to-match face identification task. We found that accuracy was superior in the stereoscopic condition compared to the monoscopic condition for frontal and intermediate views, but not profiles. Moreover, pupillary diameters were smaller when identifying stereoscopically seen faces than when viewing them without stereometric cues, which we interpret as lower processing load for the former than the latter conditions. The analysis of gaze showed that participants tended to focus on regions of the face rich in volumetric information, more so in the stereoscopic condition than the monoscopic condition. Together, these findings suggest that a 3D representation of faces may be the natural format used by the visual system when assessing face identity. Stereoscopic information, by providing depth information, assists the construction of robust facial representations in memory.

Face-Specific Perceptual Distortions Reveal A View- and Orientation-Independent Face Template

The spatial coordinate system in which a stimulus representation is embedded is known as its reference frame. Every visual representation has a reference frame [1], and the visual system uses a variety of reference frames to efficiently code visual information [e.g., 1-5]. The representation of faces in early stages of visual processing depends on retino-centered reference frames, but little is known about the reference frames that code the high-level representations used to make judgements about faces. Here, we focus on a rare and striking disorder of face perception-hemi-prosopometamorphopsia (hemi-PMO)-to investigate these reference frames. After a left splenium lesion, Patient A.D. perceives features on the right side of faces as if they had melted. The same features were distorted when faces were presented in either visual field, at different in-depth rotations, and at different picture-plane orientations including upside-down. A.D.'s results indicate faces are aligned to a view- and orientation-independent face template encoded in a face-centered reference frame, that these face-centered representations are present in both the left and right hemisphere, and that the representations of the left and right halves of a face are dissociable.

Repetition priming with no antipriming in picture identification

Previous studies have shown that the processing of a stimulus is facilitated when that stimulus is repeated compared to when it appears the first time, and this phenomenon is called repetition priming (RP). One explanation for RP is that initial processing of a stimulus strengthens connections within the visual representation, enabling subsequent processing of the same stimulus to be more efficient. More recently, it has been reported that presenting an object with features that overlap with those in a subsequent stimulus impairs the latter's processing, and this cost is termed antipriming (AP). AP is said to be the natural antithesis of RP, and it manifests when two objects share component features, thereby having overlapping representations. In two experiments, we investigated RP and AP in a picture naming task. Following previous research, we used a 4-phase paradigm, in which RP and AP were measured, respectively, by an increase or a decrease in performance for repeated or novel stimuli in Phase 4 compared with the baseline performance in Phase 2. We used a fully randomized design in Experiment 1, and a pseudo-randomized design in stimulus selection but a randomized design in presentation location in Experiment 2. We found robust RP in both experiments, but neither experiment showed any evidence of AP. Our results indicate that RP and AP do not always manifest within the same experiment, and that the relationship between these two effects may be more complex than previously understood.

Emotional learning promotes perceptual predictions by remodeling stimulus representation in visual cortex

Emotions exert powerful effects on perception and memory, notably by modulating activity in sensory cortices so as to capture attention. Here, we examine whether emotional significance acquired by a visual stimulus can also change its cortical representation by linking neuronal populations coding for different memorized versions of the same stimulus, a mechanism that would facilitate recognition across different appearances. Using fMRI, we show that after pairing a given face with threat through conditioning, viewing this face activates the representation of another viewpoint of the same person, which itself was never conditioned, leading to robust repetition-priming across viewpoints in the ventral visual stream (including medial fusiform, lateral occipital, and anterior temporal cortex). We also observed a functional-anatomical segregation for coding view-invariant and view-specific identity information. These results indicate emotional signals may induce plasticity of stimulus representations in visual cortex, serving to generate new sensory predictions about different appearances of threat-associated stimuli.

Anisotropic visual awareness of shapes

While object perception may feel instantaneous, it is an iterative process in which information is accumulated until ambiguity about identity and location is resolved. In theory, awareness of an object should depend on how efficiently this process occurs. Therefore, objects with inherently weak visual representations should be more susceptible to perceptual disruption. We tested this hypothesis by examining the perception of aspect ratio, a 2D feature of shapes with anisotropic representation (circular shapes are less robustly represented than elongated shapes in high-level visual areas). Observers viewed a target shape shown for 20-ms within an array of ellipses. The target, which varied from flat to tall, was either masked or unmasked. Observers indicated the target's aspect ratio and if it was visible. Observers reported seeing elongated shapes far more often than circular shapes, but only on trials with object-substitution masking. This effect replicated across five control experiments, even though the shapes were identical in basic image attributes (e.g., contrast, area). Our findings demonstrate that shapes with extreme aspect ratios are more readily available to awareness than shapes with ambiguous dimensionality. More generally, this work supports theories of object processing which suggest that strength of visual representation gates access to awareness.

Does Extensive Training at Individuating Novel Objects in Adulthood Lead to Visual Expertise? The Role of Facelikeness

Human adults have a rich visual experience thanks to seeing human faces since birth, which may contribute to the acquisition of perceptual processes that rapidly and automatically individuate faces. According to a generic visual expertise hypothesis, extensive experience with nonface objects may similarly lead to efficient processing of objects at the individual level. However, whether extensive training in adulthood leads to visual expertise remains debated. One key issue is the extent to which the acquisition of visual expertise depends on the resemblance of objects to faces in terms of the spatial configuration of parts. We therefore trained naive human adults to individuate a large set of novel parametric multipart objects. Critically, one group of participants trained with the objects in a "facelike" stimulus orientation, whereas a second group trained with the same objects but with the objects rotated 180° in the picture plane into a "nonfacelike" orientation. We used a fast periodic visual stimulation EEG protocol to objectively quantify participants' ability to discriminate untrained exemplars before and after training. EEG responses associated with the frequency of identity change in a fast stimulation sequence, which reflects rapid and automatic perceptual processes, were observed over lateral occipital sites for both groups before training. There was a significant, albeit small, increase in these responses after training but only for the facelike group and only to facelike stimuli. Our findings indicate that perceived facelikeness plays a role in visual expertise and highlight how the adult perceptual system exploits familiar spatial configurations when learning new object categories.

An applet for the Gabor similarity scaling of the differences between complex stimuli

It is widely accepted that after the first cortical visual area, V1, a series of stages achieves a representation of complex shapes, such as faces and objects, so that they can be understood and recognized. A major challenge for the study of complex shape perception has been the lack of a principled basis for scaling of the physical differences between stimuli so that their similarity can be specified, unconfounded by early-stage differences. Without the specification of such similarities, it is difficult to make sound inferences about the contributions of later stages to neural activity or psychophysical performance. A Web-based app is described that is based on the Malsburg Gabor-jet model (Lades et al., 1993), which allows easy specification of the V1 similarity of pairs of stimuli, no matter how intricate. The model predicts the psycho physical discriminability of metrically varying faces and complex blobs almost perfectly (Yue, Biederman, Mangini, von der Malsburg, & Amir, 2012), and serves as the input stage of a large family of contemporary neurocomputational models of vision.

Comparing object recognition from binary and bipolar edge images for visual prostheses

Visual prostheses require an effective representation method due to the limited display condition which has only 2 or 3 levels of grayscale in low resolution. Edges derived from abrupt luminance changes in images carry essential information for object recognition. Typical binary (black and white) edge images have been used to represent features to convey essential information. However, in scenes with a complex cluttered background, the recognition rate of the binary edge images by human observers is limited and additional information is required. The polarity of edges and cusps (black or white features on a gray background) carries important additional information; the polarity may provide shape from shading information missing in the binary edge image. This depth information may be restored by using bipolar edges. We compared object recognition rates from 16 binary edge images and bipolar edge images by 26 subjects to determine the possible impact of bipolar filtering in visual prostheses with 3 or more levels of grayscale. Recognition rates were higher with bipolar edge images and the improvement was significant in scenes with complex backgrounds. The results also suggest that erroneous shape from shading interpretation of bipolar edges resulting from pigment rather than boundaries of shape may confound the recognition.

Removing Eyebrows Impairs Recognition of Famous Faces, or Doesn't, Depending on How the Eyebrows are Removed

Matching photos of famous people's faces with their names was as fast with photos that had eyebrows masked by adhesive plaster as with unretouched original photos. But matching was slower with photos that had eyebrows erased and replaced with adjacent skin texture and colour than with original photos. The conjecture that erasure impairs recognition because it alters a face's configuration was examined by repeating the experiment with the photos shown upside down, the rationale being that because configural information is difficult to encode from inverted faces the erasure effect should diminish. With inverted faces, matching was no different for original, eyebrows-masked, and eyebrows-erased photos. Eyebrows appear to be less important for face recognition as informative parts and features than as sources of information about a face's configuration.

When Feature Information Comes First! Early Processing of Inverted Faces

We investigated the early stages of face recognition and the role of featural and holistic face information. We exploited the fact that, on inversion, the alienating disorientation of the eyes and mouth in thatcherised faces is hardly detectable. This effect allows featural and holistic information to be dissociated and was used to test specific face-processing hypotheses. In inverted thatcherised faces, the cardinal features are already correctly oriented, whereas in undistorted faces, the whole Gestalt is coherent but all information is disoriented. Experiment 1 and experiment 3 revealed that, for inverted faces, featural information processing precedes holistic information. Moreover, the processing of contextual information is necessary to process local featural information within a short presentation time (26 ms). Furthermore, for upright faces, holistic information seems to be available faster than for inverted faces (experiment 2). These differences in processing inverted and upright faces presumably cause the differential importance of featural and holistic information for inverted and upright faces.

Do Humans and Baboons Use the Same Information When Categorizing Human and Baboon Faces?

What information is used for sorting pictures of complex stimuli into categories? We applied a reverse correlation method to reveal the visual features mediating categorization in humans and baboons. Two baboons and 6 humans were trained to sort, by species, pictures of human and baboon faces on which random visual noise was superimposed. On ambiguous probe trials, a human-baboon morph was presented, eliciting “human” responses on some trials and “baboon” responses on others. The difference between the noise patterns that induced the two responses made explicit the information mediating the classification. Unlike the humans, the baboons based their categorization on information that closely matched that used by a theoretical observer responding solely on the basis of the pixel similarities between the probe and training images. We show that the classification-image technique and principal components analysis provide a method to make explicit the differences in the information mediating categorization in humans and animals.

Neural Tuning Size in a Model of Primate Visual Processing Accounts for Three Key Markers of Holistic Face Processing

Faces are an important and unique class of visual stimuli, and have been of interest to neuroscientists for many years. Faces are known to elicit certain characteristic behavioral markers, collectively labeled “holistic processing”, while non-face objects are not processed holistically. However, little is known about the underlying neural mechanisms. The main aim of this computational simulation work is to investigate the neural mechanisms that make face processing holistic. Using a model of primate visual processing, we show that a single key factor, “neural tuning size”, is able to account for three important markers of holistic face processing: the Composite Face Effect (CFE), Face Inversion Effect (FIE) and Whole-Part Effect (WPE). Our proof-of-principle specifies the precise neurophysiological property that corresponds to the poorly-understood notion of holism, and shows that this one neural property controls three classic behavioral markers of holism. Our work is consistent with neurophysiological evidence, and makes further testable predictions. Overall, we provide a parsimonious account of holistic face processing, connecting computation, behavior and neurophysiology.

Comparing object recognition from binary and bipolar edge features

Edges derived from abrupt luminance changes in images carry essential information for object recognition. Typical binary edge images (black edges on white background or white edges on black background) have been used to represent features (edges and cusps) in scenes. However, the polarity of cusps and edges may contain important depth information (depth from shading) which is lost in the binary edge representation. This depth information may be restored, to some degree, using bipolar edges. We compared recognition rates of 16 binary edge images, or bipolar features, by 26 subjects. Object recognition rates were higher with bipolar edges and the improvement was significant in scenes with complex backgrounds.

Neural microgenesis of personally familiar face recognition

Despite a wealth of information provided by neuroimaging research, the neural basis of familiar face recognition in humans remains largely unknown. Here, we isolated the discriminative neural responses to unfamiliar and familiar faces by slowly increasing visual information (i.e., high-spatial frequencies) to progressively reveal faces of unfamiliar or personally familiar individuals. Activation in ventral occipitotemporal face-preferential regions increased with visual information, independently of long-term face familiarity. In contrast, medial temporal lobe structures (perirhinal cortex, amygdala, hippocampus) and anterior inferior temporal cortex responded abruptly when sufficient information for familiar face recognition was accumulated. These observations suggest that following detailed analysis of individual faces in core posterior areas of the face-processing network, familiar face recognition emerges categorically in medial temporal and anterior regions of the extended cortical face network.

On the particular vulnerability of face recognition to aging: a review of three hypotheses

Age-related face recognition deficits are characterized by high false alarms to unfamiliar faces, are not as pronounced for other complex stimuli, and are only partially related to general age-related impairments in cognition. This paper reviews some of the underlying processes likely to be implicated in theses deficits by focusing on areas where contradictions abound as a means to highlight avenues for future research. Research pertaining to the three following hypotheses is presented: (i) perceptual deterioration, (ii) encoding of configural information, and (iii) difficulties in recollecting contextual information. The evidence surveyed provides support for the idea that all three factors are likely to contribute, under certain conditions, to the deficits in face recognition seen in older adults. We discuss how these different factors might interact in the context of a generic framework of the different stages implicated in face recognition. Several suggestions for future investigations are outlined.

Automatic guidance of attention during real-world visual search

Looking for objects in cluttered natural environments is a frequent task in everyday life. This process can be difficult, because the features, locations, and times of appearance of relevant objects often are not known in advance. Thus, a mechanism by which attention is automatically biased toward information that is potentially relevant may be helpful. We tested for such a mechanism across five experiments by engaging participants in real-world visual search and then assessing attentional capture for information that was related to the search set but was otherwise irrelevant. Isolated objects captured attention while preparing to search for objects from the same category embedded in a scene, as revealed by lower detection performance (Experiment 1A). This capture effect was driven by a central processing bottleneck rather than the withdrawal of spatial attention (Experiment 1B), occurred automatically even in a secondary task (Experiment 2A), and reflected enhancement of matching information rather than suppression of nonmatching information (Experiment 2B). Finally, attentional capture extended to objects that were semantically associated with the target category (Experiment 3). We conclude that attention is efficiently drawn towards a wide range of information that may be relevant for an upcoming real-world visual search. This mechanism may be adaptive, allowing us to find information useful for our behavioral goals in the face of uncertainty.

Proto-object categorisation and local gist vision using low-level spatial features

Object categorisation is a research area with significant challenges, especially in conditions with bad lighting, occlusions, different poses and similar objects. This makes systems that rely on precise information unable to perform efficiently, like a robotic arm that needs to know which objects it can reach. We propose a biologically inspired object detection and categorisation framework that relies on robust low-level object shape. Using only edge conspicuity and disparity features for scene figure-ground segregation and object categorisation, a trained neural network classifier can quickly categorise broad object families and consequently bootstrap a low-level scene gist system. We argue that similar processing is possibly located in the parietal pathway leading to the LIP cortex and, via areas V5/MT and MST, providing useful information to the superior colliculus for eye and head control.

Relations among early object recognition skills: Objects and letters

Human visual object recognition is multifaceted, with several domains of expertise. Developmental relations between young children's letter recognition and their 3-dimensional object recognition abilities are implicated on several grounds but have received little research attention. Here, we ask how preschoolers' success in recognizing letters relates to their ability to recognize 3-dimensional objects from sparse shape information alone. A relation is predicted because perception of the spatial relations is critical in both domains. Seventy-three 2 ½- to 4-year-old children completed a Letter Recognition task, measuring the ability to identify a named letter among 3 letters with similar shapes, and a "Shape Caricature Recognition" task, measuring recognition of familiar objects from sparse, abstract information about their part shapes and the spatial relations among those parts. Children also completed a control "Shape Bias" task, in which success depends on recognition of overall object shape but not of relational structure. Children's success in letter recognition was positively related to their shape caricature recognition scores, but not to their shape bias scores. The results suggest that letter recognition builds upon developing skills in attending to and representing the relational structure of object shape, and that these skills are common to both 2-dimensional and 3-dimensional object perception.

Global Shape Information Increases but Color Information Decreases the Composite Face Effect

The separation of visual shape and surface information may be useful for understanding holistic face perception—that is, the perception of a face as a single unit (Jiang, Blanz, & Rossion, 2011, Visual Cognition, 19, 1003–1034). A widely used measure of holistic face perception is the composite face effect (CFE), in which identical top face halves appear different when aligned with bottom face halves from different identities. In the present study the influences of global face shape (ie contour of the face) and color information on the CFE are investigated, with the hypothesis that global face shape supports but color impairs holistic face perception as measured in this paradigm. In experiment 1 the CFE is significantly increased when face stimuli possess natural global shape information than when cropped to a generic (ie oval) global shape; this effect is not found when the stimuli are presented inverted. In experiment 2 the CFE is significantly decreased when face stimuli are presented with color information than when presented in grayscale. These findings indicate that grayscale stimuli maintaining natural global face shape information provide the most adept measure of holistic face perception in the behavioral composite face paradigm. More generally, they show that reducing different types of information diagnostic for individual face perception can have opposite effects on the CFE, illustrating the functional dissociation between shape and surface information in face perception.

Emotion recognition (sometimes) depends on horizontal orientations

Face recognition depends critically on horizontal orientations (Goffaux & Dakin, Frontiers in Psychology, 1(143), 1-14, 2010): Face images that lack horizontal features are harder to recognize than those that have this information preserved. We asked whether facial emotional recognition also exhibits this dependency by asking observers to categorize orientation-filtered happy and sad expressions. Furthermore, we aimed to dissociate image-based orientation energy from object-based orientation by rotating images 90 deg in the picture plane. In our first experiment, we showed that the perception of emotional expression does depend on horizontal orientations, and that object-based orientation constrained performance more than image-based orientation did. In Experiment 2, we showed that mouth openness (i.e., open vs. closed mouths) also influenced the emotion-dependent reliance on horizontal information. Finally, we describe a simple computational analysis that demonstrates that the impact of mouth openness was not predicted by variation in the distribution of orientation energy across horizontal and vertical orientation bands. Overall, our results suggest that emotion recognition largely does depend on horizontal information defined relative to the face, but that this bias is modulated by multiple factors that introduce variation in appearance across and within distinct emotions.

Holistic processing and reliance on global viewing strategies in older adults' face perception

There is increasing evidence that face recognition might be impaired in older adults, but it is unclear whether the impairment is truly perceptual, and face specific. In order to address this question we compared performance in same/different matching tasks with face and non-face objects (watches) among young (mean age 23.7) and older adults (mean age 70.4) using a context congruency paradigm (Meinhardt-Injac, Persike & Meinhardt, 2010, Meinhardt-Injac, Persike and Meinhardt, 2011a). Older adults were less accurate than young adults with both object classes, while face matching was notably impaired. Effects of context congruency and inversion, measured as the hallmarks of holistic processing, were equally strong in both age groups, and were found only for faces, but not for watches. The face specific decline in older adults revealed deficits in handling internal facial features, while young adults matched external and internal features equally well. Comparison with non-face stimuli showed that this decline was face specific, and did not concern processing of object features in general. Taken together, the results indicate no age-related decline in the capabilities to process faces holistically. Rather, strong holistic effects, combined with a loss of precision in handling internal features indicate that older adults rely on global viewing strategies for faces. At the same time, access to the exact properties of inner face details becomes restricted.

Dissociation of part-based and integrated neural responses to faces by means of electroencephalographic frequency tagging

In order to isolate the repetition suppression effects for each part of a whole-face stimulus, the left and right halves of face stimuli were flickered at different frequency rates (5.88 or 7.14 Hz), changing or not changing identity at every stimulation cycle. The human electrophysiological (electroencephalographic) responses to each face half increased in amplitude when different rather than repeated face half identities were presented at every stimulation cycle. Contrary to the repetition suppression effects for whole faces, which are usually found over the right occipito-temporal cortex, these part-based repetition suppression effects were found on all posterior electrode sites and were unchanged when the two face halves were manipulated by separation, lateral misalignment, or inversion. In contrast, intermodulation components (e.g. 7.14-5.88 = 1.26 Hz) were found mainly over the right occipito-temporal cortex and were significantly reduced following the aforementioned manipulations. In addition, the intermodulation components decreased substantially for face halves belonging to different identities, which form a less coherent face than when they belong to the same face identity. These observations provide objective evidence for dissociation between part-based and integrated (i.e. holistic/configural) responses to faces in the human brain, suggesting that only responses to integrated face parts reflect high-level, possibly face-specific, representations.

Development of visual systems for faces and objects: further evidence for prolonged development of the face system

Background

The development of face and object processing has attracted much attention; however, studies that directly compare processing of both visual categories across age are rare. In the present study, we compared the developmental trajectories of face and object processing in younger children (8–10 years), older children (11–13 years), adolescents (14–16 years), and adults (20–37).

Methodology/Principal Findings

We used a congruency paradigm in which subjects compared the internal features of two stimuli, while the (unattended) external features either agreed or disagreed independent of the identity of the internal features. We found a continuous increase in matching accuracy for faces and watches across childhood and adolescence, with different magnitudes for both visual categories. In watch perception, adult levels were reached at the age of 14–16, but not in face perception. The effect of context and inversion, as measures of holistic and configural processing, were clearly restricted to faces in all age groups. This finding suggests that different mechanisms are involved in face and object perception at any age tested. Moreover, the modulation of context and inversion effects by exposure duration was strongly age-dependent, with the strongest age-related differences found for brief timings below 140 ms.

Conclusions/Significance

The results of the present study suggest prolonged development of face-specific processing up to young adulthood. The improvement in face processing is qualitatively different from the improvement of general perceptual and cognitive ability.

Individuals with 22q11.2 deletion syndrome are impaired at explicit, but not implicit, discrimination of local forms embedded in global structures

Individuals with 22q11.2 deletion syndrome (22q11.2DS) are impaired at exploring visual information in space; however, not much is known about visual form discrimination in the syndrome. Thirty-five individuals with 22q11.2DS and 41 controls completed a form discrimination task with global forms made up of local elements. Affected individuals demonstrated clear impairment in detecting local, but not global, differences. Nevertheless, 22q11.2DS participants easily discriminated the same local elements when they were displayed in isolation, and further use of a prime demonstrated preserved facilitation of local processing in 22q11.2DS. These results did not differ by age or IQ. This study illustrates the impact of visuospatial impairments on form discrimination, and suggests how these difficulties may affect visual scanning in 22q11.2DS.

Face perception is whole or none: disentangling the role of spatial contiguity and interfeature distances in the composite face illusion

Compelling evidence that faces are perceived holistically or configurally comes from the composite face illusion: identical top halves of a face are perceived as being different if they are aligned with different bottom halves. The visual illusion disappears when the top and bottom face halves are spatially misaligned. Whether this is because the two halves no longer form a whole face (ie they form two segmented parts), or because of an increase in interfeatures distance in the misaligned condition (eg eyes-mouth distance) remains unclear. Here, thirty-four participants performed a delayed matching composite task in which the amount of spatial misalignment between face halves varied parametrically (from 8.33% of face width to 100%). The difference in performance between aligned and misaligned faces (ie the composite face effect) was already of full magnitude at the smallest level of misalignment. These results imply that a small spatial misalignment is sufficient to measure the composite face effect. From a theoretical standpoint, they indicate that it is the breaking of a whole configuration rather than the increase in relative distance between the face parts that explains the presence or absence of the composite face effect, clarifying an outstanding issue concerning the nature of holistic face perception.

The face inversion effect in infants is driven by high, and not low, spatial frequencies

To investigate the mechanisms underlying development of upright face preferences in infants, the current study measured inversion effects for faces that were spatial frequency (SF) filtered, into low SF and high SF, with the notion that different SFs are analyzed by different visual mechanisms. For comparison to faces, we used object stimuli that consisted of pictures of strollers. In 4 month olds, 8 month olds, and adults, we measured the strength of the selective face inversion effect (sFIE), operationally defined as an upright over inverted looking preference that is greater for faces than objects. In Study 1, we employed unfiltered stimuli, and found a clear sFIE in both infants and adults. To determine what drove this sFIE, in Study 2, the sFIE was measured for low-SF and high-SF stimuli, with all stimuli being equated for visibility. For adults, the sFIE was equally strong for low-SF and high-SF stimuli. A different pattern was seen for infants. Infants exhibited a significantly greater sFIE for high-SF, than for low-SF, stimuli (and only for high SF was the sFIE significant). In fact, the strength of infants' upright face preference for high-SF stimuli was indistinguishable from that observed for unfiltered faces, indicating that in natural (unfiltered) stimuli, high SFs are sufficient to account for infants' upright face preferences.