Are face representations viewpoint dependent? A stereo advantage for generalizing across different views of faces

Sizing up the crowd: Assessing spatial integration difficulties in body size judgements across eating disorder symptomatology

Introduction

Body size judgements are frequently biased, or inaccurate, and these errors are further exaggerated for individuals with eating disorders. Within the eating disorder literature, it has been suggested that exaggerated errors in body size judgements are due to difficulties with integration. Across two experiments, we developed a novel integration task, named the Ebbinghaus Illusion for Bodies in Virtual Reality (VR), to assess whether nearby bodies influence the perceived size of a single body. VR was used to simulate the appearance of a small crowd around a central target body.

Method and Results

In Experiment 1 (N = 412), participants were required to judge the size of a central female target within a crowd. Experiment 1 revealed an Ebbinghaus Illusion, in which a central female appeared larger when surrounded by small distractors, but comparatively smaller when surrounded by large distractors. In other words, the findings of Experiment 1 demonstrate that surrounding crowd information is integrated when judging an individual's body size; a novel measure of spatial integration (i.e., an Ebbinghaus Illusion for Bodies in VR). In Experiment 2 (N = 96), female participants were selected based on high (n = 43) and low (n = 53) eating disorder symptomatology. We examined whether the magnitude of this illusion would differ amongst those with elevated versus low eating disorder symptomatology, in accordance with weak central coherence theory, with the high symptomatology group displaying less spatial integration relative to the low group. The results of Experiment 2 similarly found an Ebbinghaus Illusion for Bodies in VR. However, illusion magnitude did not vary across high and low symptomatology groups.

Discussion

Overall, these findings demonstrate that surrounding crowd information is integrated when judging individual body size; however, those with elevated eating disorder symptomatology did not show any integration deficit on this broader measure of spatial integration.

EXPRESS: The Influence of Three-Dimensional Cues on Body Size Judgements

Research has shown that body size judgements are frequently biased, or inaccurate. Critically, judgement biases are further exaggerated for individuals with eating disorders, a finding that has been attributed to difficulties integrating body features into a perceptual whole. However, current understanding of which body features are integrated when judging body size is lacking. In this study, we examine whether individuals integrate three-dimensional (3D) cues to body volume when making body size judgements. Computer-generated body stimuli were presented in a 3D Virtual Reality (VR) environment. Participants (N = 412) were randomly assigned to one of two conditions: in one condition the to-be-judged body was displayed binocularly (containing 3D cues to body volume), in the other, bodies were presented monocularly (2D cues only). Across 150 trials, participants were required to make a body size judgement of a target female body from a third-person point of view using an unmarked visual analogue scale (VAS). It was found that 3D cues significantly influenced body size judgements. Namely, thin 3D bodies were judged smaller, and overweight 3D bodies were judged larger, than their 2D counterpart. Furthermore, to reconcile these effects, we present evidence that the two perceptual biases, regression to the mean and serial dependence, were reduced by the additional 3D feature information. Our findings increase our understanding of how body size is perceptually encoded and creates testable predictions for clinical populations exhibiting integration difficulties.

Does stereopsis improve face identification? A study using a virtual reality display with integrated eye-tracking and pupillometry

Stereopsis is a powerful depth cue for humans, which may also contribute to object recognition. In particular, we surmise that face identification would benefit from the availability of stereoscopic depth cues, since facial perception may be based on three-dimensional (3D) representations. In this study, a virtual reality (VR) headset with integrated eye-tracking was used to present stereoscopic images of faces. As a monoscopic contrast condition, identical images of faces were displayed to the two eyes. We monitored the participants' gaze behavior and pupil diameters while they performed a sample-to-match face identification task. We found that accuracy was superior in the stereoscopic condition compared to the monoscopic condition for frontal and intermediate views, but not profiles. Moreover, pupillary diameters were smaller when identifying stereoscopically seen faces than when viewing them without stereometric cues, which we interpret as lower processing load for the former than the latter conditions. The analysis of gaze showed that participants tended to focus on regions of the face rich in volumetric information, more so in the stereoscopic condition than the monoscopic condition. Together, these findings suggest that a 3D representation of faces may be the natural format used by the visual system when assessing face identity. Stereoscopic information, by providing depth information, assists the construction of robust facial representations in memory.

A surface-based code contributes to visual shape perception

Considerable uncertainty remains regarding the types of features human vision uses for shape representation. Visual-search experiments are reported which assessed the hypothesis of a surface-based (i.e., edge-bounded polygons) code for shape representation in human vision. The results indicate slower search rates and/or longer response times when the target shape shares its constituent surfaces with distractors (conjunction condition) than when the target surfaces are unique in the display (nonconjunction condition). This demonstration is made using test conditions that strictly control any potential artifact pertaining to target-distractor similarity. The surface-based code suggested by this surface-conjunction effect is strictly 2-D, since the effect occurs even when the surfaces are shared between the target and distractors in the 2-D image but not in their 3-D instantiation. Congruently, this latter finding is unaltered by manipulations of the richness of the depth information offered by the stimuli. It is proposed that human vision uses a 2-D surface-based code for shape representation which, considering other key findings in the field, probably coexists with an alternative representation mode based on a type of structural description that can integrate information pertaining to the 3-D aspect of shapes.

View specific generalisation effects in face recognition: Front and yaw comparison views are better than pitch

It can be difficult to recognise new instances of an unfamiliar face. Recognition errors in this particular situation appear to be viewpoint dependent with error rates increasing with the angular distance between the face views. Studies using front views for comparison have shown that recognising faces rotated in yaw can be difficult and that recognition of faces rotated in pitch is more challenging still. Here we investigate the extent to which viewpoint dependent face recognition depends on the comparison view. Participants were assigned to one of four different comparison view groups: front, ¾ yaw (right), ¾ pitch-up (above) or ¾ pitch-down (below). On each trial, participants matched their particular comparison view to a range of yaw or pitch rotated test views. Results showed that groups with a front or ¾ yaw comparison view had superior overall performance and more successful generalisation to a broader range of both pitch and yaw test views compared to groups with pitch-up or pitch-down comparison views, both of which had a very restricted generalisation range. Regression analyses revealed the importance of image similarity between views for generalisation, with a lesser role for 3D face depth. These findings are consistent with a view interpolation solution to view generalisation of face recognition, with front and ¾ yaw views being most informative.

Stereo viewing modulates three-dimensional shape processing during object recognition: A high-density ERP study

The role of stereo disparity in the recognition of 3-dimensional (3D) object shape remains an unresolved issue for theoretical models of the human visual system. We examined this issue using high-density (128 channel) recordings of event-related potentials (ERPs). A recognition memory task was used in which observers were trained to recognize a subset of complex, multipart, 3D novel objects under conditions of either (bi-) monocular or stereo viewing. In a subsequent test phase they discriminated previously trained targets from untrained distractor objects that shared either local parts, 3D spatial configuration, or neither dimension, across both previously seen and novel viewpoints. The behavioral data showed a stereo advantage for target recognition at untrained viewpoints. ERPs showed early differential amplitude modulations to shape similarity defined by local part structure and global 3D spatial configuration. This occurred initially during an N1 component around 145-190 ms poststimulus onset, and then subsequently during an N2/P3 component around 260-385 ms poststimulus onset. For mono viewing, amplitude modulation during the N1 was greatest between targets and distracters with different local parts for trained views only. For stereo viewing, amplitude modulation during the N2/P3 was greatest between targets and distracters with different global 3D spatial configurations and generalized across trained and untrained views. The results show that image classification is modulated by stereo information about the local part, and global 3D spatial configuration of object shape. The findings challenge current theoretical models that do not attribute functional significance to stereo input during the computation of 3D object shape. (PsycINFO Database Record

Effects of stereoscopic disparity on early ERP components during classification of three-dimensional objects

This study investigates the effects of stereo disparity on the perception of three-dimensional (3D) object shape. We tested the hypothesis that stereo input modulates the brain activity related to perceptual analyses of 3D shape configuration during image classification. High-density (256-channel) electroencephalogram (EEG) was used to record the temporal dynamics of visual shape processing under conditions of two-dimensional (2D) and 3D visual presentation. On each trial, observers made image classification judgements ('Same'/'Different') to two briefly presented, multi-part, novel objects. On different-object trials, stimuli could either share volumetric parts but not the global 3D shape configuration and have different parts but the same global 3D shape configuration or differ on both aspects. Analyses using mass univariate contrasts showed that the earliest sensitivity to 2D versus 3D viewing appeared as a negative deflection over posterior locations on the N1 component between 160 and 220 ms post-stimulus onset. Subsequently, event-related potential (ERP) modulations during the N2 time window between 240 and 370 ms were linked to image classification. N2 activity reflected two distinct components - an early N2 (240-290 ms) and a late N2 (290-370 ms) - that showed different patterns of responses to 2D and 3D input and differential sensitivity to 3D object structure. The results revealed that stereo input modulates the neural correlates of 3D object shape. We suggest that this reflects differential perceptual processing of object shape under conditions of stereo or mono input. These findings challenge current theories that attribute no functional role for stereo input during 3D shape perception.

Taking the Perfect Selfie: Investigating the Impact of Perspective on the Perception of Higher Cognitive Variables

Taking selfies is now becoming a standard human habit. However, as a social phenomenon, research is still in the fledgling stage and the scientific framework is sparse. Selfies allow us to share social information with others in a compact format. Furthermore, we are able to control important photographic and compositional aspects, such as perspective, which have a strong impact on the assessment of a face (e.g., demonstrated by the height-weight illusion, effects of gaze direction, faceism-index). In Study 1, we focused on the impact of perspective (left/right hemiface, above/below vs. frontal presentation) on higher cognitive variables and let 172 participants rate the perceived attractiveness, helpfulness, sympathy, dominance, distinctiveness, and intelligence, plus important information on health issues (e.g., body weight), on the basis of 14 3D faces. We could show that lateral snapshots yielded higher ratings for attractiveness compared to the classical frontal view. However, this effect was more pronounced for left hemifaces and especially female faces. Compared to the frontal condition, 30° right hemifaces were rated as more helpful, but only for female faces while faces viewed from above were perceived as significant less helpful. Direct comparison between left vs. right hemifaces revealed no effect. Relating to sympathy, we only found a significant effect for 30° right male hemifaces, but only in comparison to the frontal condition. Furthermore, female 30° right hemifaces were perceived as more intelligent. Relating to body weight, we replicated the so-called “height-weight illusion.” Other variables remained unaffected. In Study 2, we investigated the impact of a typical selfie-style condition by presenting the respective faces from a lateral (left/right) and tilted (lower/higher) vantage point. Most importantly, depending on what persons wish to express with a selfie, a systematic change of perspective can strongly optimize their message; e.g., increasing their attractiveness by shooting from above left, and in contrast, decreasing their expressed helpfulness by shooting from below. We could further extent past findings relating to the height-weight illusion and showed that an additional rotation of the camera positively affected the perception of body weight (lower body weight). We discuss potential explanations for perspective-related effects, especially gender-related ones.

Use of 3D faces facilitates facial expression recognition in children

This study assessed whether presenting 3D face stimuli could facilitate children’s facial expression recognition. Seventy-one children aged between 3 and 6 participated in the study. Their task was to judge whether a face presented in each trial showed a happy or fearful expression. Half of the face stimuli were shown with 3D representations, whereas the other half of the images were shown as 2D pictures. We compared expression recognition under these conditions. The results showed that the use of 3D faces improved the speed of facial expression recognition in both boys and girls. Moreover, 3D faces improved boys’ recognition accuracy for fearful expressions. Since fear is the most difficult facial expression for children to recognize, the facilitation effect of 3D faces has important practical implications for children with difficulties in facial expression recognition. The potential benefits of 3D representation for other expressions also have implications for developing more realistic assessments of children’s expression recognition.

Stereoscopy Amplifies Emotions Elicited by Facial Expressions

Mediated facial expressions do not elicit emotions as strongly as real-life facial expressions, possibly due to the low fidelity of pictorial presentations in typical mediation technologies. In the present study, we investigated the extent to which stereoscopy amplifies emotions elicited by images of neutral, angry, and happy facial expressions. The emotional self-reports of positive and negative valence (which were evaluated separately) and arousal of 40 participants were recorded. The magnitude of perceived depth in the stereoscopic images was manipulated by varying the camera base at 15, 40, 65, 90, and 115 mm. The analyses controlled for participants' gender, gender match, emotional empathy, and trait alexithymia. The results indicated that stereoscopy significantly amplified the negative valence and arousal elicited by angry expressions at the most natural (65 mm) camera base, whereas stereoscopy amplified the positive valence elicited by happy expressions in both the narrowed and most natural (15-65 mm) base conditions. Overall, the results indicate that stereoscopy amplifies the emotions elicited by mediated emotional facial expressions when the depth geometry is close to natural. The findings highlight the sensitivity of the visual system to depth and its effect on emotions.

If you watch it move, you'll recognize it in 3D: Transfer of depth cues between encoding and retrieval

Viewing objects with stereoscopic displays provides additional depth cues through binocular disparity supporting object recognition. So far, it was unknown whether this results from the representation of specific stereoscopic information in memory or a more general representation of an object's depth structure. Therefore, we investigated whether continuous object rotation acting as depth cue during encoding results in a memory representation that can subsequently be accessed by stereoscopic information during retrieval. In Experiment 1, we found such transfer effects from continuous object rotation during encoding to stereoscopic presentations during retrieval. In Experiments 2a and 2b, we found that the continuity of object rotation is important because only continuous rotation and/or stereoscopic depth but not multiple static snapshots presented without stereoscopic information caused the extraction of an object's depth structure into memory. We conclude that an object's depth structure and not specific depth cues are represented in memory.

Functional architecture for disparity in macaque inferior temporal cortex and its relationship to the architecture for faces, color, scenes, and visual field

Binocular disparity is a powerful depth cue for object perception. The computations for object vision culminate in inferior temporal cortex (IT), but the functional organization for disparity in IT is unknown. Here we addressed this question by measuring fMRI responses in alert monkeys to stimuli that appeared in front of (near), behind (far), or at the fixation plane. We discovered three regions that showed preferential responses for near and far stimuli, relative to zero-disparity stimuli at the fixation plane. These "near/far" disparity-biased regions were located within dorsal IT, as predicted by microelectrode studies, and on the posterior inferotemporal gyrus. In a second analysis, we instead compared responses to near stimuli with responses to far stimuli and discovered a separate network of "near" disparity-biased regions that extended along the crest of the superior temporal sulcus. We also measured in the same animals fMRI responses to faces, scenes, color, and checkerboard annuli at different visual field eccentricities. Disparity-biased regions defined in either analysis did not show a color bias, suggesting that disparity and color contribute to different computations within IT. Scene-biased regions responded preferentially to near and far stimuli (compared with stimuli without disparity) and had a peripheral visual field bias, whereas face patches had a marked near bias and a central visual field bias. These results support the idea that IT is organized by a coarse eccentricity map, and show that disparity likely contributes to computations associated with both central (face processing) and peripheral (scene processing) visual field biases, but likely does not contribute much to computations within IT that are implicated in processing color.

Stereoscopic Offset Makes Objects Easier to Recognize

Binocular vision is obviously useful for depth perception, but it might also enhance other components of visual processing, such as image segmentation. We used naturalistic images to determine whether giving an object a stereoscopic offset of 15-120 arcmin of crossed disparity relative to its background would make the object easier to recognize in briefly presented (33-133 ms), temporally masked displays. Disparity had a beneficial effect across a wide range of disparities and display durations. Most of this benefit occurred whether or not the stereoscopic contour agreed with the object’s luminance contour. We attribute this benefit to an orienting of spatial attention that selected the object and its local background for enhanced 2D pattern processing. At longer display durations, contour agreement provided an additional benefit, and a separate experiment using random-dot stimuli confirmed that stereoscopic contours plausibly contributed to recognition at the longer display durations in our experiment. We conclude that in real-world situations binocular vision confers an advantage not only for depth perception, but also for recognizing objects from their luminance patterns and bounding contours.

Stereo disparity facilitates view generalization during shape recognition for solid multipart objects

Current theories of object recognition in human vision make different predictions about whether the recognition of complex, multipart objects should be influenced by shape information about surface depth orientation and curvature derived from stereo disparity. We examined this issue in five experiments using a recognition memory paradigm in which observers (N = 134) memorized and then discriminated sets of 3D novel objects at trained and untrained viewpoints under either mono or stereo viewing conditions. In order to explore the conditions under which stereo-defined shape information contributes to object recognition we systematically varied the difficulty of view generalization by increasing the angular disparity between trained and untrained views. In one series of experiments, objects were presented from either previously trained views or untrained views rotated (15°, 30°, or 60°) along the same plane. In separate experiments we examined whether view generalization effects interacted with the vertical or horizontal plane of object rotation across 40° viewpoint changes. The results showed robust viewpoint-dependent performance costs: Observers were more efficient in recognizing learned objects from trained than from untrained views, and recognition was worse for extrapolated than for interpolated untrained views. We also found that performance was enhanced by stereo viewing but only at larger angular disparities between trained and untrained views. These findings show that object recognition is not based solely on 2D image information but that it can be facilitated by shape information derived from stereo disparity.

Predictive codes of familiarity and context during the perceptual learning of facial identities

Face recognition is a key component of successful social behaviour. However, the computational processes that underpin perceptual learning and recognition as faces transition from unfamiliar to familiar are poorly understood. In predictive coding, learning occurs through prediction errors that update stimulus familiarity, but recognition is a function of both stimulus and contextual familiarity. Here we show that behavioural responses on a two-option face recognition task can be predicted by the level of contextual and facial familiarity in a computational model derived from predictive-coding principles. Using fMRI, we show that activity in the superior temporal sulcus varies with the contextual familiarity in the model, whereas activity in the fusiform face area covaries with the prediction error parameter that updated facial familiarity. Our results characterize the key computations underpinning the perceptual learning of faces, highlighting that the functional properties of face-processing areas conform to the principles of predictive coding.

The role of binocular disparity in rapid scene and pattern recognition

We investigated the contribution of binocular disparity to the rapid recognition of scenes and simpler spatial patterns using a paradigm combining backward masked stimulus presentation and short-term match-to-sample recognition. First, we showed that binocular disparity did not contribute significantly to the recognition of briefly presented natural and artificial scenes, even when the availability of monocular cues was reduced. Subsequently, using dense random dot stereograms as stimuli, we showed that observers were in principle able to extract spatial patterns defined only by disparity under brief, masked presentations. Comparing our results with the predictions from a cue-summation model, we showed that combining disparity with luminance did not per se disrupt the processing of disparity. Our results suggest that the rapid recognition of scenes is mediated mostly by a monocular comparison of the images, although we can rely on stereo in fast pattern recognition.

On the contribution of binocular disparity to the long-term memory for natural scenes

Binocular disparity is a fundamental dimension defining the input we receive from the visual world, along with luminance and chromaticity. In a memory task involving images of natural scenes we investigate whether binocular disparity enhances long-term visual memory. We found that forest images studied in the presence of disparity for relatively long times (7s) were remembered better as compared to 2D presentation. This enhancement was not evident for other categories of pictures, such as images containing cars and houses, which are mostly identified by the presence of distinctive artifacts rather than by their spatial layout. Evidence from a further experiment indicates that observers do not retain a trace of stereo presentation in long-term memory.

Visual adaptation and face perception

The appearance of faces can be strongly affected by the characteristics of faces viewed previously. These perceptual after-effects reflect processes of sensory adaptation that are found throughout the visual system, but which have been considered only relatively recently in the context of higher level perceptual judgements. In this review, we explore the consequences of adaptation for human face perception, and the implications of adaptation for understanding the neural-coding schemes underlying the visual representation of faces. The properties of face after-effects suggest that they, in part, reflect response changes at high and possibly face-specific levels of visual processing. Yet, the form of the after-effects and the norm-based codes that they point to show many parallels with the adaptations and functional organization that are thought to underlie the encoding of perceptual attributes like colour. The nature and basis for human colour vision have been studied extensively, and we draw on ideas and principles that have been developed to account for norms and normalization in colour vision to consider potential similarities and differences in the representation and adaptation of faces.

A stereo disadvantage for recognizing rotated familiar objects

We tested recognition of familiar objects in two different conditions: mono, where stimuli were displayed as flat, 2-D images, and stereo, where objects were displayed with stereoscopic depth information. In three experiments, participants performed a sequential matching task, where an object was rotated by up to 180 degrees between presentations. When the 180 degrees rotation resulted in large changes in depth for object components, recognition performance in the mono condition showed better performance at 180 degrees rotations than at smaller rotations, but stereo presentations showed a monotonic increase in response time with rotation. However, 180 degrees rotations that did not result in much depth variation showed similar patterns of results for mono and stereo conditions. These results suggest that in some circumstances, the lack of explicit 3-D information in 2-D images may influence the recognition of familiar objects when they are depicted on flat computer monitors.

Radial frequency adaptation suggests polar-based coding of local shape cues

The study of shape processing in the human visual system has frequently employed radial frequency (RF) patterns as conveniently manipulable stimuli. This study uses an adaptation paradigm to investigate how local shape information is sampled in the processing of RF contour shapes. Experiment 1 measured thresholds for detecting a fixed mean radius RF contour following adaptation to RF patterns which, in separate conditions, varied in mean radius and radial frequency. Results reveal that, adaptation is strongly tuned for RF over a range of pattern radii, but is not tuned for the number of cycles of radial modulation per visual degree of contour length; a characteristic that changes with both radius and radial frequency. Experiment 2 manipulated the polar angle separation on the fronto-parallel plane between curvature features on a fixed RF by foreshortening the pattern appearance (consistent with a rotation in depth) and shows that RF shape processing is tuned for fronto-parallel separation angles between curvature features. Results were near identical when a stereo rotation cue was added to the perspective modified RF. In the second part of Experiment 2 we showed that RF shape adaptation is also tuned for the polar angular extent of the curvature represented by the lobe at that angle. Collectively, our results indicate that the polar angle at which local curvature features appear, in addition to the angular extent of the curvature feature at that location, are both critical parameters for coding specific RF shapes.