The contribution of the body and motion to whole person recognition

Identity recognition from faces and bodies in schizophrenia spectrum disorders

Deficits in facial identity recognition and its association with poor social functioning are well documented in schizophrenia, but none of these studies have assessed the role of the body in these processes. Recent research in healthy populations shows that the body is also an important source of information in identity recognition, and the current study aimed to thoroughly examine identity recognition from both faces and bodies in schizophrenia. Sixty-five individuals with schizophrenia and forty-nine healthy controls completed three conditions of an identity matching task in which they attempted to match unidentified persons in unedited photos of faces and bodies, edited photos showing faces only, or edited photos showing bodies only. Results revealed global deficits in identity recognition in individuals with schizophrenia (ηp2 = 0.068), but both groups showed better recognition from bodies alone as compared to faces alone (ηp2 = 0.573), suggesting that the ability to extract useful information from bodies when identifying persons may remain partially preserved in schizophrenia. Further research is necessary to understand the relationship between face/body processing, identity recognition, and functional outcomes in individuals with schizophrenia-spectrum disorders.

Separated and overlapping neural coding of face and body identity

Recognising a person's identity often relies on face and body information, and is tolerant to changes in low‐level visual input (e.g., viewpoint changes). Previous studies have suggested that face identity is disentangled from low‐level visual input in the anterior face‐responsive regions. It remains unclear which regions disentangle body identity from variations in viewpoint, and whether face and body identity are encoded separately or combined into a coherent person identity representation. We trained participants to recognise three identities, and then recorded their brain activity using fMRI while they viewed face and body images of these three identities from different viewpoints. Participants' task was to respond to either the stimulus identity or viewpoint. We found consistent decoding of body identity across viewpoint in the fusiform body area, right anterior temporal cortex, middle frontal gyrus and right insula. This finding demonstrates a similar function of fusiform and anterior temporal cortex for bodies as has previously been shown for faces, suggesting these regions may play a general role in extracting high‐level identity information. Moreover, we could decode identity across fMRI activity evoked by faces and bodies in the early visual cortex, right inferior occipital cortex, right parahippocampal cortex and right superior parietal cortex, revealing a distributed network that encodes person identity abstractly. Lastly, identity decoding was consistently better when participants attended to identity, indicating that attention to identity enhances its neural representation. These results offer new insights into how the brain develops an abstract neural coding of person identity, shared by faces and bodies.

Independent contributions of the face, body, and gait to the representation of the whole person

Most studies on person perception have primarily investigated static images of faces. However, real-life person perception also involves the body and often the gait of the whole person. Whereas some studies indicated that the face dominates the representation of the whole person, others have emphasized the additional contribution of the body and gait. Here, we compared models of whole-person perception by asking whether a model that includes the body for static whole-person stimuli and also the gait for dynamic whole-person stimuli accounts better for the representation of the whole person than a model that takes into account the face alone. Participants rated the distinctiveness of static or dynamic displays of different people based on either the whole person, face, body, or gait. By fitting a linear regression model to the representation of the whole person based on the face, body, and gait, we revealed that the face and body contribute uniquely and independently to the representation of the static whole person, and that gait further contributes to the representation of the dynamic person. A complementary analysis examined whether these components are also valid dimensions of a whole-person representational space. This analysis further confirmed that the body in addition to the face as well as the gait are valid dimensions of the static and dynamic whole-person representations, respectively. These data clearly show that whole-person perception goes beyond the face and is significantly influenced by the body and gait.

Dissociating gait from static appearance: A virtual reality study of the role of dynamic identity signatures in person recognition

Studies on person recognition have primarily examined recognition of static faces, presented on a computer screen at a close distance. Nevertheless, in naturalistic situations we typically see the whole dynamic person, often approaching from a distance. In such cases, facial information may be less clear, and the motion pattern of an individual, their dynamic identity signature (DIS), may be used for person recognition. Studies that examined the role of motion in person recognition, presented videos of people in motion. However, such stimuli do not allow for the dissociation of gait from face and body form, as different identities differ both in their gait and static appearance. To examine the contribution of gait in person recognition, independently from static appearance, we used a virtual environment, and presented across participants, the same face and body form with different gaits. The virtual environment also enabled us to assess the distance at which a person is recognized as a continuous variable. Using this setting, we assessed the accuracy and distance at which identities are recognized based on their gait, as a function of gait distinctiveness. We find that the accuracy and distance at which people were recognized increased with gait distinctiveness. Importantly, these effects were found when recognizing identities in motion but not from static displays, indicating that DIS rather than attention, enabled more accurate person recognition. Overall these findings highlight that gait contributes to person recognition beyond the face and body and stress an important role for gait in real-life person recognition.

Body Language Influences on Facial Identification at Passport Control: An Exploration in Virtual Reality

Person identification at airports requires the matching of a passport photograph to its bearer. One aim of this process is to find identity impostors, who use valid identity documents of similar-looking people to avoid detection. In psychology, this process has been studied extensively with static pairs of face photographs that require identity match (same person shown) versus mismatch (two different people) decisions. However, this approach provides a limited proxy for studying how other factors, such as nonverbal behaviour, affect this task. The current study investigated the influence of body language on facial identity matching within a virtual reality airport environment, by manipulating activity levels of person avatars queueing at passport control. In a series of six experiments, detection of identity mismatches was unaffected when observers were not instructed to utilise body language. By contrast, under explicit instruction to look out for unusual body language, these cues enhanced detection of mismatches but also increased false classification of matches. This effect was driven by increased activity levels rather than body language that simply differed from the behaviour of the majority of passengers. The implications and limitations of these findings are discussed.

Can we recognize people based on their body-alone? The roles of body motion and whole person context

While most studies on person recognition examine the face alone, recent studies have shown evidence for the contribution of the body and gait to person recognition beyond the face. Nevertheless, little is known on whether person recognition can be performed based on the body alone. In this study, we examined two sources of information that may enhance body-based person recognition: body motion and whole person context. Body motion has been shown to contribute to person recognition especially when facial information is unclear. Additionally, generating whole person context, by attaching faceless heads to bodies, has been shown to activate face processing mechanisms and may therefore enhance body-based person recognition. To assess body-based person recognition, participants performed a sequential matching task in which they studied a video of a person walking followed by a headless image of the same or different identity. The role of body motion was examined by comparing recognition from dynamic vs. static headless bodies. The role of whole person context was examined by comparing bodies with and without faceless heads. Our findings show that person recognition from the body alone was better in dynamic vs. static displays indicating that body motion contributed to body-based person recognition. In addition, whole person context contributed to body-based person recognition when recognition was performed in static displays. Overall these findings show that recognizing people based on their body alone is challenging but can be performed under certain circumstances that enhance the processing of the body when seeing the whole dynamic person.

Integrating faces and bodies: Psychological and neural perspectives on whole person perception

The human "person" is a common percept we encounter. Research on person perception has been focused either on face or body perception-with less attention paid to whole person perception. We review psychological and neuroscience studies aimed at understanding how face and body processing operate in concert to support intact person perception. We address this question considering: a.) the task to be accomplished (identification, emotion processing, detection), b.) the neural stage of processing (early/late visual mechanisms), and c.) the relevant brain regions for face/body/person processing. From the psychological perspective, we conclude that the integration of faces and bodies is mediated by the goal of the processing (e.g., emotion analysis, identification, etc.). From the neural perspective, we propose a hierarchical functional neural architecture of face-body integration that retains a degree of separation between the dorsal and ventral visual streams. We argue for two centers of integration: a ventral semantic integration hub that is the result of progressive, posterior-to-anterior, face-body integration; and a social agent integration hub in the dorsal stream STS.

Interacting hands draw attention during scene observation

In this study I examined the role of the hands in scene perception. In Experiment 1, eye movements during free observation of natural scenes were analyzed. Fixations to faces and hands were compared under several conditions, including scenes with and without faces, with and without hands, and without a person. The hands were either resting (e.g., lying on the knees) or interacting with objects (e.g., holding a bottle). Faces held an absolute attentional advantage, regardless of hand presence. Importantly, fixations to interacting hands were faster and more frequent than those to resting hands, suggesting attentional priority to interacting hands. The interacting-hand advantage could not be attributed to perceptual saliency or to the hand-owner (i.e., the depicted person) gaze being directed at the interacting hand. Experiment 2 confirmed the interacting-hand advantage in a visual search paradigm with more controlled stimuli. The present results indicate that the key to understanding the role of attention in person perception is the competitive interaction among objects such as faces, hands, and objects interacting with the person.

The Acquisition of Person Knowledge

How do we learn what we know about others? Answering this question requires understanding the perceptual mechanisms with which we recognize individuals and their actions, and the processes by which the resulting perceptual representations lead to inferences about people's mental states and traits. This review discusses recent behavioral, neural, and computational studies that have contributed to this broad research program, encompassing both social perception and social cognition.

Face recognition ability does not predict person identification performance: using individual data in the interpretation of group results

There are large individual differences in people's face recognition ability. These individual differences provide an opportunity to recruit the best face-recognisers into jobs that require accurate person identification, through the implementation of ability-screening tasks. To date, screening has focused exclusively on face recognition ability; however real-world identifications can involve the use of other person-recognition cues. Here we incorporate body and biological motion recognition as relevant skills for person identification. We test whether performance on a standardised face-matching task (the Glasgow Face Matching Test) predicts performance on three other identity-matching tasks, based on faces, bodies, and biological motion. We examine the results from group versus individual analyses. We found stark differences between the conclusions one would make from group analyses versus analyses that retain information about individual differences. Specifically, tests of correlation and analysis of variance suggested that face recognition ability was related to performance for all person identification tasks. These analyses were strikingly inconsistent with the individual differences data, which suggested that the screening task was related only to performance on the face task. This study highlights the importance of individual data in the interpretation of results of person identification ability.

It's Not All About the Face: Variability Reveals Asymmetric Obligatory Processing of Faces and Bodies in Whole-Body Contexts

Researchers have sought to understand the specialized processing of faces and bodies in isolation, but recently they have considered how face and body information interact within the context of the whole body. Although studies suggest that face and body information can be integrated, it remains an open question whether this integration is obligatory and whether contributions of face and body information are symmetrical. In a selective attention task with whole-body stimuli, we focused attention on either the face or body and tested whether variation in the irrelevant part could be ignored. We manipulated orientation to determine the extent to which inversion disrupted obligatory face and body processing. Obligatory processing was evidenced as performance changes in discrimination that depended on stimulus orientation when the irrelevant region varied. For upright but not inverted face discrimination, participants could not ignore body posture variation, even when it was not diagnostic to the task. However, participants could ignore face variation for upright body posture discrimination but not for inverted posture discrimination. The extent to which face and body information necessarily influence each other in whole-body contexts appears to depend on both domain-general attentional and face- or body-specific holistic processing mechanisms.

Face recognition of full-bodied avatars by active observers in a virtual environment

Viewing faces in motion or attached to a body instead of isolated static faces improves their subsequent recognition. Here we enhanced the ecological validity of face encoding by having observers physically moving in a virtual room populated by life-size avatars. We compared the recognition performance of this active group to two control groups. The first control group watched a passive reenactment of the visual experience of the active group. The second control group saw static screenshots of the avatars. All groups performed the same old/new recognition task after learning. Half of the learned faces were shown at test in an orientation close to that experienced during learning while the others were viewed from a new viewing angle. All observers found novel views more difficult to recognize than familiar ones. Overall, the active group performed better than both other groups. Furthermore, the group learning faces from static images was the only one to be at chance level in the novel-view condition. These findings suggest that active exploration combined with a dynamic experience of the faces to learn allow for more robust face recognition and point out the value of such techniques for integrating facial visual information and enhancing recognition from novel viewpoints.