The Tipping Point of Animacy

Unconscious discrimination of social cues from eye whites in infants

Human eyes serve two key functions in face-to-face social interactions: they provide cues about a person's emotional state and attentional focus (gaze direction). Both functions critically rely on the morphologically unique human sclera and have been shown to operate even in the absence of conscious awareness in adults. However, it is not known whether the ability to respond to social cues from scleral information without conscious awareness exists early in human ontogeny and can therefore be considered a foundational feature of human social functioning. In the current study, we used event-related brain potentials (ERPs) to show that 7-mo-old infants discriminate between fearful and nonfearful eyes (experiment 1) and between direct and averted gaze (experiment 2), even when presented below the perceptual threshold. These effects were specific to the human sclera and not seen in response to polarity-inverted eyes. Our results suggest that early in ontogeny the human brain detects social cues from scleral information even in the absence of conscious awareness. The current findings support the view that the human eye with its prominent sclera serves critical communicative functions during human social interactions.

Face animacy is not all in the eyes: evidence from contrast chimeras

Observers are capable of distinguishing real faces from artificial faces of various types (eg dolls, computer-generated faces) relatively easily. While a number of diagnostic cues are potentially available to observers to accomplish this task, the appearance of the eyes has been shown to be critically important. However, eye appearance appears to interact with other cues, like the appearance of the skin, in some settings. The 'uncanny' appearance of some artificial faces appears to result from multiple visual features and their departure from typical face norms, for example, and recent results investigating how real and artificial features are perceived in chimeric faces also suggest that observers use multiple cues to measure face animacy. Presently, we examined the cues that support real-artificial face discrimination by using contrast negation and so-called 'contrast chimeras' to selectively disrupt the appearance of the eyes and the remainder of the face pattern. First, we demonstrate that, like other aspects of face perception, perceived animacy is significantly impacted by contrast negation. Second, by selectively manipulating the contrast of the eyes relative to the rest of the face, we demonstrate that these face regions are of approximately equal use to observers for animacy discrimination.

Social connection modulates perceptions of animacy

Human survival depends on identifying targets potentially capable of engaging in meaningful social connection. Using sets of morphed images created from animate (human) and inanimate (doll) faces, we found converging evidence across two studies showing that the motivation to connect with other people systematically alters the interpretation of the physical features that signal that a face is alive. Specifically, in their efforts to find and connect with other social agents, individuals who feel socially disconnected actually decrease their thresholds for what it means to be alive, consistently observing animacy when fewer definitively human cues are present. From an evolutionary perspective, overattributing animacy may be an adaptive strategy that allows people to cast a wide net when identifying possible sources of social connection and maximize their opportunities to renew social relationships.

Pupil dilation dynamics track attention to high-level information

It has long been thought that the eyes index the inner workings of the mind. Consistent with this intuition, empirical research has demonstrated that pupils dilate as a consequence of attentional effort. Recently, Smallwood et al. (2011) demonstrated that pupil dilations not only provide an index of overall attentional effort, but are time-locked to stimulus changes during attention (but not during mind-wandering). This finding suggests that pupil dilations afford a dynamic readout of conscious information processing. However, because stimulus onsets in their study involved shifts in luminance as well as information, they could not determine whether this coupling of stimulus and pupillary dynamics reflected attention to low-level (luminance) or high-level (information) changes. Here, we replicated the methodology and findings of Smallwood et al. (2011) while controlling for luminance changes. When presented with isoluminant digit sequences, participants' pupillary dilations were synchronized with stimulus onsets when attending, but not when mind-wandering. This replicates Smallwood et al. (2011) and clarifies their finding by demonstrating that stimulus-pupil coupling reflects online cognitive processing beyond sensory gain.

Perceived animacy influences the processing of human-like surface features in the fusiform gyrus

While decades of research have demonstrated that a region of the right fusiform gyrus (FG) responds selectively to faces, a second line of research suggests that the FG responds to a range of animacy cues, including biological motion and goal-directed actions, even in the absence of faces or other human-like surface features. These findings raise the question of whether the FG is indeed sensitive to faces or to the more abstract category of animate agents. The current study uses fMRI to examine whether the FG responds to all faces in a category-specific way or whether the FG is especially sensitive to the faces of animate agents. Animate agents are defined here as intentional agents with the capacity for rational goal-directed actions. Specifically, we examine how the FG responds to an entity that looks like an animate agent but that lacks the capacity for goal-directed rational action. Region-of-interest analyses reveal that the FG activates more strongly to the animate compared with the inanimate entity, even though the surface features of both animate and inanimate entities were identical. These results suggest that the FG does not respond to all faces in a category-specific way, and is instead especially sensitive to whether an entity is animate.

Early visual ERP sensitivity to the species and animacy of faces

Assessing the agency of potential actors in the visual world is a critically important aspect of social cognition. Adult observers are generally capable of distinguishing real faces from artificial faces (even allowing for recent advances in graphics technology and motion capture); even small deviations from real facial appearance can lead to profound effects on face recognition. Presently, we examined how early components of visual event-related potentials (ERPs) are affected by the "life" in human faces and animal faces. We presented participants with real and artificial faces of humans and dogs, and analyzed the response properties of the P100 and the N170 as a function of stimulus appearance and task (species categorization vs. animacy categorization). The P100 exhibited sensitivity to face species and animacy. We found that the N170's differential responses to human faces vs. dog faces depended on the task participants' performed. Also, the effect of species was only evident for real faces of humans and dogs, failing to obtain with artificial faces. These results suggest that face animacy does modulate early components of visual ERPs-the N170 is not merely a crude face detector, but reflects the tuning of the visual system to natural face appearance.

Perceptual and category processing of the Uncanny Valley hypothesis' dimension of human likeness: some methodological issues

Mori's Uncanny Valley Hypothesis^1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated characters) can evoke negative or positive affect (valence) depending on the object's degree of visual and behavioral realism along a dimension of human likeness (DHL) (Figure 1). But studies of affective valence of subjective responses to variously realistic non-human characters have produced inconsistent findings ^{3, 4, 5, 6}. One of a number of reasons for this is that human likeness is not perceived as the hypothesis assumes. While the DHL can be defined following Mori's description as a smooth linear change in the degree of physical humanlike similarity, subjective perception of objects along the DHL can be understood in terms of the psychological effects of categorical perception (CP) ⁷. Further behavioral and neuroimaging investigations of category processing and CP along the DHL and of the potential influence of the dimension's underlying category structure on affective experience are needed. This protocol therefore focuses on the DHL and allows examination of CP. Based on the protocol presented in the video as an example, issues surrounding the methodology in the protocol and the use in "uncanny" research of stimuli drawn from morph continua to represent the DHL are discussed in the article that accompanies the video. The use of neuroimaging and morph stimuli to represent the DHL in order to disentangle brain regions neurally responsive to physical human-like similarity from those responsive to category change and category processing is briefly illustrated.

Category Processing and the human likeness dimension of the Uncanny Valley Hypothesis: Eye-Tracking Data

The Uncanny Valley Hypothesis (Mori, 1970) predicts that perceptual difficulty distinguishing between a humanlike object (e.g., lifelike prosthetic hand, mannequin) and its human counterpart evokes negative affect. Research has focused on affect, with inconsistent results, but little is known about how objects along the hypothesis' dimension of human likeness (DHL) are actually perceived. This study used morph continua based on human and highly realistic computer-generated (avatar) faces to represent the DHL. Total number and dwell time of fixations to facial features were recorded while participants (N = 60) judged avatar versus human category membership of the faces in a forced choice categorization task. Fixation and dwell data confirmed the face feature hierarchy (eyes, nose, and mouth in this order of importance) across the DHL. There were no further findings for fixation. A change in the relative importance of these features was found for dwell time, with greater preferential processing of eyes and mouth of categorically ambiguous faces compared with unambiguous avatar faces. There were no significant differences between ambiguous and human faces. These findings applied for men and women, though women generally dwelled more on the eyes to the disadvantage of the nose. The mouth was unaffected by gender. In summary, the relative importance of facial features changed on the DHL's non-human side as a function of categorization ambiguity. This change was indicated by dwell time only, suggesting greater depth of perceptual processing of the eyes and mouth of ambiguous faces compared with these features in unambiguous avatar faces.

Biological Sex Determines Whether Faces Look Real

Judging whether a face is real or artificial can be done relatively rapidly and accurately, even when visual information is substantially impoverished. The perception of animacy in the face also has several interesting properties that may reflect both the underlying "tuning" of face space to preferentially represent real face appearance and the diagnosticity of individual features for categorizing faces as animate or inanimate. In the current study, we examined how sex categories interact with animacy perception by separately characterizing animacy judgments as a function of stimulus sex. We find that stimulus sex affects subjective ratings of animacy and sex categorization of real and artificial faces. Specifically, female faces look more artificial and artificial faces look more female. We discuss our results in terms of the ecology of real and artificial faces and the possible role of visual experience with artificial female faces, and the objectification of female faces.

You can take the eyes out of the doll, but

The perceived animacy of a face is well-predicted by the perceived animacy of the eyes presented in isolation. This is not true for other facial features (eg having a highly life-like nose does not appear to be crucial), suggesting that the eyes are a critical feature for perceiving life in a face. Here, we asked whether it was therefore possible to 'transplant' animacy into a face by transplanting the eyes into a face image. We conducted digital eye surgery on a series of morphed human/doll faces and found that while doll eyes make a morphed face look less alive, human eyes do not make you look more so. Thus, we cannot so easily transplant animacy into a face, but we can take it away.

Multivoxel patterns in face-sensitive temporal regions reveal an encoding schema based on detecting life in a face

More than a decade of research has demonstrated that faces evoke prioritized processing in a 'core face network' of three brain regions. However, whether these regions prioritize the detection of global facial form (shared by humans and mannequins) or the detection of life in a face has remained unclear. Here, we dissociate form-based and animacy-based encoding of faces by using animate and inanimate faces with human form (humans, mannequins) and dog form (real dogs, toy dogs). We used multivariate pattern analysis of BOLD responses to uncover the representational similarity space for each area in the core face network. Here, we show that only responses in the inferior occipital gyrus are organized by global facial form alone (human vs dog) while animacy becomes an additional organizational priority in later face-processing regions: the lateral fusiform gyri (latFG) and right superior temporal sulcus. Additionally, patterns evoked by human faces were maximally distinct from all other face categories in the latFG and parts of the extended face perception system. These results suggest that once a face configuration is perceived, faces are further scrutinized for whether the face is alive and worthy of social cognitive resources.

The group-member mind trade-off: attributing mind to groups versus group members

People attribute minds to other individuals and make inferences about those individuals' mental states to explain and predict their behavior. Little is known, however, about whether people also attribute minds to groups and believe that collectives, companies, and corporations can think, have intentions, and make plans. Even less is known about the consequences of these attributions for both groups and group members. We investigated the attribution of mind and responsibility to groups and group members, and we demonstrated that people make a trade-off: The more a group is attributed a group mind, the less members of that group are attributed individual minds. Groups that are judged to have more group mind are also judged to be more cohesive and responsible for their collective actions. These findings have important implications for how people perceive the minds of groups and group members, and for how attributions of mind influence attributions of responsibility to groups and group members.

Infant monkeys' concept of animacy: the role of eyes and fluffiness

Both human and nonhuman primates have been suggested to possess some essential knowledge about animate entities, but it remains unclear whether the concept of animacy is shared across species, which properties are used as an "animacy marker," and whether such ability is present at birth. We investigated infant Japanese monkeys' looking responses towards novel objects varying in both physical appearance and self-propelled motion, with the aim of depicting the role of eyes and fluffiness in the early recognition of animacy. Presented with an inanimate natural stone, three-month-old monkeys showed longer looking times at the stone's self-propelled motion than at its baseline still posture. This effect became significantly smaller when artificial fur was attached to the stone, while adding artificial eyes did not elicit a departing pattern in their looking behavior. In contrast, one-month-old monkeys showed no systematic differences in their looking behavior. This suggests that the concept of animacy in terms of self-propelledness may develop between one and three months of age, with sensitivity to texture emerging by three months. Development of biological knowledge is discussed in relation to social knowledge from both ontogenetic and phylogenetic perspectives.

Mind perception: real but not artificial faces sustain neural activity beyond the N170/VPP

Faces are visual objects that hold special significance as the icons of other minds. Previous researchers using event-related potentials (ERPs) have found that faces are uniquely associated with an increased N170/vertex positive potential (VPP) and a more sustained frontal positivity. Here, we examined the processing of faces as objects vs. faces as cues to minds by contrasting images of faces possessing minds (human faces), faces lacking minds (doll faces), and non-face objects (i.e., clocks). Although both doll and human faces were associated with an increased N170/VPP from 175–200 ms following stimulus onset, only human faces were associated with a sustained positivity beyond 400 ms. Our data suggest that the N170/VPP reflects the object-based processing of faces, whether of dolls or humans; on the other hand, the later positivity appears to uniquely index the processing of human faces—which are more salient and convey information about identity and the presence of other minds.