Monkey visual behavior falls into the uncanny valley

The robot eyes don't have it. The presence of eyes on collaborative robots yields marginally higher user trust but lower performance

Eye gaze is a prominent feature of human social lives, but little is known on whether fitting eyes on machines makes humans trust them more. In this study we compared subjective and objective markers of human trust when collaborating with eyed and non-eyed robots of the same type. We used virtual reality scenes in which we manipulated distance and the presence of eyes on a robot's display during simple collaboration scenes. We found that while collaboration with eyed cobots resulted in slightly higher subjective trust ratings, the objective markers such as pupil size and task completion time indicated it was in fact less comfortable to collaborate with eyed robots. These findings are in line with recent suggestions that anthropomorphism may be actually a detrimental feature of collaborative robots. These findings also show the complex relationship between human objective and subjective markers of trust when collaborating with artificial agents.

Naturalistic neuroscience and virtual reality

Virtual reality (VR) is one of the techniques that became particularly popular in neuroscience over the past few decades. VR experiments feature a closed-loop between sensory stimulation and behavior. Participants interact with the stimuli and not just passively perceive them. Several senses can be stimulated at once, large-scale environments can be simulated as well as social interactions. All of this makes VR experiences more natural than those in traditional lab paradigms. Compared to the situation in field research, a VR simulation is highly controllable and reproducible, as required of a laboratory technique used in the search for neural correlates of perception and behavior. VR is therefore considered a middle ground between ecological validity and experimental control. In this review, I explore the potential of VR in eliciting naturalistic perception and behavior in humans and non-human animals. In this context, I give an overview of recent virtual reality approaches used in neuroscientific research.

Interactive neurorobotics: Behavioral and neural dynamics of agent interactions

Interactive neurorobotics is a subfield which characterizes brain responses evoked during interaction with a robot, and their relationship with the behavioral responses. Gathering rich neural and behavioral data from humans or animals responding to agents can act as a scaffold for the design process of future social robots. This research seeks to study how organisms respond to artificial agents in contrast to biological or inanimate ones. This experiment uses the novel affordances of the robotic platforms to investigate complex dynamics during minimally structured interactions that would be difficult to capture with classical experimental setups. We then propose a general framework for such experiments that emphasizes naturalistic interactions combined with multimodal observations and complementary analysis pipelines that are necessary to render a holistic picture of the data for the purpose of informing robotic design principles. Finally, we demonstrate this approach with an exemplar rat-robot social interaction task which included simultaneous multi-agent tracking and neural recordings.

Monkey visual attention does not fall into the uncanny valley

Very humanlike artificial agents can induce feelings of uneasiness in human perceivers. Stimuli that generate this response are said to occupy "the uncanny valley". Given inconsistent findings in the literature, whether or not nonhuman animals experience the uncanny valley is unclear. Here, we recorded the visual attention of eleven male rhesus monkeys as they viewed faces varying in realness across five levels, with visual attention measured by both number and duration of visual fixations on faces as a whole and on areas of interest within the faces (e.g., eyes, mouth). Face stimuli varied in terms of the realism of the image and behavior depicted by the face (lipsmack, threat, bared teeth, and neutral). We largely found no support that rhesus monkeys perceive an uncanny valley when viewing our stimuli; however, monkeys did generally pay more attention to eyes and less attention to mouths in real images compared to less realistic images. Across all stimuli, monkeys' visual attention was drawn to the mouths of images when teeth were visible. These findings suggest that rhesus monkeys in our study did not display an uncanny valley effect when viewing realistic stimuli but did percieve affective information depicted by faces regardless of how real those faces appear.

Virtual Reality for Safe Testing and Development in Collaborative Robotics: Challenges and Perspectives

Collaborative robots (cobots) could help humans in tasks that are mundane, dangerous or where direct human contact carries risk. Yet, the collaboration between humans and robots is severely limited by the aspects of the safety and comfort of human operators. In this paper, we outline the use of extended reality (XR) as a way to test and develop collaboration with robots. We focus on virtual reality (VR) in simulating collaboration scenarios and the use of cobot digital twins. This is specifically useful in situations that are difficult or even impossible to safely test in real life, such as dangerous scenarios. We describe using XR simulations as a means to evaluate collaboration with robots without putting humans at harm. We show how an XR setting enables combining human behavioral data, subjective self-reports, and biosignals signifying human comfort, stress and cognitive load during collaboration. Several works demonstrate XR can be used to train human operators and provide them with augmented reality (AR) interfaces to enhance their performance with robots. We also provide a first attempt at what could become the basis for a human–robot collaboration testing framework, specifically for designing and testing factors affecting human–robot collaboration. The use of XR has the potential to change the way we design and test cobots, and train cobot operators, in a range of applications: from industry, through healthcare, to space operations.

A Meta-analysis of the Uncanny Valley's Independent and Dependent Variables

The uncanny valley (UV) effect is a negative affective reaction to human-looking artificial entities. It hinders comfortable, trust-based interactions with android robots and virtual characters. Despite extensive research, a consensus has not formed on its theoretical basis or methodologies. We conducted a meta-analysis to assess operationalizations of human likeness (independent variable) and the UV effect (dependent variable). Of 468 studies, 72 met the inclusion criteria. These studies employed 10 different stimulus creation techniques, 39 affect measures, and 14 indirect measures. Based on 247 effect sizes, a three-level meta-analysis model revealed the UV effect had a large effect size, Hedges’ g = 1.01 [0.80, 1.22]. A mixed-effects meta-regression model with creation technique as the moderator variable revealed face distortion produced the largest effect size, g = 1.46 [0.69, 2.24], followed by distinct entities, g = 1.20 [1.02, 1.38], realism render, g = 0.99 [0.62, 1.36], and morphing, g = 0.94 [0.64, 1.24]. Affective indices producing the largest effects were threatening, likable, aesthetics, familiarity , and eeriness , and indirect measures were dislike frequency, categorization reaction time, like frequency, avoidance , and viewing duration . This meta-analysis—the first on the UV effect—provides a methodological foundation and design principles for future research.

Great apes' understanding of biomechanics: eye-tracking experiments using three-dimensional computer-generated animations

Visual processing of the body movements of other animals is important for adaptive animal behaviors. It is widely known that animals can distinguish articulated animal movements even when they are just represented by points of light such that only information about biological motion is retained. However, the extent to which nonhuman great apes comprehend the underlying structural and physiological constraints affecting each moving body part, i.e., biomechanics, is still unclear. To address this, we examined the understanding of biomechanics in bonobos (Pan paniscus) and chimpanzees (Pan troglodytes), following a previous study on humans (Homo sapiens). Apes underwent eye tracking while viewing three-dimensional computer-generated (CG) animations of biomechanically possible or impossible elbow movements performed by a human, robot, or nonhuman ape. Overall, apes did not differentiate their gaze between possible and impossible movements of elbows. However, some apes looked at elbows for longer when viewing impossible vs. possible robot movements, which indicates that they may have had knowledge of biomechanics and that this knowledge could be extended to a novel agent. These mixed results make it difficult to draw a firm conclusion regarding the extent to which apes understand biomechanics. We discuss some methodological features that may be responsible for the results, as well as implications for future nonhuman animal studies involving the presentation of CG animations or measurement of gaze behaviors.

Levels of naturalism in social neuroscience research

In order to understand ecologically meaningful social behaviors and their neural substrates in humans and other animals, researchers have been using a variety of social stimuli in the laboratory with a goal of extracting specific processes in real-life scenarios. However, certain stimuli may not be sufficiently effective at evoking typical social behaviors and neural responses. Here, we review empirical research employing different types of social stimuli by classifying them into five levels of naturalism. We describe the advantages and limitations while providing selected example studies for each level. We emphasize the important trade-off between experimental control and ecological validity across the five levels of naturalism. Taking advantage of newly emerging tools, such as real-time videos, virtual avatars, and wireless neural sampling techniques, researchers are now more than ever able to adopt social stimuli at a higher level of naturalism to better capture the dynamics and contingency of real-life social interaction.

Shape-invariant encoding of dynamic primate facial expressions in human perception

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.

Creepy cats and strange high houses: Support for configural processing in testing predictions of nine uncanny valley theories

In 1970, Masahiro Mori proposed the uncanny valley (UV), a region in a human-likeness continuum where an entity risks eliciting a cold, eerie, repellent feeling. Recent studies have shown that this feeling can be elicited by entities modeled not only on humans but also nonhuman animals. The perceptual and cognitive mechanisms underlying the UV effect are not well understood, although many theories have been proposed to explain them. To test the predictions of nine classes of theories, a within-subjects experiment was conducted with 136 participants. The theories' predictions were compared with ratings of 10 classes of stimuli on eeriness and coldness indices. One type of theory, configural processing, predicted eight out of nine significant effects. Atypicality, in its extended form, in which the uncanny valley effect is amplified by the stimulus appearing more human, also predicted eight. Threat avoidance predicted seven; atypicality, perceptual mismatch, and mismatch+ predicted six; category+, novelty avoidance, mate selection, and psychopathy avoidance predicted five; and category uncertainty predicted three. Empathy's main prediction was not supported. Given that the number of significant effects predicted depends partly on our choice of hypotheses, a detailed consideration of each result is advised. We do, however, note the methodological value of examining many competing theories in the same experiment.

Experimental evidence for yawn contagion in orangutans (Pongo pygmaeus)

Yawning is highly contagious, yet both its proximate mechanism(s) and its ultimate causation remain poorly understood. Scholars have suggested a link between contagious yawning (CY) and sociality due to its appearance in mostly social species. Nevertheless, as findings are inconsistent, CY’s function and evolution remains heavily debated. One way to understand the evolution of CY is by studying it in hominids. Although CY has been found in chimpanzees and bonobos, but is absent in gorillas, data on orangutans are missing despite them being the least social hominid. Orangutans are thus interesting for understanding CY’s phylogeny. Here, we experimentally tested whether orangutans yawn contagiously in response to videos of conspecifics yawning. Furthermore, we investigated whether CY was affected by familiarity with the yawning individual (i.e. a familiar or unfamiliar conspecific and a 3D orangutan avatar). In 700 trials across 8 individuals, we found that orangutans are more likely to yawn in response to yawn videos compared to control videos of conspecifics, but not to yawn videos of the avatar. Interestingly, CY occurred regardless of whether a conspecific was familiar or unfamiliar. We conclude that CY was likely already present in the last common ancestor of humans and great apes, though more converging evidence is needed.

Macaque Gaze Responses to the Primatar: A Virtual Macaque Head for Social Cognition Research

Following the expanding use and applications of virtual reality in everyday life, realistic virtual stimuli are of increasing interest in cognitive studies. They allow for control of features such as gaze, expression, appearance, and movement, which may help to overcome limitations of using photographs or video recordings to study social responses. In using virtual stimuli however, one must be careful to avoid the uncanny valley effect, where realistic stimuli can be perceived as eerie, and induce an aversion response. At the same time, it is important to establish whether responses to virtual stimuli mirror responses to depictions of a real conspecific. In the current study, we describe the development of a new virtual monkey head with realistic facial features for experiments with nonhuman primates, the "Primatar." As a first step toward validation, we assessed how monkeys respond to facial images of a prototype of this Primatar compared to images of real monkeys (RMs), and an unrealistic model. We also compared gaze responses between original images and scrambled as well as obfuscated versions of these images. We measured looking time to images in six freely moving long-tailed macaques (Macaca fascicularis) and gaze exploration behavior in three rhesus macaques (Macaca mulatta). Both groups showed more signs of overt attention to original images than scrambled or obfuscated images. In addition, we found no evidence for an uncanny valley effect; since for both groups, looking times did not differ between real, realistic, or unrealistic images. These results provide important data for further development of our Primatar for use in social cognition studies and more generally for cognitive research with virtual stimuli in nonhuman primates. Future research on the absence of an uncanny valley effect in macaques is needed, to elucidate the roots of this mechanism in humans.

Overprecision increases subsequent surprise

Overconfident people should be surprised that they are so often wrong. Are they? Three studies examined the relationship between confidence and surprise in order to shed light on the psychology of overprecision in judgment. Participants reported ex-ante confidence in their beliefs, and after receiving accuracy feedback, they then reported ex-post surprise. Results show that more ex-ante confidence produces less ex-post surprise for correct answers; this relationship reverses for incorrect answers. However, this sensible pattern only holds for some measures of confidence; it fails for confidence-interval measures. The results can help explain the robust durability of overprecision in judgment.

A Naturalistic Dynamic Monkey Head Avatar Elicits Species-Typical Reactions and Overcomes the Uncanny Valley

Research on social perception in monkeys may benefit from standardized, controllable, and ethologically valid renditions of conspecifics offered by monkey avatars. However, previous work has cautioned that monkeys, like humans, show an adverse reaction toward realistic synthetic stimuli, known as the "uncanny valley" effect. We developed an improved naturalistic rhesus monkey face avatar capable of producing facial expressions (fear grin, lip smack and threat), animated by motion capture data of real monkeys. For validation, we additionally created decreasingly naturalistic avatar variants. Eight rhesus macaques were tested on the various videos and avoided looking at less naturalistic avatar variants, but not at the most naturalistic or the most unnaturalistic avatar, indicating an uncanny valley effect for the less naturalistic avatar versions. The avoidance was deepened by motion and accompanied by physiological arousal. Only the most naturalistic avatar evoked facial expressions comparable to those toward the real monkey videos. Hence, our findings demonstrate that the uncanny valley reaction in monkeys can be overcome by a highly naturalistic avatar.

Studying the visual brain in its natural rhythm

How the brain fluidly orchestrates visual behavior is a central question in cognitive neuroscience. Researchers studying neural responses in humans and nonhuman primates have mapped out visual response profiles and cognitive modulation in a large number of brain areas, most often using pared down stimuli and highly controlled behavioral paradigms. The historical emphasis on reductionism has placed most studies at one pole of an inherent trade-off between strictly controlled experimental variables and open designs that monitor the brain during its natural modes of operation. This bias toward simplified experiments has strongly shaped the field of visual neuroscience, with little guarantee that the principles and concepts established within that framework will apply more generally. In recent years, a growing number of studies have begun to relax strict experimental control with the aim of understanding how the brain responds under more naturalistic conditions. In this article, we survey research that has explicitly embraced the complexity and rhythm of natural vision. We focus on those studies most pertinent to understanding high-level visual specializations in brains of humans and nonhuman primates. We conclude that representationalist concepts borne from conventional visual experiments fall short in their ability to capture the real-life visual operations undertaken by the brain. More naturalistic approaches, though fraught with experimental and analytic challenges, provide fertile ground for neuroscientists seeking new inroads to investigate how the brain supports core aspects of our daily visual experience.

A parameterized digital 3D model of the Rhesus macaque face for investigating the visual processing of social cues

BACKGROUND

Rhesus macaques are the most popular model species for studying the neural basis of visual face processing and social interaction using intracranial methods. However, the challenge of creating realistic, dynamic, and parametric macaque face stimuli has limited the experimental control and ethological validity of existing approaches.

NEW METHOD

We performed statistical analyses of in vivo computed tomography data to generate an anatomically accurate, three-dimensional representation of Rhesus macaque cranio-facial morphology. The surface structures were further edited, rigged and textured by a professional digital artist with careful reference to photographs of macaque facial expression, colouration and pelage.

RESULTS

The model offers precise, continuous, parametric control of craniofacial shape, emotional expression, head orientation, eye gaze direction, and many other parameters that can be adjusted to render either static or dynamic high-resolution faces. Example single-unit responses to such stimuli in macaque inferotemporal cortex demonstrate the value of parametric control over facial appearance and behaviours.

COMPARISON WITH EXISTING METHOD(S)

The generation of such a high-dimensionality and systematically controlled stimulus set of conspecific faces, with accurate craniofacial modelling and professional finalization of facial details, is currently not achievable using existing methods.

CONCLUSIONS

The results herald a new set of possibilities in adaptive sampling of a high-dimensional and socially meaningful feature space, thus opening the door to systematic testing of hypotheses about the abundant neural specialization for faces found in the primate.

Judging Others by Your Own Standards: Attractiveness of Primate Faces as Seen by Human Respondents

The aspects of facial attractiveness have been widely studied, especially within the context of evolutionary psychology, which proposes that aesthetic judgements of human faces are shaped by biologically based standards of beauty reflecting the mate quality. However, the faces of primates, who are very similar to us yet still considered non-human, remain neglected. In this paper, we aimed to study the facial attractiveness of non-human primates as judged by human respondents. We asked 286 Czech respondents to score photos of 107 primate species according to their perceived "beauty." Then, we analyzed factors affecting the scores including morphology, colors, and human-likeness. We found that the three main primate groups were each scored using different cues. The proportions of inner facial features and distinctiveness are cues widely reported to affect human facial attractiveness. Interestingly, we found that these factors also affected the attractiveness scores of primate faces, but only within the Catarrhines, i.e., the primate group most similar to humans. Within this group, human-likeness positively affected the attractiveness scores, and facial extremities such as a prolonged nose or exaggerated cheeks were considered the least attractive. On the contrary, the least human-like prosimians were scored as the most attractive group. The results are discussed in the context of the "uncanny valley," the widely discussed empirical rule.

Parent-offspring facial resemblance increases with age in rhesus macaques

Kin recognition is a key ability which facilitates the acquisition of inclusive fitness benefits and enables optimal outbreeding. In primates, phenotype matching is considered particularly important for the recognition of patrilineal relatives, as information on paternity is unlikely to be available via social familiarity. Phenotypic cues to both paternal and maternal relatedness exist in the facial features of humans and other primates. However, theoretical models suggest that in systems with uncertainty parentage it may be adaptive for offspring to conceal such cues when young, in order to avoid potential costs of being discriminated against by unrelated adults. Using experienced human raters, we demonstrate in a computer-based task that detection of parent-offspring resemblances in the faces of rhesus macaques (Macaca mulatta) increases significantly with offspring age. Moreover, this effect is specific to information about kinship, as raters were extremely successful at discriminating individuals even among the youngest animals. To our knowledge, this is the first evidence in non-humans for the age-dependent expression of visual cues used in kin recognition.

Comparative thanatology, an integrative approach: exploring sensory/cognitive aspects of death recognition in vertebrates and invertebrates

Evolutionary thanatology benefits from broad taxonomic comparisons of non-human animals' responses to death. Furthermore, exploring the sensory and cognitive bases of these responses promises to allow classification of the underlying mechanisms on a spectrum from phylogenetically ancient to more derived traits. We draw on studies of perception and cognition in invertebrate and vertebrate taxa (with a focus on arthropods, corvids, proboscids, cetaceans and primates) to explore the cues that these animals use to detect life and death in others, and discuss proximate and ultimate drivers behind their capacities to do so. Parallels in thanatological behaviour exhibited by the last four taxa suggest similar sensory-cognitive processing rules for dealing with corpses, the evolution of which may have been driven by complex social environments. Uniting these responses is a phenomenon we term 'animacy detection malfunction', whereupon the corpse, having both animate and inanimate attributes, creates states of fear/curiosity manifested as approach/avoidance behaviours in observers. We suggest that integrating diverse lines of evidence (including the 'uncanny valley' effect originating from the field of robotics) provides a promising way to advance the field, and conclude by proposing avenues for future research.This article is part of the theme issue 'Evolutionary thanatology: impacts of the dead on the living in humans and other animals'.

Uncanny valley as a window into predictive processing in the social brain

Uncanny valley refers to humans' negative reaction to almost-but-not-quite-human agents. Theoretical work proposes prediction violation as an explanation for uncanny valley but no empirical work has directly tested it. Here, we provide evidence that supports this theory using event-related brain potential recordings from the human scalp. Human subjects were presented images and videos of three agents as EEG was recorded: a real human, a mechanical robot, and a realistic robot in between. The real human and the mechanical robot had congruent appearance and motion whereas the realistic robot had incongruent appearance and motion. We hypothesize that the appearance of the agent would provide a context to predict her movement, and accordingly the perception of the realistic robot would elicit an N400 effect indicating the violation of predictions, whereas the human and the mechanical robot would not. Our data confirmed this hypothesis suggesting that uncanny valley could be explained by violation of one's predictions about human norms when encountered with realistic but artificial human forms. Importantly, our results implicate that the mechanisms underlying perception of other individuals in our environment are predictive in nature.

Creepiness Creeps In: Uncanny Valley Feelings Are Acquired in Childhood

The uncanny valley posits that very human-like robots are unsettling, a phenomenon amply demonstrated in adults but unexplored in children. Two hundred forty 3- to 18-year-olds viewed one of two robots (machine-like or very human-like) and rated their feelings toward (e.g., "Does the robot make you feel weird or happy?") and perceptions of the robot's capacities (e.g., "Does the robot think for itself?"). Like adults, children older than 9 judged the human-like robot as creepier than the machine-like robot-but younger children did not. Children's perceptions of robots' mental capacities predicted uncanny feelings: children judge robots to be creepy depending on whether they have human-like minds. The uncanny valley is therefore acquired over development and relates to changing conceptions about robot minds.

Avoidance of Novelty Contributes to the Uncanny Valley

A hypothesis suggests that objects with a high degree of visual similarity to real humans trigger negative impressions (i.e., the uncanny valley). Previous studies have suggested that difficulty in object categorization elicits negative emotional reactions to enable the avoidance of potential threats. The present study further investigated this categorization-difficulty hypothesis. In an experiment, observers categorized morphed images of photographs and human doll faces as "photograph" or "doll" and evaluated the perceived eeriness of the images. Additionally, we asked the observers to answer questionnaires on behavioral inhibition systems (BIS). The results indicated that individual differences in the BIS score were associated with enhanced eeriness in the objects with a specific human likeness. These findings suggest that the tendency to avoid a potentially threatening novel experience contributes to promoting the perceived eeriness of objects with some degree of visual similarity to real humans.

Understanding the Uncanny: Both Atypical Features and Category Ambiguity Provoke Aversion toward Humanlike Robots

Robots intended for social contexts are often designed with explicit humanlike attributes in order to facilitate their reception by (and communication with) people. However, observation of an “uncanny valley”—a phenomenon in which highly humanlike entities provoke aversion in human observers—has lead some to caution against this practice. Both of these contrasting perspectives on the anthropomorphic design of social robots find some support in empirical investigations to date. Yet, owing to outstanding empirical limitations and theoretical disputes, the uncanny valley and its implications for human-robot interaction remains poorly understood. We thus explored the relationship between human similarity and people's aversion toward humanlike robots via manipulation of the agents' appearances. To that end, we employed a picture-viewing task (N_agents = 60) to conduct an experimental test (N_participants = 72) of the uncanny valley's existence and the visual features that cause certain humanlike robots to be unnerving. Across the levels of human similarity, we further manipulated agent appearance on two dimensions, typicality (prototypic, atypical, and ambiguous) and agent identity (robot, person), and measured participants' aversion using both subjective and behavioral indices. Our findings were as follows: (1) Further substantiating its existence, the data show a clear and consistent uncanny valley in the current design space of humanoid robots. (2) Both category ambiguity, and more so, atypicalities provoke aversive responding, thus shedding light on the visual factors that drive people's discomfort. (3) Use of the Negative Attitudes toward Robots Scale did not reveal any significant relationships between people's pre-existing attitudes toward humanlike robots and their aversive responding—suggesting positive exposure and/or additional experience with robots is unlikely to affect the occurrence of an uncanny valley effect in humanoid robotics. This work furthers our understanding of both the uncanny valley, as well as the visual factors that contribute to an agent's uncanniness.

Ethorobotics: A New Approach to Human-Robot Relationship

Here we aim to lay the theoretical foundations of human-robot relationship drawing upon insights from disciplines that govern relevant human behaviors: ecology and ethology. We show how the paradox of the so called “uncanny valley hypothesis” can be solved by applying the “niche” concept to social robots, and relying on the natural behavior of humans. Instead of striving to build human-like social robots, engineers should construct robots that are able to maximize their performance in their niche (being optimal for some specific functions), and if they are endowed with appropriate form of social competence then humans will eventually interact with them independent of their embodiment. This new discipline, which we call ethorobotics, could change social robotics, giving a boost to new technical approaches and applications.

Three-dimensional computer graphic animations for studying social approach behaviour in medaka fish: Effects of systematic manipulation of morphological and motion cues

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

Technical and conceptual considerations for using animated stimuli in studies of animal behavior

Rapid technical advances in the field of computer animation (CA) and virtual reality (VR) have opened new avenues in animal behavior research. Animated stimuli are powerful tools as they offer standardization, repeatability, and complete control over the stimulus presented, thereby "reducing" and "replacing" the animals used, and "refining" the experimental design in line with the 3Rs. However, appropriate use of these technologies raises conceptual and technical questions. In this review, we offer guidelines for common technical and conceptual considerations related to the use of animated stimuli in animal behavior research. Following the steps required to create an animated stimulus, we discuss (I) the creation, (II) the presentation, and (III) the validation of CAs and VRs. Although our review is geared toward computer-graphically designed stimuli, considerations on presentation and validation also apply to video playbacks. CA and VR allow both new behavioral questions to be addressed and existing questions to be addressed in new ways, thus we expect a rich future for these methods in both ultimate and proximate studies of animal behavior.

Circling Around the Uncanny Valley: Design Principles for Research Into the Relation Between Human Likeness and Eeriness

The uncanny valley effect (UVE) is a negative emotional response experienced when encountering entities that appear almost human. Research on the UVE typically investigates individual, or collections of, near human entities but may be prone to methodological circularity unless the properties that give rise to the emotional response are appropriately defined and quantified. In addition, many studies do not sufficiently control the variation in human likeness portrayed in stimulus images, meaning that the nature of stimuli that elicit the UVE is also not well defined or quantified. This article describes design criteria for UVE research to overcome the above problems by measuring three variables (human likeness, eeriness, and emotional response) and by using stimuli spanning the artificial to human continuum. These criteria allow results to be plotted and compared with the hypothesized uncanny valley curve and any effect observed can be quantified. The above criteria were applied to the methods used in a subset of existing UVE studies. Although many studies made use of some of the necessary measurements and controls, few used them all. The UVE is discussed in relation to this result and research methodology more broadly.

Familiar face + novel face = familiar face? Representational bias in the perception of morphed faces in chimpanzees

Highly social animals possess a well-developed ability to distinguish the faces of familiar from novel conspecifics to induce distinct behaviors for maintaining society. However, the behaviors of animals when they encounter ambiguous faces of familiar yet novel conspecifics, e.g., strangers with faces resembling known individuals, have not been well characterised. Using a morphing technique and preferential-looking paradigm, we address this question via the chimpanzee’s facial–recognition abilities. We presented eight subjects with three types of stimuli: (1) familiar faces, (2) novel faces and (3) intermediate morphed faces that were 50% familiar and 50% novel faces of conspecifics. We found that chimpanzees spent more time looking at novel faces and scanned novel faces more extensively than familiar or intermediate faces. Interestingly, chimpanzees looked at intermediate faces in a manner similar to familiar faces with regards to the fixation duration, fixation count, and saccade length for facial scanning, even though the participant was encountering the intermediate faces for the first time. We excluded the possibility that subjects merely detected and avoided traces of morphing in the intermediate faces. These findings suggest a bias for a feeling-of-familiarity that chimpanzees perceive familiarity with an intermediate face by detecting traces of a known individual, as 50% alternation is sufficient to perceive familiarity.

Conserved evolutionary history for quick detection of threatening faces

Humans quickly recognize threats such as snakes and threatening faces, suggesting that human ancestors evolved specialized visual systems to detect biologically relevant threat stimuli. Although non-human primates also detect snakes quickly, it is unclear whether primates share the efficient visual systems to process the threatening faces of their conspecifics. Primates may not necessarily process conspecific threats by facial expressions, because threats from conspecifics in natural situations are often accompanied by other cues such as threatening actions (or attacks) and vocal calls. Here, we show a similar threat superiority effect in both humans and macaque Japanese monkeys. In visual search tasks, monkeys and humans both responded to pictures of a threatening face of an unfamiliar adult male monkey among neutral faces faster than to pictures of a neutral face among threatening faces. However, the monkeys' response times to detect deviant pictures of a non-face stimulus were not slower when it was shown among threat faces than when it was shown among neutral faces. These results provide the first evidence that monkeys have an attentional bias toward the threatening faces of conspecifics and suggest that threatening faces are evolutionarily relevant fear stimuli. The subcortical visual systems in primates likely process not only snakes, but also more general biological threat-relevant stimuli, including threatening conspecific faces.

The Uncanny Valley: Existence and Explanations

More than 40 years ago, Japanese roboticist Masahiro Mori (1970/2005) proposed the “uncanny valley” hypothesis, which predicted a nonlinear relation between robots’ perceived human likeness and their likability. Although some studies have corroborated this hypothesis and proposed explanations for its existence, the evidence on both fronts has been mixed and open to debate. We first review the literature to ascertain whether the uncanny valley exists. We then try to explain the uncanny phenomenon by reviewing hypotheses derived from diverse theoretical and methodological perspectives within psychology and allied fields, including evolutionary, social, cognitive, and psychodynamic approaches. Next, we provide an evaluation and critique of these studies by focusing on their methodological limitations, leading us to question the accepted definition of the uncanny valley. We examine the definitions of human likeness and likability, and propose a statistical test to preliminarily quantify their nonlinear relation. We argue that the uncanny valley hypothesis is ultimately an engineering problem that bears on the possibility of building androids that may some day become indistinguishable from humans. In closing, we propose a dehumanization hypothesis to explain the uncanny phenomenon.

Infant discrimination of humanoid robots

Recently, extremely humanlike robots called “androids” have been developed, some of which are already being used in the field of entertainment. In the context of psychological studies, androids are expected to be used in the future as fully controllable human stimuli to investigate human nature. In this study, we used an android to examine infant discrimination ability between human beings and non-human agents. Participants (N = 42 infants) were assigned to three groups based on their age, i.e., 6- to 8-month-olds, 9- to 11-month-olds, and 12- to 14-month-olds, and took part in a preferential looking paradigm. Of three types of agents involved in the paradigm—a human, an android modeled on the human, and a mechanical-looking robot made from the android—two at a time were presented side-by-side as they performed a grasping action. Infants’ looking behavior was measured using an eye tracking system, and the amount of time spent focusing on each of three areas of interest (face, goal, and body) was analyzed. Results showed that all age groups predominantly looked at the robot and at the face area, and that infants aged over 9 months watched the goal area for longer than the body area. There was no difference in looking times and areas focused on between the human and the android. These findings suggest that 6- to 14-month-olds are unable to discriminate between the human and the android, although they can distinguish the mechanical robot from the human.

Reducing consistency in human realism increases the uncanny valley effect; increasing category uncertainty does not

Human replicas may elicit unintended cold, eerie feelings in viewers, an effect known as the uncanny valley. Masahiro Mori, who proposed the effect in 1970, attributed it to inconsistencies in the replica's realism with some of its features perceived as human and others as nonhuman. This study aims to determine whether reducing realism consistency in visual features increases the uncanny valley effect. In three rounds of experiments, 548 participants categorized and rated humans, animals, and objects that varied from computer animated to real. Two sets of features were manipulated to reduce realism consistency. (For humans, the sets were eyes-eyelashes-mouth and skin-nose-eyebrows.) Reducing realism consistency caused humans and animals, but not objects, to appear eerier and colder. However, the predictions of a competing theory, proposed by Ernst Jentsch in 1906, were not supported: The most ambiguous representations-those eliciting the greatest category uncertainty-were neither the eeriest nor the coldest.

A Bayesian Model of the Uncanny Valley Effect for Explaining the Effects of Therapeutic Robots in Autism Spectrum Disorder

One of the core features of autism spectrum disorder (ASD) is impaired reciprocal social interaction, especially in processing emotional information. Social robots are used to encourage children with ASD to take the initiative and to interact with the robotic tools to stimulate emotional responses. However, the existing evidence is limited by poor trial designs. The purpose of this study was to provide computational evidence in support of robot-assisted therapy for children with ASD. We thus propose an emotional model of ASD that adapts a Bayesian model of the uncanny valley effect, which holds that a human-looking robot can provoke repulsion and sensations of eeriness. Based on the unique emotional responses of children with ASD to the robots, we postulate that ASD induces a unique emotional response curve, more like a cliff than a valley. Thus, we performed numerical simulations of robot-assisted therapy to evaluate its effects. The results showed that, although a stimulus fell into the uncanny valley in the typical condition, it was effective at avoiding the uncanny cliff in the ASD condition. Consequently, individuals with ASD may find it more comfortable, and may modify their emotional response, if the robots look like deformed humans, even if they appear "creepy" to typical individuals. Therefore, we suggest that our model explains the effects of robot-assisted therapy in children with ASD and that human-looking robots may have potential advantages for improving social interactions in ASD.

Persistence of the uncanny valley: the influence of repeated interactions and a robot's attitude on its perception

The uncanny valley theory proposed by Mori has been heavily investigated in the recent years by researchers from various fields. However, the videos and images used in these studies did not permit any human interaction with the uncanny objects. Therefore, in the field of human-robot interaction it is still unclear what, if any, impact an uncanny-looking robot will have in the context of an interaction. In this paper we describe an exploratory empirical study using a live interaction paradigm that involved repeated interactions with robots that differed in embodiment and their attitude toward a human. We found that both investigated components of the uncanniness (likeability and eeriness) can be affected by an interaction with a robot. Likeability of a robot was mainly affected by its attitude and this effect was especially prominent for a machine-like robot. On the other hand, merely repeating interactions was sufficient to reduce eeriness irrespective of a robot's embodiment. As a result we urge other researchers to investigate Mori's theory in studies that involve actual human-robot interaction in order to fully understand the changing nature of this phenomenon.

A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness

The uncanny valley hypothesis, proposed already in the 1970s, suggests that almost but not fully humanlike artificial characters will trigger a profound sense of unease. This hypothesis has become widely acknowledged both in the popular media and scientific research. Surprisingly, empirical evidence for the hypothesis has remained inconsistent. In the present article, we reinterpret the original uncanny valley hypothesis and review empirical evidence for different theoretically motivated uncanny valley hypotheses. The uncanny valley could be understood as the naïve claim that any kind of human-likeness manipulation will lead to experienced negative affinity at close-to-realistic levels. More recent hypotheses have suggested that the uncanny valley would be caused by artificial–human categorization difficulty or by a perceptual mismatch between artificial and human features. Original formulation also suggested that movement would modulate the uncanny valley. The reviewed empirical literature failed to provide consistent support for the naïve uncanny valley hypothesis or the modulatory effects of movement. Results on the categorization difficulty hypothesis were still too scarce to allow drawing firm conclusions. In contrast, good support was found for the perceptual mismatch hypothesis. Taken together, the present review findings suggest that the uncanny valley exists only under specific conditions. More research is still needed to pinpoint the exact conditions under which the uncanny valley phenomenon manifests itself.

Animal-oriented virtual environments: illusion, dilation, and discovery

As a research tool, virtual environments (VEs) hold immense promise for brain scientists. Yet to fully realize this potential in non-human systems, theoretical and conceptual perspectives must be developed. When selectively coupled to nervous systems, virtual environments can help us better understand the functional architecture of animals’ brains during naturalistic behaviors. While this will no doubt allow us to further our understanding of the neural basis of behavior, there is also an opportunity to uncover the diversity inherent in brain activity and behavior. This is due to two properties of virtual environments: the ability to create sensory illusions, and the ability to dilate space and/or time. These and other potential manipulations will be characterized as the effects of virtuality. In addition, the systems-level outcomes of virtual environment enhanced perception will be discussed in the context of the uncanny valley and other expected relationships between emotional valence, cognition, and training. These effects and their usefulness for brain science will be understood in the context of three types of neurobehavioral phenomena: sensorimotor integration, spatial navigation, and interactivity. For each of these behaviors, a combination of illusory and space/time dilation examples will be reviewed. Once these examples are presented, the implications for improving upon virtual models for more directly inducing the mental phenomena of illusion and space/time dilation will be considered. To conclude, future directions for integrating the use of VEs into a strategy of broader biological inquiry will be presented.

A reappraisal of the uncanny valley: categorical perception or frequency-based sensitization?

The uncanny valley (UCV) hypothesis describes a non-linear relationship between perceived human-likeness and affective response. The “uncanny valley” refers to an intermediate level of human-likeness that is associated with strong negative affect. Recent studies have suggested that the uncanny valley might result from the categorical perception of human-like stimuli during identification. When presented with stimuli sharing human-like traits, participants attempt to segment the continuum in “human” and “non-human” categories. Due to the ambiguity of stimuli located at a category boundary, categorization difficulty gives rise to a strong, negative affective response. Importantly, researchers who have studied the UCV in terms of categorical perception have focused on categorization responses rather than affective ratings. In the present study, we examined whether the negative affect associated with the UCV might be explained in terms of an individual's degree of exposure to stimuli. In two experiments, we tested a frequency-based model against a categorical perception model using a category-learning paradigm. We manipulated the frequency of exemplars that were presented to participants from two categories during a training phase. We then examined categorization and affective responses functions, as well as the relationship between categorization and affective responses. Supporting previous findings, categorization responses suggested that participants acquired novel category structures that reflected a category boundary. These category structures appeared to influence affective ratings of eeriness. Crucially, participants' ratings of eeriness were additionally affected by exemplar frequency. Taken together, these findings suggest that the UCV is determined by both categorical properties as well as the frequency of individual exemplars retained in memory.

The evolution of speech: vision, rhythm, cooperation

A full account of human speech evolution must consider its multisensory, rhythmic, and cooperative characteristics. Humans, apes, and monkeys recognize the correspondence between vocalizations and their associated facial postures, and gain behavioral benefits from them. Some monkey vocalizations even have a speech-like acoustic rhythmicity but lack the concomitant rhythmic facial motion that speech exhibits. We review data showing that rhythmic facial expressions such as lip-smacking may have been linked to vocal output to produce an ancestral form of rhythmic audiovisual speech. Finally, we argue that human vocal cooperation (turn-taking) may have arisen through a combination of volubility and prosociality, and provide comparative evidence from one species to support this hypothesis.

Facial expressions and the evolution of the speech rhythm

In primates, different vocalizations are produced, at least in part, by making different facial expressions. Not surprisingly, humans, apes, and monkeys all recognize the correspondence between vocalizations and the facial postures associated with them. However, one major dissimilarity between monkey vocalizations and human speech is that, in the latter, the acoustic output and associated movements of the mouth are both rhythmic (in the 3- to 8-Hz range) and tightly correlated, whereas monkey vocalizations have a similar acoustic rhythmicity but lack the concommitant rhythmic facial motion. This raises the question of how we evolved from a presumptive ancestral acoustic-only vocal rhythm to the one that is audiovisual with improved perceptual sensitivity. According to one hypothesis, this bisensory speech rhythm evolved through the rhythmic facial expressions of ancestral primates. If this hypothesis has any validity, we expect that the extant nonhuman primates produce at least some facial expressions with a speech-like rhythm in the 3- to 8-Hz frequency range. Lip smacking, an affiliative signal observed in many genera of primates, satisfies this criterion. We review a series of studies using developmental, x-ray cineradiographic, EMG, and perceptual approaches with macaque monkeys producing lip smacks to further investigate this hypothesis. We then explore its putative neural basis and remark on important differences between lip smacking and speech production. Overall, the data support the hypothesis that lip smacking may have been an ancestral expression that was linked to vocal output to produce the original rhythmic audiovisual speech-like utterances in the human lineage.

Empirical evaluation of the uncanny valley hypothesis fails to confirm the predicted effect of motion

The uncanny valley hypothesis states that the acceptability of an artificial character will not increase linearly in relation to its likeness to human form. Instead, after an initial rise in acceptability there will be a pronounced decrease when the character is similar, but not identical to human form (Mori, 1970/2012). Moreover, it has been claimed but never directly tested that movement would accentuate this dip and make moving characters less acceptable. We used a number of full-body animated computer characters along with a parametrically defined motion set to examine the effect of motion quality on the uncanny valley. We found that improving the motion quality systematically improved the acceptability of the characters. In particular, the character classified in the deepest location of the uncanny valley became more acceptable when it was animated. Our results showed that although an uncanny valley was found for static characters, the deepening of the valley with motion, originally predicted by Mori (1970/2012), was not obtained.