Embodied artificial agents for understanding human social cognition

Nonconscious Cognitive Suffering: Considering Suffering Risks of Embodied Artificial Intelligence

Strong arguments have been formulated that the computational limits of disembodied artificial intelligence (AI) will, sooner or later, be a problem that needs to be addressed. Similarly, convincing cases for how embodied forms of AI can exceed these limits makes for worthwhile research avenues. This paper discusses how embodied cognition brings with it other forms of information integration and decision-making consequences that typically involve discussions of machine cognition and similarly, machine consciousness. N. Katherine Hayles’s novel conception of nonconscious cognition in her analysis of the human cognition-consciousness connection is discussed in relation to how nonconscious cognition can be envisioned and exacerbated in embodied AI. Similarly, this paper offers a way of understanding the concept of suffering in a way that is different than the conventional sense of attributing it to either a purely physical state or a conscious state, instead of grounding at least a type of suffering in this form of cognition.

From social brains to social robots: applying neurocognitive insights to human-robot interaction

Amidst the fourth industrial revolution, social robots are resolutely moving from fiction to reality. With sophisticated artificial agents becoming ever more ubiquitous in daily life, researchers across different fields are grappling with the questions concerning how humans perceive and interact with these agents and the extent to which the human brain incorporates intelligent machines into our social milieu. This theme issue surveys and discusses the latest findings, current challenges and future directions in neuroscience- and psychology-inspired human-robot interaction (HRI). Critical questions are explored from a transdisciplinary perspective centred around four core topics in HRI: technical solutions for HRI, development and learning for HRI, robots as a tool to study social cognition, and moral and ethical implications of HRI. Integrating findings from diverse but complementary research fields, including social and cognitive neurosciences, psychology, artificial intelligence and robotics, the contributions showcase ways in which research from disciplines spanning biological sciences, social sciences and technology deepen our understanding of the potential and limits of robotic agents in human social life. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

In natural interaction with embodied robots, we prefer it when they follow our gaze: a gaze-contingent mobile eyetracking study

Initiating joint attention by leading someone's gaze is a rewarding experience which facilitates social interaction. Here, we investigate this experience of leading an agent's gaze while applying a more realistic paradigm than traditional screen-based experiments. We used an embodied robot as our main stimulus and recorded participants' eye movements. Participants sat opposite a robot that had either of two 'identities'-'Jimmy' or 'Dylan'. Participants were asked to look at either of two objects presented on screens to the left and the right of the robot. Jimmy then looked at the same object in 80% of the trials and at the other object in the remaining 20%. For Dylan, this proportion was reversed. Upon fixating on the object of choice, participants were asked to look back at the robot's face. We found that return-to-face saccades were conducted earlier towards Jimmy when he followed the gaze compared with when he did not. For Dylan, there was no such effect. Additional measures indicated that our participants also preferred Jimmy and liked him better. This study demonstrates (a) the potential of technological advances to examine joint attention where ecological validity meets experimental control, and (b) that social reorienting is enhanced when we initiate joint attention. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.

Intentional Mindset Toward Robots-Open Questions and Methodological Challenges

Natural and effective interaction with humanoid robots should involve social cognitive mechanisms of the human brain that normally facilitate social interaction between humans. Recent research has indicated that the presence and efficiency of these mechanisms in human-robot interaction (HRI) might be contingent on the adoption of a set of attitudes, mindsets, and beliefs concerning the robot's inner machinery. Current research is investigating the factors that influence these mindsets, and how they affect HRI. This review focuses on a specific mindset, namely the “intentional mindset” in which intentionality is attributed to another agent. More specifically, we focus on the concept of adopting the intentional stance toward robots, i.e., the tendency to predict and explain the robots' behavior with reference to mental states. We discuss the relationship between adoption of intentional stance and lower-level mechanisms of social cognition, and we provide a critical evaluation of research methods currently employed in this field, highlighting common pitfalls in the measurement of attitudes and mindsets.

On the role of eye contact in gaze cueing

Most experimental protocols examining joint attention with the gaze cueing paradigm are "observational" and "offline", thereby not involving social interaction. We examined whether within a naturalistic online interaction, real-time eye contact influences the gaze cueing effect (GCE). We embedded gaze cueing in an interactive protocol with the iCub humanoid robot. This has the advantage of ecological validity combined with excellent experimental control. Critically, before averting the gaze, iCub either established eye contact or not, a manipulation enabled by an algorithm detecting position of the human eyes. For non-predictive gaze cueing procedure (Experiment 1), only the eye contact condition elicited GCE, while for counter-predictive procedure (Experiment 2), only the condition with no eye contact induced GCE. These results reveal an interactive effect of strategic (gaze validity) and social (eye contact) top-down components on the reflexive orienting of attention induced by gaze cues. More generally, we propose that naturalistic protocols with an embodied presence of an agent can cast a new light on mechanisms of social cognition.

A neurocognitive investigation of the impact of socialising with a robot on empathy for pain.. [DOI: 10.1101/470534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

To what extent can humans form social relationships with robots? In the present study, we combined functional neuroimaging with a robot socialising intervention to probe the flexibility of empathy, a core component of social relationships, toward robots. Twenty-six individuals underwent identical fMRI sessions before and after being issued a social robot to take home and interact with over the course of a week. While undergoing fMRI, participants observed videos of a human actor or a robot experiencing pain or pleasure in response to electrical stimulation. Repetition suppression of activity in the pain network, a collection of brain regions associated with empathy and emotional responding, was measured to test whether socialising with a social robot leads to greater overlap in neural mechanisms when observing human and robotic agents experiencing pain or pleasure. In contrast to our hypothesis, functional region-of-interest analyses revealed no change in neural overlap for agents after the socialising intervention. Similarly, no increase in activation when observing a robot experiencing pain emerged post-socialising. Whole-brain analysis showed that, before the socialising intervention, superior parietal and early visual regions are sensitive to novel agents, while after socialising, medial temporal regions show agent sensitivity. A region of the inferior parietal lobule was sensitive to novel emotions, but only during the pre-socialising scan session. Together, these findings suggest that a short socialisation intervention with a social robot does not lead to discernible differences in empathy toward the robot, as measured by behavioural or brain responses. We discuss the extent to which longer term socialisation with robots might shape social cognitive processes and ultimately our relationships with these machines.

Fair Proxy Communication: Using Social Robots to Modify the Mechanisms of Implicit Social Cognition

This article introduces a new communicational format called Fair Proxy Communication. Fair Proxy Communication is a specific communicational setting in which a teleoperated robot is used to remove perceptual cues of implicit biases in order to increase the perceived fairness of decision-related communications. The envisaged practical applications of Fair Proxy Communication range from assessment communication (e.g. job interviews at Affirmative Action Employers) to conflict mediation, negotiation and other communication scenarios that require direct dialogue but where decision-making maybe negatively affected by implicit social biases. The theoretical significance of Fair Proxy Communication pertains primarily to the investigation of 'mechanisms' of implicit social cognition in neuropsychology, but this new communicational format also raises many research questions for the fields of organisational psychology, negotiation and conflict research and business ethics. Fair Proxy Communication is currently investigated by an interdisciplinary research team at Aarhus University, Denmark.

Response Coordination Emerges in Cooperative but Not Competitive Joint Task

Effective social interactions rely on humans' ability to attune to others within social contexts. Recently, it has been proposed that the emergence of shared representations, as indexed by the Joint Simon effect (JSE), might result from interpersonal coordination (Malone et al., 2014). The present study aimed at examining interpersonal coordination in cooperative and competitive joint tasks. To this end, in two experiments we investigated response coordination, as reflected in instantaneous cross-correlation, when co-agents cooperate (Experiment 1) or compete against each other (Experiment 2). In both experiments, participants performed a go/no-go Simon task alone and together with another agent in two consecutive sessions. In line with previous studies, we found that social presence differently affected the JSE under cooperative and competitive instructions. Similarly, cooperation and competition were reflected in co-agents response coordination. For the cooperative session (Experiment 1), results showed higher percentage of interpersonal coordination for the joint condition, relative to when participants performed the task alone. No difference in the coordination of responses occurred between the individual and the joint conditions when co-agents were in competition (Experiment 2). Finally, results showed that interpersonal coordination between co-agents implies the emergence of the JSE. Taken together, our results suggest that shared representations seem to be a necessary, but not sufficient, condition for interpersonal coordination.

Intentionality of a co-actor influences sensorimotor synchronisation with a virtual partner

Interpersonal sensorimotor synchronisation requires individuals to anticipate and adapt to their partner's movement timing. Research has demonstrated that the intentionality of a co-actor affects joint action planning, however, less is known about whether co-actor intentionality affects sensorimotor synchronisation. Explicit and implicit knowledge of a synchronisation partner's intentionality may influence coordination by modulating temporal anticipation and adaptation processes. We used a computer-controlled virtual partner (VP) consisting of tempo-changing auditory pacing sequences to simulate either an intentional or unintentional synchronisation partner. The VP was programmed to respond to the participant with low or moderate degrees of error correction, simulating a slightly or moderately adaptive human, respectively. In addition, task instructions were manipulated so that participants were told they were synchronising with either another person or a computer. Results indicated that synchronisation performance improved with the more adaptive VP. In addition, there was an influence of the explicit partner instruction, but this was dependent upon the degree of VP adaptivity and was modulated by subjective preferences for either the human or the computer partner. Beliefs about the intentionality of a synchronisation partner may thus influence interpersonal sensorimotor synchronisation in a manner that is modulated by preferences for interacting with intentional agents.

From automata to animate beings: the scope and limits of attributing socialness to artificial agents

Understanding the mechanisms and consequences of attributing socialness to artificial agents has important implications for how we can use technology to lead more productive and fulfilling lives. Here, we integrate recent findings on the factors that shape behavioral and brain mechanisms that support social interactions between humans and artificial agents. We review how visual features of an agent, as well as knowledge factors within the human observer, shape attributions across dimensions of socialness. We explore how anthropomorphism and dehumanization further influence how we perceive and interact with artificial agents. Based on these findings, we argue that the cognitive reconstruction within the human observer is likely to be far more crucial in shaping our interactions with artificial agents than previously thought, while the artificial agent's visual features are possibly of lesser importance. We combine these findings to provide an integrative theoretical account based on the "like me" hypothesis, and discuss the key role played by the Theory-of-Mind network, especially the temporal parietal junction, in the shift from mechanistic to social attributions. We conclude by highlighting outstanding questions on the impact of long-term interactions with artificial agents on the behavioral and brain mechanisms of attributing socialness to these agents.

Perception of social synchrony induces mother-child gamma coupling in the social brain

The recent call to move from focus on one brain’s functioning to two-brain communication initiated a search for mechanisms that enable two humans to coordinate brain response during social interactions. Here, we utilized the mother–child context as a developmentally salient setting to study two-brain coupling. Mothers and their 9-year-old children were videotaped at home in positive and conflictual interactions. Positive interactions were microcoded for social synchrony and conflicts for overall dialogical style. Following, mother and child underwent magnetoencephalography while observing the positive vignettes. Episodes of behavioral synchrony, compared to non-synchrony, increased gamma-band power in the superior temporal sulcus (STS), hub of social cognition, mirroring and mentalizing. This neural pattern was coupled between mother and child. Brain-to-brain coordination was anchored in behavioral synchrony; only during episodes of behavioral synchrony, but not during non-synchronous moments, mother’s and child's STS gamma power was coupled. Importantly, neural synchrony was not found during observation of unfamiliar mother-child interaction Maternal empathic/dialogical conflict style predicted mothers’ STS activations whereas child withdrawal predicted attenuated STS response in both partners. Results define a novel neural marker for brain-to-brain synchrony, highlight the role of rapid bottom-up oscillatory mechanisms for neural coupling and indicate that behavior-based processes may drive synchrony between two brains during social interactions.

Robot Faces that Follow Gaze Facilitate Attentional Engagement and Increase Their Likeability

Gaze behavior of humanoid robots is an efficient mechanism for cueing our spatial orienting, but less is known about the cognitive–affective consequences of robots responding to human directional cues. Here, we examined how the extent to which a humanoid robot (iCub) avatar directed its gaze to the same objects as our participants affected engagement with the robot, subsequent gaze-cueing, and subjective ratings of the robot’s characteristic traits. In a gaze-contingent eyetracking task, participants were asked to indicate a preference for one of two objects with their gaze while an iCub avatar was presented between the object photographs. In one condition, the iCub then shifted its gaze toward the object chosen by a participant in 80% of the trials (joint condition) and in the other condition it looked at the opposite object 80% of the time (disjoint condition). Based on the literature in human–human social cognition, we took the speed with which the participants looked back at the robot as a measure of facilitated reorienting and robot-preference, and found these return saccade onset times to be quicker in the joint condition than in the disjoint condition. As indicated by results from a subsequent gaze-cueing tasks, the gaze-following behavior of the robot had little effect on how our participants responded to gaze cues. Nevertheless, subjective reports suggested that our participants preferred the iCub following participants’ gaze to the one with a disjoint attention behavior, rated it as more human-like and as more likeable. Taken together, our findings show a preference for robots who follow our gaze. Importantly, such subtle differences in gaze behavior are sufficient to influence our perception of humanoid agents, which clearly provides hints about the design of behavioral characteristics of humanoid robots in more naturalistic settings.

Folk-Psychological Interpretation of Human vs. Humanoid Robot Behavior: Exploring the Intentional Stance toward Robots

People rely on shared folk-psychological theories when judging behavior. These theories guide people's social interactions and therefore need to be taken into consideration in the design of robots and other autonomous systems expected to interact socially with people. It is, however, not yet clear to what degree the mechanisms that underlie people's judgments of robot behavior overlap or differ from the case of human or animal behavior. To explore this issue, participants (N = 90) were exposed to images and verbal descriptions of eight different behaviors exhibited either by a person or a humanoid robot. Participants were asked to rate the intentionality, controllability and desirability of the behaviors, and to judge the plausibility of seven different types of explanations derived from a recently proposed psychological model of lay causal explanation of human behavior. Results indicate: substantially similar judgments of human and robot behavior, both in terms of (1a) ascriptions of intentionality/controllability/desirability and in terms of (1b) plausibility judgments of behavior explanations; (2a) high level of agreement in judgments of robot behavior - (2b) slightly lower but still largely similar to agreement over human behaviors; (3) systematic differences in judgments concerning the plausibility of goals and dispositions as explanations of human vs. humanoid behavior. Taken together, these results suggest that people's intentional stance toward the robot was in this case very similar to their stance toward the human.

Robots As Intentional Agents: Using Neuroscientific Methods to Make Robots Appear More Social

Robots are increasingly envisaged as our future cohabitants. However, while considerable progress has been made in recent years in terms of their technological realization, the ability of robots to interact with humans in an intuitive and social way is still quite limited. An important challenge for social robotics is to determine how to design robots that can perceive the user's needs, feelings, and intentions, and adapt to users over a broad range of cognitive abilities. It is conceivable that if robots were able to adequately demonstrate these skills, humans would eventually accept them as social companions. We argue that the best way to achieve this is using a systematic experimental approach based on behavioral and physiological neuroscience methods such as motion/eye-tracking, electroencephalography, or functional near-infrared spectroscopy embedded in interactive human-robot paradigms. This approach requires understanding how humans interact with each other, how they perform tasks together and how they develop feelings of social connection over time, and using these insights to formulate design principles that make social robots attuned to the workings of the human brain. In this review, we put forward the argument that the likelihood of artificial agents being perceived as social companions can be increased by designing them in a way that they are perceived as intentional agents that activate areas in the human brain involved in social-cognitive processing. We first review literature related to social-cognitive processes and mechanisms involved in human-human interactions, and highlight the importance of perceiving others as intentional agents to activate these social brain areas. We then discuss how attribution of intentionality can positively affect human-robot interaction by (a) fostering feelings of social connection, empathy and prosociality, and by (b) enhancing performance on joint human-robot tasks. Lastly, we describe circumstances under which attribution of intentionality to robot agents might be disadvantageous, and discuss challenges associated with designing social robots that are inspired by neuroscientific principles.

Attending to and neglecting people: bridging neuroscience, psychology and sociology

Human behaviour is context-dependent-based on predictions and influenced by the environment and other people. We live in a dynamic world where both the social stimuli and their context are constantly changing. Similar dynamic, natural stimuli should, in the future, be increasingly used to study social brain functions, with parallel development of appropriate signal-analysis methods. Understanding dynamic neural processes also requires accurate time-sensitive characterization of the behaviour. To go beyond the traditional stimulus-response approaches, brain activity should be recorded simultaneously from two interacting subjects to reveal why human social interaction is critically different from just reacting to each other. This theme issue on Attending to and neglecting people contains original work and review papers on person perception and social interaction. The articles cover research from neuroscience, psychology, robotics, animal interaction research and microsociology. Some of the papers are co-authored by scientists who presented their own, independent views in the recent Attention and Performance XXVI conference but were brave enough to join forces with a colleague having a different background and views. In the future, information needs to converge across disciplines to provide us a more holistic view of human behaviour, its interactive nature, as well as the temporal dynamics of our social world.