Role of Gaze Cues in Interpersonal Motor Coordination: Towards Higher Affiliation in Human-Robot Interaction

The potential of robot eyes as predictive cues in HRI-an eye-tracking study

Robots currently provide only a limited amount of information about their future movements to human collaborators. In human interaction, communication through gaze can be helpful by intuitively directing attention to specific targets. Whether and how this mechanism could benefit the interaction with robots and how a design of predictive robot eyes in general should look like is not well understood. In a between-subjects design, four different types of eyes were therefore compared with regard to their attention directing potential: a pair of arrows, human eyes, and two anthropomorphic robot eye designs. For this purpose, 39 subjects performed a novel, screen-based gaze cueing task in the laboratory. Participants' attention was measured using manual responses and eye-tracking. Information on the perception of the tested cues was provided through additional subjective measures. All eye models were overall easy to read and were able to direct participants' attention. The anthropomorphic robot eyes were most efficient at shifting participants' attention which was revealed by faster manual and saccadic reaction times. In addition, a robot equipped with anthropomorphic eyes was perceived as being more competent. Abstract anthropomorphic robot eyes therefore seem to trigger a reflexive reallocation of attention. This points to a social and automatic processing of such artificial stimuli.

The Components of Interpersonal Synchrony in the Typical Population and in Autism: A Conceptual Analysis

Interpersonal synchrony – the tendency for social partners to temporally co-ordinate their behaviour when interacting – is a ubiquitous feature of social interactions. Synchronous interactions play a key role in development, and promote social bonding and a range of pro-social behavioural outcomes across the lifespan. The process of achieving and maintaining interpersonal synchrony is highly complex, with inputs required from across perceptual, temporal, motor, and socio-cognitive domains. In this conceptual analysis, we synthesise evidence from across these domains to establish the key components underpinning successful non-verbal interpersonal synchrony, how such processes interact, and factors that may moderate their operation. We also consider emerging evidence that interpersonal synchrony is reduced in autistic populations. We use our account of the components contributing to interpersonal synchrony in the typical population to identify potential points of divergence in interpersonal synchrony in autism. The relationship between interpersonal synchrony and broader aspects of social communication in autism are also considered, together with implications for future research.

A meta-analysis on the effectiveness of anthropomorphism in human-robot interaction

The application of anthropomorphic design features is widely assumed to facilitate human-robot interaction (HRI). However, a considerable number of study results point in the opposite direction. There is currently no comprehensive common ground on the circumstances under which anthropomorphism promotes interaction with robots. Our meta-analysis aims to close this gap. A total of 4856 abstracts were scanned. After an extensive evaluation, 78 studies involving around 6000 participants and 187 effect sizes were included in this meta-analysis. The majority of the studies addressed effects on perceptual aspects of robots. In addition, effects on attitudinal, affective, and behavioral aspects were also investigated. Overall, a medium positive effect size was found, indicating a beneficial effect of anthropomorphic design features on human-related outcomes. However, closer scrutiny of the lowest variable level revealed no positive effect for perceived safety, empathy, and task performance. Moreover, the analysis suggests that positive effects of anthropomorphism depend heavily on various moderators. For example, anthropomorphism was in contrast to other fields of application, constantly facilitating social HRI. The results of this analysis provide insights into how design features can be used to improve the quality of HRI. Moreover, they reveal areas in which more research is needed before any clear conclusions about the effects of anthropomorphic robot design can be drawn.

Optimizing Android Facial Expressions Using Genetic Algorithms

Because the internal structure, degree of freedom, skin control position and range of the android face are different, it is very difficult to generate facial expressions by applying existing facial expression generation methods. In addition, facial expressions differ among robots because they are designed subjectively. To address these problems, we developed a system that can automatically generate robot facial expressions by combining an android, a recognizer capable of classifying facial expressions and a genetic algorithm. We have developed two types (older men and young women) of android face robots that can simulate human skin movements. We selected 16 control positions to generate the facial expressions of these robots. The expressions were generated by combining the displacements of 16 motors. A chromosome comprising 16 genes (motor displacements) was generated by applying real-coded genetic algorithms; subsequently, it was used to generate robot facial expressions. To determine the fitness of the generated facial expressions, expression intensity was evaluated through a facial expression recognizer. The proposed system was used to generate six facial expressions (angry, disgust, fear, happy, sad, surprised); the results confirmed that they were more appropriate than manually generated facial expressions.

Eye gaze as a means of giving and seeking information during musical interaction

During skilled music ensemble performance, a multi-layered network of interaction processes allows musicians to negotiate common interpretations of ambiguously-notated music in real-time. This study investigated the conditions that encourage visual interaction during duo performance. Duos recorded performances of a new piece before and after a period of rehearsal. Mobile eye tracking and motion capture were used in combination to map uni- and bidirectional eye gaze patterns. Musicians watched each other more during temporally-unstable passages than during regularly-timed passages. They also watched each other more after rehearsal than before. Duo musicians may seek visual interaction with each other primarily, but not exclusively, when coordination is threatened by temporal instability. Visual interaction increases as musicians become familiar with the piece, suggesting that they visually monitor each other once a shared interpretation of the piece is established. Visual monitoring of co-performers' movements and attention may facilitate feelings of engagement and high-level creative collaboration.

Solo versus joint bimanual coordination

Understanding the differences between solo and joint action control is an important goal in psychology. The present study represented a novel approach in which participants performed a bimanual finger oscillation task, either alone or in pairs. It was hypothesized that performance of this task relies heavily on attention and utilizes two independent processes that differentially affect solo and joint performance. One process attempts to align the fingers correctly regardless of oscillation speed, and this is reflected in an alignment error evident even at slow oscillations. A second process attempts to minimize the time lag between the fingers as the oscillation speed increases, reflected in a temporal error indexed by the rate of error increase with increasing movement speed. In three experiments, alignment and temporal error in the finger oscillation task were compared in solo and joint actors. Overall, solo actors had much lower alignment error than joint actors. Solo actors also showed a reduction in temporal error when the fingers moved in a symmetrical rather than parallel fashion, consistent with previous research showing an increase in error with increasing movement speed. However, the effect of symmetry on temporal error did not occur with joint actors. Similar results were found with one hand inverted, suggesting that the pattern of results was not due to the use of homologous muscles. To test the role of visual feedback, we examined the effect of denying visual feedback to one of the actors in the joint condition. Paradoxically, under these conditions, there was lower temporal error in the symmetrical condition. These results are interpreted in terms of the organization of solo versus joint actions and the control of bimanual tasks in general.

Patterns of Joint Improvisation in Adults with Autism Spectrum Disorder

Recent research on autism spectrum disorders (ASDs) suggests that individuals with autism may have a basic deficit in synchronizing with others, and that this difficulty may lead to more complex social and communicative deficits. Here, we examined synchronization during an open-ended joint improvisation (JI) paradigm, called the mirror game (MG). In the MG, two players take turns leading, following, and jointly improvising motion using two handles set on parallel tracks, while their motion tracks are recorded with high temporal and spatial resolution. A series of previous studies have shown that players in the MG attain moments of highly synchronized co-confident (CC) motion, in which there is no typical kinematic pattern of leader and reactive follower. It has been suggested that during these moments players act as a coupled unit and feel high levels of connectedness. Here, we aimed to assess whether participants with ASD are capable of attaining CC, and whether their MG performance relates to broader motor and social skills. We found that participants with ASD (n = 34) can indeed attain CC moments when playing with an expert improviser, though their performance was attenuated in several ways, compared to typically developing (TD) participants (n = 35). Specifically, ASD participants had lower rates of CC, compared with TD participants, which was most pronounced during the following rounds. In addition, the duration of their CC segments was shorter, across all rounds. When controlling for participants' motor skills (both on the MG console, and more broadly) some of the variability in MG performance was explained, but group differences remained. ASD participants' alexithymia further correlated with their difficulty following another's lead; though other social skills did not relate to MG performance. Participants' subjective reports of the game suggest that other cognitive and emotional factors, such as attention, motivation, and reward-processing, which were not directly measured in the experiment, may impact their performance. Together, these results show that ASD participants can attain moments of high motor synchronization with an expert improviser, even during an open-ended task. Future studies should examine the ways in which these skills may be further harnessed in clinical settings.

Synchrony in Joint Action Is Directed by Each Participant's Motor Control System

In this work, we ask how the probability of achieving synchrony in joint action is affected by the choice of motion parameters of each individual. We use the mirror game paradigm to study how changes in leader's motion parameters, specifically frequency and peak velocity, affect the probability of entering the state of co-confidence (CC) motion: a dyadic state of synchronized, smooth and co-predictive motions. In order to systematically study this question, we used a one-person version of the mirror game, where the participant mirrored piece-wise rhythmic movements produced by a computer on a graphics tablet. We systematically varied the frequency and peak velocity of the movements to determine how these parameters affect the likelihood of synchronized joint action. To assess synchrony in the mirror game we used the previously developed marker of co-confident (CC) motions: smooth, jitter-less and synchronized motions indicative of co-predicative control. We found that when mirroring movements with low frequencies (i.e., long duration movements), the participants never showed CC, and as the frequency of the stimuli increased, the probability of observing CC also increased. This finding is discussed in the framework of motor control studies showing an upper limit on the duration of smooth motion. We confirmed the relationship between motion parameters and the probability to perform CC with three sets of data of open-ended two-player mirror games. These findings demonstrate that when performing movements together, there are optimal movement frequencies to use in order to maximize the possibility of entering a state of synchronized joint action. It also shows that the ability to perform synchronized joint action is constrained by the properties of our motor control systems.

Humanoid robots versus humans: How is emotional valence of facial expressions recognized by individuals with schizophrenia? An exploratory study

BACKGROUND

The use of humanoid robots to play a therapeutic role in helping individuals with social disorders such as autism is a newly emerging field, but remains unexplored in schizophrenia. As the ability for robots to convey emotion appear of fundamental importance for human-robot interactions, we aimed to evaluate how schizophrenia patients recognize positive and negative facial emotions displayed by a humanoid robot.

METHODS

We included 21 schizophrenia outpatients and 17 healthy participants. In a reaction time task, they were shown photographs of human faces and of a humanoid robot (iCub) expressing either positive or negative emotions, as well as a non-social stimulus. Patients' symptomatology, mind perception, reaction time and number of correct answers were evaluated.

RESULTS

Results indicated that patients and controls recognized better and faster the emotional valence of facial expressions expressed by humans than by the robot. Participants were faster when responding to positive compared to negative human faces and inversely were faster for negative compared to positive robot faces. Importantly, participants performed worse when they perceived iCub as being capable of experiencing things (experience subscale of the mind perception questionnaire). In schizophrenia patients, negative correlations emerged between negative symptoms and both robot's and human's negative face accuracy.

CONCLUSIONS

Individuals do not respond similarly to human facial emotion and to non-anthropomorphic emotional signals. Humanoid robots have the potential to convey emotions to patients with schizophrenia, but their appearance seems of major importance for human-robot interactions.