Human–Robot Interaction in Autism Treatment: A Case Study on Three Pairs of Autistic Children as Twins, Siblings, and Classmates

People with Autism Spectrum Disorder Could Interact More Easily with a Robot than with a Human: Reasons and Limits

Individuals with Autism Spectrum Disorder show deficits in communication and social interaction, as well as repetitive behaviors and restricted interests. Interacting with robots could bring benefits to this population, notably by fostering communication and social interaction. Studies even suggest that people with Autism Spectrum Disorder could interact more easily with a robot partner rather than a human partner. We will be looking at the benefits of robots and the reasons put forward to explain these results. The interest regarding robots would mainly be due to three of their characteristics: they can act as motivational tools, and they are simplified agents whose behavior is more predictable than that of a human. Nevertheless, there are still many challenges to be met in specifying the optimum conditions for using robots with individuals with Autism Spectrum Disorder.

Robots with tears can convey enhanced sadness and elicit support intentions

The behaviour of shedding tears is a unique human expression of emotion. Human tears have an emotional signalling function that conveys sadness and a social signalling function that elicits support intention from others. The present study aimed to clarify whether the tears of robots have the same emotional and social signalling functions as human tears, using methods employed in previous studies conducted on human tears. Tear processing was applied to robot pictures to create pictures with and without tears, which were used as visual stimuli. In Study 1, the participants viewed pictures of robots with and without tears and rated the intensity of the emotion experienced by the robot in the picture. The results showed that adding tears to a robot's picture significantly increased the rated intensity of sadness. Study 2 measured support intentions towards a robot by presenting a robot's picture with a scenario. The results showed that adding tears to the robot's picture also increased the support intentions indicating that robot tears have emotional and social signalling functions similar to those of human tears.

How Do We Perceive Our Trainee Robots? Exploring the Impact of Robot Errors and Appearance when Performing Domestic Physical Tasks on Teachers’ Trust and Evaluations

To be successful, robots that can learn new tasks from humans should interact effectively with them while being trained, and humans should be able to trust the robots’ abilities after teaching. Typically, human learners make mistakes, and teachers tolerate those errors, especially in case students exhibit acceptable progress overall. But how do errors and appearance of a trainee robot affect human teachers’ trust while the robot is generally improving in performing a task? First, an online survey with 173 participants investigated perceived severity of robot errors in performing a cooking task. These findings were then used in an interactive online experiment with 138 participants, in which participants were able to remotely teach their food preparation preferences to trainee robots with two different appearances. Compared to an untidy-looking robot, a tidy-looking robot was rated as more professional, without impacting participants’ trust. Furthermore, while larger errors at the end of iterative training had a greater impact, even a small error could significantly reduce trust in a trainee robot performing the domestic physical task of food preparation, regardless of the robot’s appearance. The present study extends HRI knowledge about teachers’ perception of trainee robots, particularly when teachers observe them accomplishing domestic physical tasks.

Robots in Games

During the past two decades, robots have been increasingly deployed in games. Researchers use games to better understand human-robot interaction and, in turn, the inclusion of social robots during gameplay creates new opportunities for novel game experiences. The contributions from social robotics and games communities cover a large spectrum of research questions using a wide variety of scenarios. In this article, we present the first comprehensive survey of the deployment of robots in games. We organise our findings according to four dimensions: (1) the societal impact of robots in games, (2) games as a research platform, (3) social interactions in games, and (4) game scenarios and materials. We discuss some significant research achievements and potential research avenues for the gaming and social robotics communities. This article describes the state of the art of the research on robots in games in the hope that it will assist researchers to contextualise their work in the field, to adhere to best practices and to identify future areas of research and multidisciplinary collaboration.

Utilizing an Emotional Robot Capable of Lip-Syncing in Robot-Assisted Speech Therapy Sessions for Children with Language Disorders

This study scrutinizes the impacts of utilizing a socially assistive robot, the RASA robot, during speech therapy sessions for children with language disorders. Two capabilities were developed for the robotic platform to enhance children-robot interactions during speech therapy interventions: facial expression communication (containing recognition and expression) and lip-syncing. Facial expression recognition was conducted by training several well-known CNN architectures on one of the most extensive facial expressions databases, the AffectNet database, and then modifying them using the transfer learning strategy performed on the CK+ dataset. The robot's lip-syncing capability was designed in two steps. The first step was concerned with designing precise schemes of the articulatory elements needed during the pronunciation of the Persian phonemes (i.e., consonants and vowels). The second step included developing an algorithm to pronounce words by disassembling them into their components (including consonants and vowels) and then morphing them into each other successively. To pursue the study's primary goal, two comparable groups of children with language disorders were considered, the intervention and control groups. The intervention group attended therapy sessions in which the robot acted as the therapist's assistant, while the control group only communicated with the human therapist. The study's first purpose was to compare the children's engagement while playing a mimic game with the affective robot and the therapist, conducted via video coding. The second objective was to assess the efficacy of the robot's presence in the speech therapy sessions alongside the therapist, accomplished by administering the Persian Test of Language Development, Persian TOLD. According to the first scenario, playing with the affective robot is more engaging than playing with the therapist. Furthermore, the statistical analysis of the study's results indicates that participating in robot-assisted speech therapy (RAST) sessions enhances children with language disorders' achievements in comparison with taking part in conventional speech therapy interventions.

Extending Robot Therapy for Children with Autism Using Mobile and Web Application

Robot treatments for children with autism have proven to be successful and effective. However, the resources needed for the treatments do not always meet the needs of the children. We overcame the lack of equipment and staff by extending the concept of robot therapy using a web and mobile application. This application enables greater availability and personification of the therapy itself. Its use in the majority of respondents contributes to improving their condition. This approach increases the flexibility of the therapy itself and makes it more accessible, enabling the patients to progress more rapidly. Although the robotic treatment presented in this paper is specific to children with autism, this approach can be generalized and applied to other areas where there are similar types of therapies.

The Quantitative Case-by-Case Analyses of the Socio-Emotional Outcomes of Children with ASD in Robot-Assisted Autism Therapy

With its focus on robot-assisted autism therapy, this paper presents case-by-case analyses of socio-emotional outcomes of 34 children aged 3–12 years old, with different cases of Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD). We grouped children by the following characteristics: ASD alone (n = 22), ASD+ADHD (n = 12), verbal (n = 11), non-verbal (n = 23), low-functioning autism (n = 24), and high-functioning autism (n = 10). This paper provides a series of separate quantitative analyses across the first and last sessions, adaptive and non-adaptive sessions, and parent and no-parent sessions, to present child experiences with the NAO robot, during play-based activities. The results suggest that robots are able to interact with children in social ways and influence their social behaviors over time. Each child with ASD is a unique case and needs an individualized approach to practice and learn social skills with the robot. We, finally, present specific child–robot intricacies that affect how children engage and learn over time as well as across different sessions.

A Systematic Review of Research on Robot-Assisted Therapy for Children with Autism

Recent studies have shown that children with autism may be interested in playing with an interactive robot. Moreover, the robot can engage these children in ways that demonstrate essential aspects of human interaction, guiding them in therapeutic sessions to practice more complex forms of interaction found in social human-to-human interactions. We review published articles on robot-assisted autism therapy (RAAT) to understand the trends in research on this type of therapy for children with autism and to provide practitioners and researchers with insights and possible future directions in the field. Specifically, we analyze 38 articles, all of which are refereed journal articles, that were indexed on Web of Science from 2009 onward, and discuss the distribution of the articles by publication year, article type, database and journal, research field, robot type, participant age range, and target behaviors. Overall, the results show considerable growth in the number of journal publications on RAAT, reflecting increased interest in the use of robot technology in autism therapy as a salient and legitimate research area. Factors, such as new advances in artificial intelligence techniques and machine learning, have spurred this growth.

Robo Ludens

The use of games as vehicles to study human-robot interaction (HRI) has been established as a suitable solution to create more realistic and naturalistic opportunities to investigate human behavior. In particular, multiplayer games that involve at least two human players and one or more robots have raised the attention of the research community. This article proposes a scoping review to qualitatively examine the literature on the use of multiplayer games in HRI scenarios employing embodied robots aiming to find experimental patterns and common game design elements. We find that researchers have been using multiplayer games in a wide variety of applications in HRI, including training, entertainment and education, allowing robots to take different roles. Moreover, robots have included different capabilities and sensing technologies, and elements such as external screens or motion controllers were used to foster gameplay. Based on our findings, we propose a design taxonomy called Robo Ludens, which identifies HRI elements and game design fundamentals and classifies important components used in multiplayer HRI scenarios. The Robo Ludens taxonomy covers considerations from a robot-oriented perspective as well as game design aspects to provide a comprehensive list of elements that can foster gameplay and bring enjoyable experiences in HRI scenarios.

Enriching Human-Robot Interaction with Mobile App in Interventions of Children with Autism Spectrum Disorder

Introduction: Autism spectrum disorder (ASD) is a group of complex lifelong neurodevelopmental disorders, characterized by difficulties in social communication and stereotyped behaviours. Due to the increasing number of children with ASD, it is important to continue developing interventions as well as invent new ones. Human-robot interaction can contribute to better outcomes for these children. There are several robots such as Nao, Kaspar, ZENO, Probo, ZECA, etc. which are used in autism interventions. Many mobile and web applications are in constant growth, too. They target skills such as collaboration, social skills, language skills, social competence, and communication. Aim: To explore the usability of the humanoid robot Kaspar and a complementary app in interventions of children with ASD. Sample: 20 children with ASD, aged between 23 and 76 months old. Method: As an added intervention for this group of children, we used the robot Kaspar and its complementary app. Kaspar is a child-sized humanoid robot that uses bodily expressions, facial expressions, gestures, and pre-recorded speech to interact with a human. Results: This intervention achieved certain positive shifts in eight of the eleven measured developmental domains, such as communication functions and means, turn taking, imitation, language skills, play, attention and daily life skills. The three categories that had inconsiderable improvement are vocalization and speech, cause and effect and coping skills. Conclusion: Based on the measurements before and after the use of Kaspar and its complementary app, there is improvement, primarily in the domains of language, imitation and communication skills and attention.

Fostering Emotion Recognition in Children with Autism Spectrum Disorder

Facial expressions are of utmost importance in social interactions, allowing communicative prompts for a speaking turn and feedback. Nevertheless, not all have the ability to express themselves socially and emotionally in verbal and non-verbal communication. In particular, individuals with Autism Spectrum Disorder (ASD) are characterized by impairments in social communication, repetitive patterns of behaviour, and restricted activities or interests. In the literature, the use of robotic tools is reported to promote social interaction with children with ASD. The main goal of this work is to develop a system capable of automatic detecting emotions through facial expressions and interfacing them with a robotic platform (Zeno R50 Robokind® robotic platform, named ZECA) in order to allow social interaction with children with ASD. ZECA was used as a mediator in social communication activities. The experimental setup and methodology for a real-time facial expression (happiness, sadness, anger, surprise, fear, and neutral) recognition system was based on the Intel® RealSense™ 3D sensor and on facial features extraction and multiclass Support Vector Machine classifier. The results obtained allowed to infer that the proposed system is adequate in support sessions with children with ASD, giving a strong indication that it may be used in fostering emotion recognition and imitation skills.

One-shot Learning from Demonstration Approach Toward a Reciprocal Sign Language-based HRI

This paper addresses the lack of proper Learning from Demonstration (LfD) architectures for Sign Language-based Human-Robot Interactions to make them more extensible. The paper proposes and implements a Learning from Demonstration structure for teaching new Iranian Sign Language signs to a teacher assistant social robot, RASA. This LfD architecture utilizes one-shot learning techniques and Convolutional Neural Network to learn to recognize and imitate a sign after seeing its demonstration (using a data glove) just once. Despite using a small, low diversity data set (~ 500 signs in 16 categories), the recognition module reached a promising 4-way accuracy of 70% on the test data and showed good potential for increasing the extensibility of sign vocabulary in sign language-based human-robot interactions. The expansibility and promising results of the one-shot Learning from Demonstration technique in this study are the main achievements of conducting such machine learning algorithms in social Human-Robot Interaction.

Human-robot interaction for social skill development in children with ASD: A literature review

Human-robot interaction has been demonstrated to be a promising methodology for developing socio-communicational skills of children and adolescents with autism spectrum disorder (ASD). This paper systematically reviews studies that report experimental results on this topic published in scientific journals between the years 2010 and2018. A total of 1805 articles from various literature were filtered based on relevance and transparency. In the first set of criteria, article titles are screened and in the second both titles and abstracts. The final number of articles which were subsequently thoroughly reviewed was 32 (N = 32). The findings suggest that there are benefits in using human-robot interaction to assist with the development of social skills for children with ASD. Specifically, it was found that the majority of studies used humanoid robots, 64% relied on a small number of participants and sessions, while few of the studies included a control group or follow-up sessions. Based on these findings, this paper tried to identify areas that have not been extensively addressed to propose several directions for future improvements for studies in this field, such as control groups with typical developmental children, minimum number of sessions and participants, as well as standardization of criteria for assessing the level of functionality for ASD children.

Social Robots in Special Education: A Systematic Review

In recent years, social robots have become part of a variety of human activities, especially in applications involving children, e.g., entertainment, education, companionship. The interest of this work lies in the interaction of social robots with children in the field of special education. This paper seeks to present a systematic review of the use of robots in special education, with the ultimate goal of highlighting the degree of integration of robots in this field worldwide. This work aims to explore the technologies of robots that are applied according to the impairment type of children. The study showed a large number of attempts to apply social robots to the special education of children with various impairments, especially in recent years, as well as a wide variety of social robots from the market involved in such activities. The main conclusion of this work is the finding that the specific field of application of social robots is at the first development step; however, it is expected to be of great concern to the research community in the coming years.

“Does cinema form the future of robotics?”: a survey on fictional robots in sci-fi movies

Abstract

Robotics and Artificial Intelligence (AI) have always been among the most popular topics in science fiction (sci-fi) movies. This paper endeavors to review popular movies containing Fictional Robots (FR) to extract the most common characteristics and interesting design ideas of robots portrayed in science fiction. To this end, 134 sci-fi films, including 108 unique FRs, were investigated regarding the robots’ different design aspects (e.g., appearance design, interactive design and artificial intelligence, and ethical and social design). Also, in each section of this paper, some characteristics of FRs are compared with real social robots. Since some researches point to the significant role of the cinema in forming the community’s expectations, it is very important to consider these characteristics and differences in choosing the future pathway of robotics. As some examples of findings, we have found that unlike the non-metallic skins/covers of real social robots, most FRs are still covered by highly detailed metal components. Moreover, the FR ability of interactions are generally (more than 90%) shown to be similar or even more advanced than normal Human–Human interactions, and this milestone was achieved by ignoring the AI challenges of real HRI. On the other hand, the ethical aspects of movies do inspire us to consider the potential ethical aspects of real robot design. All in all, according to popularity of movies, studying FR could be a step toward more appropriate development of robotics and AI entities to be accepted by general users in the real world.

Highlights:

Design and Implementation of the Voice Command Recognition and the Sound Source Localization System for Human–Robot Interaction

Human–robot interaction (HRI) is becoming more important nowadays. In this paper, a low-cost communication system for HRI is designed and implemented on the Scout robot and a robotic face. A hidden Markov model-based voice command detection system is proposed and a non-native database has been collected by Persian speakers, which contains 10 desired English commands. The experimental results confirm that the proposed system is capable to recognize the voice commands, and properly performs the task or expresses the right answer. Comparing the system with a trained system on the Julius native database shows a better true detection (about 10%).

A Case Study of a Robot-Assisted Speech Therapy for Children with Language Disorders

The aim of this study was to explore the potential of using a social robot in speech therapy interventions in children. A descriptive and explorative case study design was implemented involving the intervention for language disorder in five children with different needs with an age ranging from 9 to 12 years. Children participated in sessions with a NAO-type robot in individual sessions. Qualitative methods were used to collect data on aspects of viability, usefulness, barriers and facilitators for the child as well as for the therapist in order to obtain an indication of the effects on learning and the achievement of goals. The main results pointed out the affordances and possibilities of the use of a NAO robot in achieving speech therapy and educational goals. A NAO can contribute towards eliciting motivation, readiness towards learning and improving attention span of the children. The results of the study showed the potential that NAO has in therapy and education for children with different disabilities. More research is needed to gain insight into how a NAO can be applied best in speech therapy to make a more inclusive education conclusions.

Robot-mediated therapy to reduce worrying in persons with visual and intellectual disabilities

BACKGROUND

The study explored the use of a robot-mediated therapeutic intervention in persons with visual and intellectual disabilities.

METHOD

Three robot-mediated intervention sessions were developed to teach three coping skills for worrying. Effectiveness was examined using a multiple-baseline case study design (N = 7). Baseline, pre-intervention and post-intervention assessments included social validity, severity of worrying (PSWQ-C-NL), and observations by caregivers (SDQ). Short checklists on worrying were repeated throughout baseline and intervention stages. Transcripts of the sessions were analysed for participants' emotional openness.

RESULTS

Social validity was equally high before and after the intervention. The intervention did not impact the severity of worrying, although mentor caregivers reported a lower impact of personal difficulties for participants. We found no change in self-disclosure towards the robot over sessions.

CONCLUSIONS

The participants' positive responses warrant further exploration of using robot-mediated therapy for persons with visual and intellectual disabilities. Recommendations for additional adaptations are discussed.

Integration of a Social Robot in a Pedagogical and Logopedic Intervention with Children: A Case Study

The effectiveness of social robots such as NAO in pedagogical therapies presents a challenge. There is abundant literature focused on therapies using robots with children with autism, but there is a gap to be filled in other educational different needs. This paper describes an experience of using a NAO as an assistant in a logopedic and pedagogical therapy with children with different needs. Even if the initial robot architecture is based on genericbehaviors, the loading and execution time for each specific requirement and the needs of each child in therapy, made it necessary to develop "Adaptive Behaviors". These evolve into an adaptive architecture, appliedto the engineer-therapist-child interaction, requiring the engineer-programmer to be always present during the sessions. Benefits from the point of view of the therapist and the children and the acceptance of NAO in therapy are shown. A robot in speech-therapy sessions can play a positive role in several logopedic aspectsserving as a motivating factor for the children.Future works should be oriented in developing intelligent algorithms so as to eliminate the presence of the engineer-programmer in the sessions. Additional work proposals should consider deepening the psychological aspects of using humanoid robots in educational therapy.

Social Robotics, Education, and Religion in the Islamic World: An Iranian Perspective

The social impact of robotics applied to domains such as education, religion, nursing, and therapy across the world depends on the level of technology as well as the culture in which it is used. By studying how robots are used in Iran, a technologically-savvy country with a long history and a rich culture, we explore their possible impact on interrelated areas of religious and ethical features in education in an Islamic society. To accomplish this task, a preliminary exploratory study was conducted using two social robots as teaching assistants in Islamic religion classes for 42 elementary students. More than 90% of the participants in the study absolutely preferred the robot-assisted religion class over one taught by a human. Building on the results from the students' viewpoints and exam scores, the acceptability and potential of using social robots in the education of Islamic religious concepts in Iran are further discussed in this paper.

Examining joint attention with the use of humanoid robots-A new approach to study fundamental mechanisms of social cognition

This article reviews methods to investigate joint attention and highlights the benefits of new methodological approaches that make use of the most recent technological developments, such as humanoid robots for studying social cognition. After reviewing classical approaches that address joint attention mechanisms with the use of controlled screen-based stimuli, we describe recent accounts that have proposed the need for more natural and interactive experimental protocols. Although the recent approaches allow for more ecological validity, they often face the challenges of experimental control in more natural social interaction protocols. In this context, we propose that the use of humanoid robots in interactive protocols is a particularly promising avenue for targeting the mechanisms of joint attention. Using humanoid robots to interact with humans in naturalistic experimental setups has the advantage of both excellent experimental control and ecological validity. In clinical applications, it offers new techniques for both diagnosis and therapy, especially for children with autism spectrum disorder. The review concludes with indications for future research, in the domains of healthcare applications and human-robot interaction in general.

Computer- and Robot-Assisted Therapies to Aid Social and Intellectual Functioning of Children with Autism Spectrum Disorder

BACKGROUND AND OBJECTIVES

Children with autism spectrum disorder (ASD) experience challenges with social interactions, a core feature of the disorder. Social skills therapy has been shown to be helpful. Over the past several years, computer-assisted and robot-assisted therapies have been infiltrating the social skills teaching environment. Rapid progress in the field of technology, especially in the robotics area, offers tremendous possibilities for innovation and treatment or even education for individuals with ASD. This paper's purpose is to drive awareness of these innovative interventions in order to support the social lives of children with ASD. The aims of the paper are identifying (1) the types of Information Technology platforms that are being evaluated in computer and robot-assisted therapies for children with ASD; (2) the various disciplines or professions studying and utilizing these computer and robot-assisted social skill therapies; (3) the outcomes being evaluated in each trial; and (4) if results demonstrate benefits to children with autism.

MATERIALS AND METHODS

PubMed, CINAHL, Science Direct, and Web of Science databases were searched for clinical trials published over the past five years. Search terms incorporated the subject intersection of autism, and computer or robot-assisted therapy. Results were mined for pediatric populations only and study designs establishing controlled comparisons.

RESULTS

Eighteen unique international studies were identified that utilize robot interventions (11 studies) and serious computer game interventions (seven studies). Most demonstrated promising results in improving outcomes for children with ASD. Study implications reveal a rapidly evolving assistive technology for ASD social skills therapy.

CONCLUSIONS

These interventions show considerable promise, but more effectiveness and cost effectiveness research of high quality should be carried out with larger numbers of children. Also, further studies are necessary to evaluate these technologies' effectiveness amongst adults with ASD and within unique subsets of the higher functioning autism population.

ICT and autism care: state of the art

PURPOSE OF REVIEW

Over the past 10 years, the use of information and communication technologies (ICTs) has increased in regard to the treatment of individuals with autism spectrum disorders (ASDs). ICT support mechanisms (e.g. computers, laptops, robots) are particularly attractive and are adapted to children with ASD. In addition, ICT algorithms can offer new perspectives for clinicians, outside direct apps or gaming proposals. Here, we will focus on the use of serious games and robots because of their attractiveness and their value in working on social skills.

RECENT FINDINGS

The latest knowledge regarding the use of ICT in the forms of serious games and robotics applied to individuals with ASD shows that the field of serious games has already achieved interesting and promising results, although the clinical validations are not always complete. In the field of robotics, there are still many limitations on the use of ICT (e.g. most interaction are similar to the wizard of Oz), and questions remain concerning their eventual effectiveness.

SUMMARY

To describe the implications of the findings for clinical practice or research, we describe two large projects, namely, JEMImE and Michelangelo, as examples of current studies that are aimed at enhancing social skills in children with ASD by including novel algorithms with clinical insights in robots or serious games.