Using a social robot to teach gestural recognition and production in children with autism spectrum disorders

Interactive mirrOring Games wIth sOCial rObot (IOGIOCO): a pilot study on the use of intransitive gestures in a sample of Italian preschool children with autism spectrum disorder

Background

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by persistent deficits in social communication, social interaction, and restricted behaviors. The importance of early intervention has been widely demonstrated, and developmental trajectories in ASD emphasize the importance of nonverbal communication, such as intransitive gesture production, as a possible positive prognostic factor for language development. The use of technological tools in the therapy of individuals with ASD has also become increasingly important due to their higher engagement and responsiveness to technological objects, such as robots.

Materials and methods

We developed a training protocol using the humanoid robot NAO, called IOGIOCO (Interactive mirroring Games wIth sOCial rObot), based on the use of intransitive gestures embedded in naturalistic dialogues, stimulating a triadic interaction between child, robot and therapist. The training was divided into six levels; the first 2 levels were called "familiarization levels," and the other 4 were "training levels". The technological setup includes different complexity levels, from mirroring tasks to building spontaneous interactions. We tested the protocol on 10 preschool children with ASD (aged 2-6 years) for 14 weeks. We assessed them at recruitment (T0), at the end of training (T1), and after 6 months (T2).

Results

We demonstrated the tolerability of the protocol. We found that one group (n=4, males and 2 females) reached the training level, while another and group (n=6 males) remained at a familiarization level (mirroring), we analyzed the results for the two groups. In the group that reached the training levels, we found promising results, such as an improvement in the Social Adaptive Domain of the ABAS-II questionnaire between T0 and T2.

Conclusion

While current results will need a Randomized Controlled Trial to be confirmed, the present work sets an important milestone in using social robots for ASD treatment, aimed at impacting social and communication skills in everyday life.

People with Autism Spectrum Disorder Could Interact More Easily with a Robot than with a Human: Reasons and Limits

Individuals with Autism Spectrum Disorder show deficits in communication and social interaction, as well as repetitive behaviors and restricted interests. Interacting with robots could bring benefits to this population, notably by fostering communication and social interaction. Studies even suggest that people with Autism Spectrum Disorder could interact more easily with a robot partner rather than a human partner. We will be looking at the benefits of robots and the reasons put forward to explain these results. The interest regarding robots would mainly be due to three of their characteristics: they can act as motivational tools, and they are simplified agents whose behavior is more predictable than that of a human. Nevertheless, there are still many challenges to be met in specifying the optimum conditions for using robots with individuals with Autism Spectrum Disorder.

Information and communication technologies-based interventions for children with autism spectrum conditions: a systematic review of randomized control trials from a positive technology perspective

Information and communication technologies (ICTs) have become more widely used in the past years to help people with autism spectrum conditions (ASC). Serious games embedded into computers or tablets, as well as social robots, are the most employed ICT-related tools that are appealing to and appropriate for autistic children. The goal of ICT applications is to enhance behavioral abnormalities associated with ASC while also creating an interactive link between one person and one computer. Comparatively, to human-based therapy, ICT tools aid to inspire autistic children by providing predictability and regularity of tasks. Regaining social skills is the primary behavioral goal for which ICT tools have been designed and implemented. In the past several years, many studies have been created to show how effective it is at improving targeted behaviors. However, only a small number of researchers have used an RCT approach to evaluate its effectiveness. In this systematic review, we only included RCT studies where ICT technologies were used to help children with ASC in improving their social skills. Only 14 RCT studies satisfied the criteria and 12 described significant improvements, showing how the use of technology in educational contexts produced better improvement in developing several social skill facets with respect to the traditional face-to-face approach. Some studies used interventions and outcome measures focused on the core ASC symptoms, but many others addressed neurocognitive functions directly, like social cognition or emotional regulation, while other more general functions such as language or adaptive behaviors. We propose a classification based on processes and outcome measures to foster future research in this specific area of research. The behavioral intervention mediated by technological tools such as computer-based, tablet, and social robotics, undoubtedly provides a comfortable environment that promotes constant learning for people with ASC. Evidence provided in this review highlights the translational potential of this field of study in primary care practice and educational settings.

Robot dramas may improve joint attention of Chinese-speaking low-functioning children with autism: stepped wedge trials

INTRODUCTION

Children with autism spectrum disorder (ASD), especially those with low cognitive functioning, have deficits in joint attention. Previous research has found that these children are interested in engaging with social robots.

PURPOSE

In the present study, we designed a robot drama intervention for promoting responses to joint attention abilities (RJA) of children with low-functioning autism (IQs < 70).

MATERIALS AND METHODS

Using a stepped wedge design, Chinese-speaking children aged six to eight were randomly assigned to three tiers (N = 18). Children of all three tiers had comparable autism severity, language and cognitive function, and joint attention abilities. Tier 1 first received intervention, followed by Tiers 2 and 3. They watched six dramas in which social robots demonstrated RJA behaviours.

RESULTS

The RJA of children of all tiers improved after intervention and such improvement was maintained over time. Despite initiation of joint attention (IJA) not being explicitly taught, IJA of all children was found to improve after intervention.

CONCLUSIONS

It was, therefore, concluded that a robot drama could enhance the joint attention of children with low-functioning ASD.Implications for rehabilitationIn comparison to typically developing children, children with autism spectrum disorder (ASD), especially those with low cognitive functioning, have deficits in joint attention.Robot-based drama intervention program was developed to promote responses to joint attention (RJA) abilities of children with low-functioning autism (IQs < 70).Modelling RJA in robot dramas can promote RJA abilities in these children.An improvement in RJA also yielded an increase in the initiation of joint attention (IJA) abilities.Children with low-functioning autism might be able to extract the initiation of joint attention skills from the drama, even though these behaviours were not explicitly taught.

SD, Munera M, Casas D, Villa-Moreno A, Frizera-Neto A, Stoelen MF, Belpaeme T, Cifuentes CA. Do different robot appearances change emotion recognition in children with ASD?

Introduction

Socially Assistive Robotics has emerged as a potential tool for rehabilitating cognitive and developmental disorders in children with autism. Social robots found in the literature are often able to teach critical social skills, such as emotion recognition and physical interaction. Even though there are promising results in clinical studies, there is a lack of guidelines on selecting the appropriate robot and how to design and implement the child-robot interaction.

Methods

This work aims to evaluate the impacts of a social robot designed with three different appearances according to the results of a participatory design (PD) process with the community. A validation study in the emotion recognition task was carried out with 21 children with autism.

Results

Spectrum disorder results showed that robot-like appearances reached a higher percentage of children's attention and that participants performed better when recognizing simple emotions, such as happiness and sadness.

Discussion

This study offers empirical support for continuing research on using SAR to promote social interaction with children with ASD. Further long-term research will help to identify the differences between high and low-functioning children.

Influence of a Socially Assistive Robot on Physical Activity, Social Play Behavior, and Toy-Use Behaviors of Children in a Free Play Environment: A Within-Subjects Study

Background: Play is critical for children's physical, cognitive, and social development. Technology-based toys like robots are especially of interest to children. This pilot study explores the affordances of the play area provided by developmentally appropriate toys and a mobile socially assistive robot (SAR). The objective of this study is to assess the role of the SAR on physical activity, play behavior, and toy-use behavior of children during free play. Methods: Six children (5 females, Mage = 3.6 ± 1.9 years) participated in the majority of our pilot study's seven 30-minute-long weekly play sessions (4 baseline and 3 intervention). During baseline sessions, the SAR was powered off. During intervention sessions, the SAR was teleoperated to move in the play area and offered rewards of lights, sounds, and bubbles to children. Thirty-minute videos of the play sessions were annotated using a momentary time sampling observation system. Mean percentage of time spent in behaviors of interest in baseline and intervention sessions were calculated. Paired-Wilcoxon signed rank tests were conducted to assess differences between baseline and intervention sessions. Results: There was a significant increase in children's standing (∼15%; Z = -2.09; p = 0.037) and a tendency for less time sitting (∼19%; Z = -1.89; p = 0.059) in the intervention phase as compared to the baseline phase. There was also a significant decrease (∼4.5%, Z = -2.70; p = 0.007) in peer interaction play and a tendency for greater (∼4.5%, Z = -1.89; p = 0.059) interaction with adults in the intervention phase as compared to the baseline phase. There was a significant increase in children's interaction with the robot (∼11.5%, Z = -2.52; p = 0.012) in the intervention phase as compared to the baseline phase. Conclusion: These results may indicate that a mobile SAR provides affordances through rewards that elicit children's interaction with the SAR and more time standing in free play. This pilot study lays a foundation for exploring the role of SARs in inclusive play environments for children with and without mobility disabilities in real-world settings like day-care centers and preschools.

Improving Human-Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.

Design of a Robotic Coach for Motor, Social and Cognitive Skills Training Toward Applications With ASD Children

Socially assistive robots may help the treatment of autism spectrum disorder(ASD), through games using dyadic interactions to train social skills. Existing systems are mainly based on simplified protocols which qualitatively evaluate subject performance. We propose a robotic coaching platform for training social, motor and cognitive capabilities, with two main contributions: (i) using triadic interactions(adult, robot and child), with robotic mirroring, and (ii) providing quantitative performance indicators. The key system features were accurately designed, including type of protocols, feedback systems and evaluation metrics, contemplating the requirements for applications with ASD children. We implemented two protocols, Robot-Master and Adult-Master, where children performed different gestures guided by the robot or the adult respectively, eventually receiving feedback about movement execution. In both, the robot mirrors the subject during the movement. To assess system functionalities, with a homogeneous group of subjects, tests were carried out with 28 healthy subjects; one preliminary acquisition was done with an ASD child. Data analysis was customized to design protocol-specific parameters for movement characterization. Our tests show that robotic mirroring execution depends on the complexity and standardization of movements, as well as on the robot technical features. The feedback system evaluated movement phases and successfully estimated the completion of the exercises. Future work includes improving platform flexibility and adaptability, and clinical trials with ASD children to test the impact of the robotic coach on reducing symptoms. We trust that the proposed quantitative performance indicators extend the current state-of-the-art towards clinical usage of robotic-based coaching systems.

Social Robots in Special Education: A Systematic Review

In recent years, social robots have become part of a variety of human activities, especially in applications involving children, e.g., entertainment, education, companionship. The interest of this work lies in the interaction of social robots with children in the field of special education. This paper seeks to present a systematic review of the use of robots in special education, with the ultimate goal of highlighting the degree of integration of robots in this field worldwide. This work aims to explore the technologies of robots that are applied according to the impairment type of children. The study showed a large number of attempts to apply social robots to the special education of children with various impairments, especially in recent years, as well as a wide variety of social robots from the market involved in such activities. The main conclusion of this work is the finding that the specific field of application of social robots is at the first development step; however, it is expected to be of great concern to the research community in the coming years.

Are social robots ready yet to be used in care and therapy of autism spectrum disorder: A systematic review of randomized controlled trials

Autism is a neurodevelopmental disorder that affects the everyday life of people who have this lifelong condition. Robots hold great promise for uplifting therapy and care of the affected population. We searched Scopus, Medline, ScienceDirect, Web of Science, and PubMed databases for randomized controlled trials that had evaluated robot use in the therapy of people with autism, to see how effective social robots have been incorporated in autism care. Out of 240 papers initially identified, 19 satisfied the inclusion criteria and were fully evaluated. Overall, 10 different robots were utilized in the trials, out of which, four were non-humanoids. The number of papers with positive results for using robots on the main and secondary parameters was 11 and 5, respectively. Three papers reported that robot groups did not achieve better results than others. Robots in the papers included here were mainly added as the "entertainment agent" to elicit greater engagement from the participants, which is understandable, as robots at this stage might not be ready yet to deliver high-end care.

An Open-Source Social Robot Based on Compliant Soft Robotics for Therapy with Children with ASD

Therapy with robotic tools is a promising way to help improve verbal and nonverbal communication in children. The robotic tools are able to increase aspects such as eye contact and the ability to follow instructions and to empathize with others. This work presents the design methodology, development, and experimental validation of a novel social robot based on CompliAnt SofT Robotics called the CASTOR robot, which intends to be used as an open-source platform for the long-term therapy of children with autism spectrum disorder (CwASD). CASTOR integrates the concepts of soft actuators and compliant mechanisms to create a replicable robotic platform aimed at real therapy scenarios involving physical interaction between the children and the robot. The validation shows promising results in terms of robustness and the safety of the user and robot. Likewise, mechanical tests assess the robot’s response to blocking conditions for two critical modules (i.e., neck and arm) in interaction scenarios. Future works should focus on the validation of the robot’s effectiveness in the therapy of CwASD.

Social robots as treatment agents: Pilot randomized controlled trial to deliver a behavior change intervention

Social robots are increasingly demonstrating effectiveness as low-intensity behavior change agents. Key targets for these behavioral interventions include daily lifestyle behaviors with significant health consequences, such as the consumption of high-calorie foods and drinks ('snacks'). A pilot randomized controlled trial using a stepped-wedge design was conducted to determine the efficacy of a motivational intervention by an autonomous robot, to help reduce high-calorie snacks. Twenty-six adults were randomized to receive Immediate or 4-week Delayed treatment, with assessments at Baseline and Weeks 4 and 8. The treatment comprised motivation enhancement and self-management training using mental imagery (Functional Imagery Training). A significant condition by time effect for snack episode reduction was obtained, F(2, 32.06) = 4.30, p = .022. The Immediate condition significantly reduced snacking between Baseline and Week 4 (d = -1.06), while the Delayed condition did not (d = -0.08). Immediate participants maintained their improvement between Weeks 4 and 8 (d = -0.18), and Delayed participants then showed a significant fall (d = -1.42). Overall, 'Immediate' participants decreased their snack episodes by 54% and 'Delayed' decreased by 62% from Baseline to Week 8, and an average weight reduction of 4.4 kg was seen across over the first 2 weeks of treatment. Four weeks after starting the intervention, both conditions had significant increases in perceived confidence to control snack intake for time duration, specific scenarios and emotional states (d = 0.61 to 1.42). Working alliance was significantly correlated with reduced snack episodes. The pilot's results appear to suggest that the robot-delivered intervention may be as effective as a human clinician delivering a similar intervention. The robot-delivered pilot achieved similar snack episode reduction in the first four weeks (FIT-R, 55%) when compared with the human-delivered version by a trained clinician (FIT-H, 49%). Overall, the results provide preliminary evidence for an autonomous social robot to deliver a low-intensity treatment on dietary intake without the need for human intervention. Future trials should extend the deployment of the robot-delivered intervention protocol to other low-intensity behavioral outcomes.

Psychosocial Health Interventions by Social Robots: Systematic Review of Randomized Controlled Trials

Background

Social robots that can communicate and interact with people offer exciting opportunities for improved health care access and outcomes. However, evidence from randomized controlled trials (RCTs) on health or well-being outcomes has not yet been clearly synthesized across all health domains where social robots have been tested.

Objective

This study aimed to undertake a systematic review examining current evidence from RCTs on the effects of psychosocial interventions by social robots on health or well-being.

Methods

Medline, PsycInfo, ScienceDirect, Scopus, and Engineering Village searches across all years in the English language were conducted and supplemented by forward and backward searches. The included papers reported RCTs that assessed changes in health or well-being from interactions with a social robot across at least 2 measurement occasions.

Results

Out of 408 extracted records, 27 trials met the inclusion criteria: 6 in child health or well-being, 9 in children with autism spectrum disorder, and 12 with older adults. No trials on adolescents, young adults, or other problem areas were identified, and no studies had interventions where robots spontaneously modified verbal responses based on speech by participants. Most trials were small (total N=5 to 415; median=34), only 6 (22%) reported any follow-up outcomes (2 to 12 weeks; median=3.5) and a single-blind assessment was reported in 8 (31%). More recent trials tended to have greater methodological quality. All papers reported some positive outcomes from robotic interventions, although most trials had some measures that showed no difference or favored alternate treatments.

Conclusions

Controlled research on social robots is at an early stage, as is the current range of their applications to health care. Research on social robot interventions in clinical and health settings needs to transition from exploratory investigations to include large-scale controlled trials with sophisticated methodology, to increase confidence in their efficacy.

Robot-based intervention may reduce delay in the production of intransitive gestures in Chinese-speaking preschoolers with autism spectrum disorder

Background

Past studies have shown that robot-based intervention was effective in improving gestural use in children with autism spectrum disorders (ASD). The present study examined whether children with ASD could catch up to the level of gestural production found in age-matched children with typical development and whether they showed an increase in verbal imitation after the completion of robot-based training. We also explored the cognitive and motor skills associated with gestural learning.

Methods

Children with ASD were randomly assigned to two groups. Four- to 6-year-old children with ASD in the intervention group (N = 15) received four 30-min robot-based gestural training sessions. In each session, a social robot, NAO, narrated five stories and gestured (e.g., both hands clapping for an awesome expression). Children with ASD were told to imitate the gestures during training. Age-matched children with ASD in the wait-list control group (N = 15) and age-matched children with typical development (N = 15) received the gestural training after the completion of research. Standardized pretests and posttests (both immediate and delayed) were administered to assess the accuracy and appropriateness of gestural production in both training and novel stories. Children's language and communication abilities, gestural recognition skills, fine motor proficiencies, and attention skills were also examined.

Results

Children with ASD in the intervention condition were more likely to produce accurate or appropriate intransitive gestures in training and novel stories than those in the wait-list control. The positive learning outcomes were maintained in the delayed posttests. The level of gestural production accuracy in children with ASD in the delayed posttest of novel stories was comparable to that in children with typical development, suggesting that children with ASD could catch up to the level of gestural production found in children with typical development. Children with ASD in the intervention condition were also more likely to produce verbal markers while gesturing than those in the wait-list control. Gestural recognition skills were found to significantly predict the learning of gestural production accuracy in the children with ASD, with such relation partially mediated via spontaneous imitation.

Conclusions

Robot-based intervention may reduce the gestural delay in children with ASD in their early childhood.