Relaxing Gaze Aversion of Adolescents With Autism Spectrum Disorder in Consecutive Conversations With Human and Android Robot-A Preliminary Study

Interactive mirrOring Games wIth sOCial rObot (IOGIOCO): a pilot study on the use of intransitive gestures in a sample of Italian preschool children with autism spectrum disorder

Background

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by persistent deficits in social communication, social interaction, and restricted behaviors. The importance of early intervention has been widely demonstrated, and developmental trajectories in ASD emphasize the importance of nonverbal communication, such as intransitive gesture production, as a possible positive prognostic factor for language development. The use of technological tools in the therapy of individuals with ASD has also become increasingly important due to their higher engagement and responsiveness to technological objects, such as robots.

Materials and methods

We developed a training protocol using the humanoid robot NAO, called IOGIOCO (Interactive mirroring Games wIth sOCial rObot), based on the use of intransitive gestures embedded in naturalistic dialogues, stimulating a triadic interaction between child, robot and therapist. The training was divided into six levels; the first 2 levels were called "familiarization levels," and the other 4 were "training levels". The technological setup includes different complexity levels, from mirroring tasks to building spontaneous interactions. We tested the protocol on 10 preschool children with ASD (aged 2-6 years) for 14 weeks. We assessed them at recruitment (T0), at the end of training (T1), and after 6 months (T2).

Results

We demonstrated the tolerability of the protocol. We found that one group (n=4, males and 2 females) reached the training level, while another and group (n=6 males) remained at a familiarization level (mirroring), we analyzed the results for the two groups. In the group that reached the training levels, we found promising results, such as an improvement in the Social Adaptive Domain of the ABAS-II questionnaire between T0 and T2.

Conclusion

While current results will need a Randomized Controlled Trial to be confirmed, the present work sets an important milestone in using social robots for ASD treatment, aimed at impacting social and communication skills in everyday life.

Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting

Background

Robots offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Determining the optimal motion of robots when interacting with individuals with ASD is important for achieving more natural human-robot interactions and for exploiting the full potential of robotic interventions. Most prior studies have used supervised machine learning (ML) of user behavioral data to enable robot perception of affective states (i.e., arousal and valence) and engagement. It has previously been suggested that including personal demographic information in the identification of individuals with ASD is important for developing an automated system to perceive individual affective states and engagement. In this study, we hypothesized that assessing self-administered questionnaire data would contribute to the development of an automated estimation of the affective state and engagement when individuals with ASD are interviewed by an Android robot, which will be linked to implementing long-term interventions and maintaining the motivation of participants.

Methods

Participants sat across a table from an android robot that played the role of the interviewer. Each participant underwent a mock job interview. Twenty-five participants with ASD (males 22, females 3, average chronological age = 22.8, average IQ = 94.04) completed the experiment. We collected multimodal data (i.e., audio, motion, gaze, and self-administered questionnaire data) to train a model to correctly classify the state of individuals with ASD when interviewed by an android robot. We demonstrated the technical feasibility of using ML to enable robot perception of affect and engagement of individuals with ASD based on multimodal data.

Results

For arousal and engagement, the area under the curve (AUC) values of the model estimates and expert coding were relatively high. Overall, the AUC values of arousal, valence, and engagement were improved by including self-administered questionnaire data in the classification.

Discussion

These findings support the hypothesis that assessing self-administered questionnaire data contributes to the development of an automated estimation of an individual's affective state and engagement. Given the efficacy of including self-administered questionnaire data, future studies should confirm the effectiveness of such long-term intervention with a robot to maintain participants' motivation based on the proposed method of emotion estimation.

Development of depression assessment tools using humanoid robots -Can tele-operated robots talk with depressive persons like humans?

BACKGROUND

Depression is a common mental disorder and causes significant social loss. Early intervention for depression is important. Nonetheless, depressed patients tend to conceal their symptoms from others based on shame and stigma, thus hesitate to visit psychiatrists especially during early phase. We hypothesize that application of humanoid robots would be a novel solution. Depressed patients may feel more comfortable talking with such robots than humans.

METHODS

We recruited 13 patients with major depressive disorder (MDD) and 27 healthy volunteers as controls. Participants took both tele-operated humanoid robot and human interviews to evaluate severity of depression using the Hamilton Depression Rating Scale (HDRS). In addition, participants completed a self-administered questionnaire asking about their impressions of the robot interview.

RESULTS

Confidence interval and t-test analysis have revealed that the HDRS scores are equally reliable between robot and human interviews. No significant differences were observed between the two interviews regarding "nervousness about the interview" and "hesitancy to talk about depressed moods and suicidal ideation." Compared to human interviews, robot interviews yielded significantly lower scores on shame-related factors especially among patients with MDD.

LIMITATION

Small sample size, and the evaluator is male only.

CONCLUSIONS

This is the first report to show the reliability of tele-operated humanoid robot interviews for assessment of depression. Robot interviews are potentially equally reliable as human interviews. Robot interviews are suggested to be more appropriate in assessing shame-related suppressed emotions and hidden thoughts of depressed patients in clinical practice, which may reduce the stigma associated with depression.

Measuring Social Modulation of Gaze in Autism Spectrum Condition With Virtual Reality Interviews

Gaze behavior in dyadic conversations can indicate active listening and attention. However, gaze behavior that is different from the engagement expected during neurotypical social interaction cues may be interpreted as uninterested or inattentive, which can be problematic in both personal and professional situations. Neurodivergent individuals, such as those with autism spectrum conditions, often exhibit social communication differences broadly including via gaze behavior. This project aims to support situational social gaze practice through a virtual reality (VR) mock job interview practice using the HTC Vive Pro Eye VR headset. We show how gaze behavior varies in the mock job interview between neurodivergent and neurotypical participants. We also investigate the social modulation of gaze behavior based on conversational role (speaking and listening). Our three main contributions are: (i) a system for fully-automatic analysis of social modulation of gaze behavior using a portable VR headset with a novel realistic mock job interview, (ii) a signal processing pipeline, which employs Kalman filtering and spatial-temporal density-based clustering techniques, that can improve the accuracy of the headset's built-in eye-tracker, and (iii) being the first to investigate social modulation of gaze behavior among neurotypical/divergent individuals in the realm of immersive VR.

Sharing Worlds: Design of a Real-Time Attention Classifier for Robotic Therapy of ASD Children. IEEE Int Conf Rehabil Robot 2022;2022:1-6. [PMID: 36176158 DOI: 10.1109/icorr55369.2022.9896506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Joint attention is the capacity of sharing attention between two agents and an aspect of the environment, through the use of different cues, namely gaze. This capacity is of paramount importance for social skills. People with Autism Spectrum Disorder (ASD) present certain deficits in joint attention. Therefore, there is an increasing interest in finding therapies to improve this skill. Some of these therapies include robots since they are known to be attractive to people with autism due to their motivation ability and predictability when compared with humans. In this line, we have designed a real-time attention classifier for a triadic robotic therapy, using Gaze360 and geometrical considerations of the scene. We were able to classify the gaze of the therapist and the one of the child during the whole session, even in a highly unconstrained scenario with a single camera, achieving a mean accuracy of 59%. This classifier can be used for the measurement of joint attention, an important metric for the development of adaptive robotic therapies, where increasing levels of difficulty and engagement are provided dependent on the ASD children, who are characterised by high heterogeneity. Future work will pass by the calculation of this metric and integration on a robotic platform for ASD therapy to understand the impact of these robotic therapies in improving ASD symptoms, specifically on how ASD children share their attention with other people present in the rehabilitation scenarios.

Robotics in Clinical and Developmental Psychology

Technological advances in robotics over the last 20 years have allowed us to explore the use of robots in different healthcare contexts, in which robots can be deployed as tools for intervention and rehabilitation programs. This chapter intends to analyze, in a lifespan perspective (childhood, adulthood, and elderly age), the potentialities that the use of robots can offer in clinical practices without neglecting the robot's technical constraints and the methodological limitations of the studies. We will provide suggestions for future research and indications for the clinical application of robots according to the different pathologies and ages.

Differences in the Optimal Motion of Android Robots for the Ease of Communications Among Individuals With Autism Spectrum Disorders

Android robots are employed in various fields. Many individuals with autism spectrum disorders (ASD) have the motivation and aptitude for using such robots. Interactions with these robots are structured to resemble social situations in which certain social behaviors can occur and to simulate daily life. Considering that individuals with ASD have strong likes and dislikes, ensuring not only the optimal appearance but also the optimal motion of robots is important to achieve smooth interaction and to draw out the potential of robotic interventions. We investigated whether individuals with ASD found it easier to talk to an android robot with little motion (i.e., only opening and closing its mouth during speech) or an android robot with much motion (i.e., in addition to opening and closing its mouth during speech, moving its eyes from side to side and up and down, blinking, deeply breathing, and turning or moving its head or body at random). This was a crossover study in which a total of 25 participants with ASD experienced mock interviews conducted by an android robot with much spontaneous facial and bodily motion and an android robot with little motion. We compared demographic data between participants who answered that the android robot with much motion was easier to talk to than android robot with little motion and those who answered the opposite. In addition, we investigated how each type of demographic data was related to participants' feeling of comfort in an interview setting with an android robot. Fourteen participants indicated that the android robot with little motion was easier to talk to than the robot with much motion, whereas 11 participants answered the opposite. There were significant differences between these two groups in the sensory sensitivity score, which reflects the tendency to show a low neurological threshold. In addition, we found correlations between the sensation seeking score, which reflects the tendency to show a high neurological threshold, and self-report ratings of comfort in each condition. These results provide preliminary support for the importance of setting the motion of an android robot considering the sensory traits of ASD.

Enhance the Language Ability of Humanoid Robot NAO through Deep Learning to Interact with Autistic Children

Autism spectrum disorder (ASD) is a life-long neurological disability, and a cure has not yet been found. ASD begins early in childhood and lasts throughout a person’s life. Through early intervention, many actions can be taken to improve the quality of life of children. Robots are one of the best choices for accompanying children with autism. However, for most robots, the dialogue system uses traditional techniques to produce responses. Robots cannot produce meaningful answers when the conversations have not been recorded in a database. The main contribution of our work is the incorporation of a conversation model into an actual robot system for supporting children with autism. We present the use a neural network model as the generative conversational agent, which aimed at generating meaningful and coherent dialogue responses given the dialogue history. The proposed model shares an embedding layer between the encoding and decoding processes through adoption. The model is different from the canonical Seq2Seq model in which the encoder output is used only to set-up the initial state of the decoder to avoid favoring short and unconditional responses with high prior probability. In order to improve the sensitivity to context, we changed the input method of the model to better adapt to the utterances of children with autism. We adopted transfer learning to make the proposed model learn the characteristics of dialogue with autistic children and to solve the problem of the insufficient corpus of dialogue. Experiments showed that the proposed method was superior to the canonical Seq2sSeq model and the GAN-based dialogue model in both automatic evaluation indicators and human evaluation, including pushing the BLEU precision to 0.23, the greedy matching score to 0.69, the embedding average score to 0.82, the vector extrema score to 0.55, the skip-thought score to 0.65, the KL divergence score to 5.73, and the EMD score to 12.21.

Emotional Expression in Children With ASD: A Pre-Study on a Two-Group Pre-Post-Test Design Comparing Robot-Based and Computer-Based Training

Several studies have found a delay in the development of facial emotion recognition and expression in children with an autism spectrum condition (ASC). Several interventions have been designed to help children to fill this gap. Most of them adopt technological devices (i.e., robots, computers, and avatars) as social mediators and reported evidence of improvement. Few interventions have aimed at promoting emotion recognition and expression abilities and, among these, most have focused on emotion recognition. Moreover, a crucial point is the generalization of the ability acquired during treatment to naturalistic interactions. This study aimed to evaluate the effectiveness of two technological-based interventions focused on the expression of basic emotions comparing a robot-based type of training with a "hybrid" computer-based one. Furthermore, we explored the engagement of the hybrid technological device introduced in the study as an intermediate step to facilitate the generalization of the acquired competencies in naturalistic settings. A two-group pre-post-test design was applied to a sample of 12 children (M = 9.33; ds = 2.19) with autism. The children were included in one of the two groups: group 1 received a robot-based type of training (n = 6); and group 2 received a computer-based type of training (n = 6). Pre- and post-intervention evaluations (i.e., time) of facial expression and production of four basic emotions (happiness, sadness, fear, and anger) were performed. Non-parametric ANOVAs found significant time effects between pre- and post-interventions on the ability to recognize sadness [t ₍₁₎ = 7.35, p = 0.006; pre: M (ds) = 4.58 (0.51); post: M (ds) = 5], and to express happiness [t ₍₁₎ = 5.72, p = 0.016; pre: M (ds) = 3.25 (1.81); post: M (ds) = 4.25 (1.76)], and sadness [t ₍₁₎ = 10.89, p < 0; pre: M (ds) = 1.5 (1.32); post: M (ds) = 3.42 (1.78)]. The group^*time interactions were significant for fear [t ₍₁₎ = 1.019, p = 0.03] and anger expression [t ₍₁₎ = 1.039, p = 0.03]. However, Mann-Whitney comparisons did not show significant differences between robot-based and computer-based training. Finally, no difference was found in the levels of engagement comparing the two groups in terms of the number of voice prompts given during interventions. Albeit the results are preliminary and should be interpreted with caution, this study suggests that two types of technology-based training, one mediated via a humanoid robot and the other via a pre-settled video of a peer, perform similarly in promoting facial recognition and expression of basic emotions in children with an ASC. The findings represent the first step to generalize the abilities acquired in a laboratory-trained situation to naturalistic interactions.

Autism - A Comprehensive Array of Prominent Signs and Symptoms

BACKGROUND

Autism Spectrum Disorder (ASD) is a multifaceted neurodevelopmental condition characterized by multiple psychological and physiological impairments in young children. According to the recent reports, 1 out of every 58 newly-born children is suffering from autism. The aetiology of the disorder is complex and poorly understood, hindering the adaptation of targeted and effective therapies. There are no well- established diagnostic biomarkers for autism. Hence the analysis of symptoms by the pediatricians plays a critical role in the early intervention.

METHODS

In the present report, we have emphasized 24 behavioral, psychological and clinical symptoms of autism.

RESULTS

Impaired social interaction, restrictive and narrow interests, anxiety, depression; aggressive, repetitive, rigid and self-injurious behavior, lack of consistency, short attention span, fear, shyness and phobias, hypersensitivity and rapid mood alterations, high level of food and toy selectivity; inability to establish friendships or follow the instructions; fascination by round spinning objects and eating non-food materials are common psychological characteristics of autism. Speech or hearing impairments, poor cognitive function, gastrointestinal problems, weak immunity, disturbed sleep and circadian rhythms, weak motor neuromuscular interaction, lower level of serotonin and neurotransmitters, headache and body pain are common physiological symptoms.

CONCLUSION

A variable qualitative and quantitative impact of this wide range of symptoms is perceived in each autistic individual, making him/her distinct, incomparable and exceptional. Selection and application of highly personalized medical and psychological therapies are therefore recommended for the management and treatment of autism.

Group-Based Online Job Interview Training Program Using Virtual Robot for Individuals With Autism Spectrum Disorders

The rapid expansion of online job interviews during the COVID-19 pandemic is expected to continue after the pandemic has subsided. These interviews are a significant barrier for individuals with autism spectrum disorders (ASD). There is little evidence-based training for online job interviews for individuals with ASD, and the development of new trainings is expected. In an effort to facilitate online job interview skill acquisition for individuals with ASD, we developed a group-based online job interview training program using a virtual robot (GOT). In GOT, the interviewer and interviewee are projected as virtual robots on the screen. Five participants were grouped and performed the role of interviewee, interviewer, and evaluator. The participants performed all roles in a random order. Each session consisted of a first job interview session, feedback session, and second job interview session. The participants experienced 25 sessions. Before and after GOT, the participants underwent a mock online job interview with a human professional interviewer (MOH) to evaluate the effect of GOT. In total, 15 individuals with ASD took part in the study. The GOT improved self-confidence, motivation, the understanding of others' perspectives, verbal competence, non-verbal competence, and interview performance scores. There was also a significant increase in the recognition of the importance of the point of view of interviewers and evaluators after the second MOH compared to after the first MOH. Using a VR robot and learning the importance of interview skills by experiencing other perspectives (i.e., viewpoint of interviewer and evaluator) may have sustained their motivation and enabled greater self-confidence. Given the promising results of this study and to draw definitive conclusions regarding the efficacy of virtual reality (VR) robots for mock online job interview training, further studies with larger, more diverse samples of individuals with ASD using a longitudinal design are warranted.

Optimal robot for intervention for individuals with autism spectrum disorders

With recent rapid advances in technology, human-like robots have begun functioning in a variety of ways. As increasing anecdotal evidence suggests, robots may offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Individuals with ASD often achieve a higher degree of task engagement through the interaction with robots than through interactions with human trainees. The type and form of robots to be used for individuals with ASD have been meticulously considered. Simple robots and animal robots are acceptable because of their simplicity and the ease of interesting and engaging interactions. Android robots have the benefit of the potential of generalization into daily life to some extent. Considering the affinity between robots and users is important to draw out the potential capabilities of robotic intervention to the fullest extent. In the robotic condition, factors such as the appearance, biological motion, clothes, hairstyle, and disposition are important. Many factors of a user, such as age, sex, and IQ, may also affect the affinity of individuals with ASD toward a robot. The potential end-users of this technology may be unaware or unconvinced of the potential roles of robots in ASD interventions. If trainers have extensive experience in using robots, they can identify many potential roles of robots based on their experience. To date, only a few studies have been conducted in the field of robotics for providing assistance to individuals with ASD, and future studies are needed to realize an optimal robot for this purpose.

Collaborative Research Project: Developing and Testing a Robot-Assisted Intervention for Children With Autism

The present work is a collaborative research aimed at testing the effectiveness of the robot-assisted intervention administered in real clinical settings by real educators. Social robots dedicated to assisting persons with autism spectrum disorder (ASD) are rarely used in clinics. In a collaborative effort to bridge the gap between innovation in research and clinical practice, a team of engineers, clinicians and researchers working in the field of psychology developed and tested a robot-assisted educational intervention for children with low-functioning ASD (N = 20) A total of 14 lessons targeting requesting and turn-taking were elaborated, based on the Pivotal Training Method and principles of Applied Analysis of Behavior. Results showed that sensory rewards provided by the robot elicited more positive reactions than verbal praises from humans. The robot was of greatest benefit to children with a low level of disability. The educators were quite enthusiastic about children's progress in learning basic psychosocial skills from interactions with the robot. The robot nonetheless failed to act as a social mediator, as more prosocial behaviors were observed in the control condition, where instead of interacting with the robot children played with a ball. We discuss how to program robots to the distinct needs of individuals with ASD, how to harness robots' likability in order to enhance social skill learning, and how to arrive at a consensus about the standards of excellence that need to be met in interdisciplinary co-creation research. Our intuition is that robotic assistance, obviously judged as to be positive by educators, may contribute to the dissemination of innovative evidence-based practice for individuals with ASD.

Socially Assistive Robots for Older Adults and People with Autism: An Overview

Over one billion people in the world suffer from some form of disability. Nevertheless, according to the World Health Organization, people with disabilities are particularly vulnerable to deficiencies in services, such as health care, rehabilitation, support, and assistance. In this sense, recent technological developments can mitigate these deficiencies, offering less-expensive assistive systems to meet users’ needs. This paper reviews and summarizes the research efforts toward the development of these kinds of systems, focusing on two social groups: older adults and children with autism.