Innovative computer technology in music-based interventions for individuals with autism moving beyond traditional interactive music therapy techniques

Extended Reality-Based Mobile App Solutions for the Therapy of Children With Autism Spectrum Disorders: Systematic Literature Review

BACKGROUND

The increasing prevalence of autism spectrum disorder (ASD) has driven research interest on the therapy of individuals with autism, especially children, as early diagnosis and appropriate treatment can lead to improvement in the condition. With the widespread availability of virtual reality, augmented reality (AR), and mixed reality technologies to the public and the increasing popularity of mobile devices, the interest in the use of applications and technologies to provide support for the therapy of children with autism is growing.

OBJECTIVE

This study aims to describe the literature on the potential of virtual reality, AR, and mixed reality technologies in the context of therapy for children with ASD. We propose to investigate and analyze the temporal distribution of relevant papers, identify the target audience for studies related to extended reality apps in ASD therapy, examine the technologies used in the development of these apps, assess the skills targeted for improvement in primary studies, explore the purposes of the proposed solutions, and summarize the results obtained from their application.

METHODS

For the systematic literature review, 6 research questions were defined in the first phase, after which 5 international databases (Web of Science, Scopus, ScienceDirect, IEEE Xplore Digital Library, and ACM Digital Library) were searched using specific search strings. Results were centralized, filtered, and processed applying eligibility criteria and using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The results were refined using a technical and IT-oriented approach. The quality criteria assessed whether the research addressed ASDs, focused on children's therapy, involved targeted technologies, deployed solutions on mobile devices, and produced results relevant to our study.

RESULTS

In the first step, 179 publications were identified in Zotero reference manager software (Corporation for Digital Scholarship). After excluding articles that did not meet the eligibility or quality assessment criteria, 28 publications were finalized. The analysis revealed an increase in publications related to apps for children with autism starting in 2015 and peaking in 2019. Most studies (22/28, 79%) focused on mobile AR solutions for Android devices, which were developed using the Unity 3D platform and the Vuforia engine. Although 68% (19/28) of these apps were tested with children, 32% (9/28) were tested exclusively by developers. More than half (15/28, 54%) of the studies used interviews as an evaluation method, yielding mostly favorable although preliminary results, indicating the need for more extensive testing.

CONCLUSIONS

The findings reported in the studies highlight the fact that these technologies are appropriate for the therapy of children with ASD. Several studies showed a distinct trend toward the use of AR technology as an educational tool for people with ASD. This trend entails multidisciplinary cooperation and an integrated research approach, with an emphasis on comprehensive empirical evaluations and technology ethics.

A wearable device-enabled therapeutic approach to improve joint attention in autism spectrum disorder: a prospective pilot study

It has been previously proposed that interventions aimed at integrating and co-activating music processing and motor control systems could have therapeutic potential for priming social skill development in children with autism spectrum disorder (ASD). In this study, we assessed this hypothesis through a wearable sensor platform called Ambulosono ("Ambulo"-walk; "sono"-sound) in which pleasurable children's musical stimuli are contingently linked to effortful motor action (locomotor step size), thus creating a motivational state proposed to be conducive to joint attention (JA) operation. Five participants were recruited from a community-based partner and were assessed by scoring responses following therapist-directed bids for JA. Multiple assessment sessions through a repeated time-series design were conducted to determine baseline and post-intervention scores. The intervention session consisted of approximately 15 min of Ambulosono exposure. Baseline and post-intervention data were aggregated and analyzed using a linear mixed-effect model. The wearable sensor and wireless headphones of the Ambulosono system were tolerated by the participants, and there were no adverse effects associated with the use of the device. We found an increase in the average responses to bids for JA during the Ambulosono intervention phase compared to baseline across participants. This increase did not appear to result from enhanced general arousal. Our pilot data support feasibility and further testing of Ambulosono as a therapeutic aid for integration into community-based ASD programs to augment shared child-therapist social interactions.

Experience of Online Learning from COVID-19: Preparing for the Future of Digital Transformation in Education

COVID-19 has affected traditional instructional activities. Home-based isolation and restrictive movement measures have forced most learning activities to move from an offline to an online environment. Multiple studies have also demonstrated that teaching with virtual tools during the COVID-19 pandemic is always ineffective. This study examines the different characteristics and challenges that virtual tools brought to online education in the pre-pandemic and pandemic era, with the aim of providing experience of how virtual tools supported purely online learning during a health crisis. By searching keywords in public databases and review publications, this study tries to summarize the major topics related to the research theme. These topics are the characteristics of learning supported by technologies in pre-pandemic and pandemic era, the challenges that education systems have faced during the COVID-19 pandemic. This study also compares the functions, advantages and limitations of typical virtual tools, which has rarely been done in previous studies. This study tries to present the features of virtual tools that support online learning and the challenges regarding real-life risk scenarios, and tries to provide educational institutions with a distinct perspective for efficient teaching and learning in future potential health crises.

Design of Semantic Matching Model of Folk Music in Occupational Therapy Based on Audio Emotion Analysis

The main semantic symbol systems for people to express their emotions include natural language and music. The analysis and establishment of semantic association between language and music is helpful to provide more accurate retrieval and recommendation services for text and music. Existing researches mainly focus on the surface symbolic features and association of natural language and music, which limits the performance and interpretability of applications based on semantic association of natural language and music. Emotion is the main meaning of music expression, and the semantic range of text expression includes emotion. In this paper, the semantic features of music are extracted from audio features, and the semantic matching model of audio emotion analysis is constructed to analyze ethnic music audio emotion through feature extraction ability of deep structure. The model is based on the framework of emotional semantic matching technology and realizes the emotional semantic matching of music fragments and words through semantic emotional recognition algorithm. Multiple experiments show that when W = 0.65, the recognition rate of multichannel fusion model is 88.42%, and the model can reasonably realize audio emotion analysis. When the spatial dimension of music data changes, the classification accuracy reaches the highest when the spatial dimension is 25. Analysing the semantic association of audio promotes the application of folk music in occupational therapy.

Evaluation of the Use of Real-time 3D Graphics to Augment Therapeutic Music Sessions for Young People on the Autism Spectrum

The present research evaluates the effectiveness of CymaSense, a real-time 3D visualisation application developed by the authors, as a means of improving the communicative behaviours of autistic participants through the addition of a visual modality within therapeutic music sessions. Autism spectrum condition (ASC) is a lifelong neurodevelopmental disorder that can affect people in a number of ways, commonly through difficulties in communication. A multi-sensory approach within music sessions encourages people with ASC to engage more with the act of creating music, and with the therapists, increasing their level of communication and social interaction beyond the sessions.

This article presents a study evaluating the use of CymaSense within a series of therapeutic music sessions, and a follow-up series of semi-structured interviews. Eight adults with ASC participated in 12 sessions using a single case experimental design approach over a total period of 19 weeks. Using qualitative and quantitative data, the results show an increase in communicative behaviour, for both verbal and non-verbal participants, resulting from the use of CymaSense. Qualitative feedback from interviews provided insight into the factors that contribute to the successful use of the application, as well as aspects that could be improved.

Measuring the Behavioral Response to Spatial Audio within a Multi-Modal Virtual Reality Environment in Children with Autism Spectrum Disorder

Virtual Reality (VR) has been an active area of research in the development of interactive interventions for individuals with autism spectrum disorder (ASD) for over two decades. These immersive environments create a safe platform in which therapy can address the core symptoms associated with this condition. Recent advancements in spatial audio rendering techniques for VR now allow for the creation of realistic audio environments that accurately match their visual counterparts. However, reported auditory processing impairments associated with autism may affect how an individual interacts with their virtual therapy application. This study aims to investigate if these difficulties in processing audio information would directly impact how individuals with autism interact with a presented virtual spatial audio environment. Two experiments were conducted with participants diagnosed with ASD (n = 29) that compared: (1) behavioral reaction between spatialized and non-spatialized audio; and (2) the effect of background noise on participant interaction. Participants listening to binaural-based spatial audio showed higher spatial attention towards target auditory events. In addition, the amount of competing background audio was reported to influence spatial attention and interaction. These findings suggest that despite associated sensory processing difficulties, those with ASD can correctly decode the auditory cues simulated in current spatial audio rendering techniques.