A Mobile Game Platform for Improving Social Communication in Children with Autism: A Feasibility Study

The wide world of technological telerehabilitation for pediatric neurologic and neurodevelopmental disorders - a systematic review

Introduction

The use of Information and Communication Technology (ICT) for assessing and treating cognitive and motor disorders is promoting home-based telerehabilitation. This approach involves ongoing monitoring within a motivating context to help patients generalize their skills. It can also reduce healthcare costs and geographic barriers by minimizing hospitalization. This systematic review focuses on investigating key aspects of telerehabilitation protocols for children with neurodevelopmental or neurological disorders, including technology used, outcomes, caregiver involvement, and dosage, to guide clinical practice and future research.

Method

This systematic review adhered to PRISMA guidelines and was registered in PROSPERO. The PICO framework was followed to define the search strategy for technology-based telerehabilitation interventions targeting the pediatric population (aged 0-18) with neurological or neurodevelopmental disorders. The search encompassed Medline/PubMed, EMBASE, and Web of Science databases. Independent reviewers were responsible for selecting relevant papers and extracting data, while data harmonization and analysis were conducted centrally.

Results

A heterogeneous and evolving situation emerged from our data. Our findings reported that most of the technologies adopted for telerehabilitation are commercial devices; however, research prototypes and clinical software were also employed with a high potential for personalization and treatment efficacy. The efficacy of these protocols on health or health-related domains was also explored by categorizing the outcome measures according to the International Classification of Functioning, Disability, and Health (ICF). Most studies targeted motor and neuropsychological functions, while only a minority of papers explored language or multi-domain protocols. Finally, although caregivers were rarely the direct target of intervention, their role was diffusely highlighted as a critical element of the home-based rehabilitation setting.

Discussion

This systematic review offers insights into the integration of technological devices into telerehabilitation programs for pediatric neurologic and neurodevelopmental disorders. It highlights factors contributing to the effectiveness of these interventions and suggests the need for further development, particularly in creating dynamic and multi-domain rehabilitation protocols. Additionally, it emphasizes the importance of promoting home-based and family-centered care, which could involve caregivers more actively in the treatment, potentially leading to improved clinical outcomes for children with neurological or neurodevelopmental conditions.

Systematic review registration

PROSPERO (CRD42020210663).

A Perspective on Crowdsourcing and Human-in-the-Loop Workflows in Precision Health

Modern machine learning approaches have led to performant diagnostic models for a variety of health conditions. Several machine learning approaches, such as decision trees and deep neural networks, can, in principle, approximate any function. However, this power can be considered to be both a gift and a curse, as the propensity toward overfitting is magnified when the input data are heterogeneous and high dimensional and the output class is highly nonlinear. This issue can especially plague diagnostic systems that predict behavioral and psychiatric conditions that are diagnosed with subjective criteria. An emerging solution to this issue is crowdsourcing, where crowd workers are paid to annotate complex behavioral features in return for monetary compensation or a gamified experience. These labels can then be used to derive a diagnosis, either directly or by using the labels as inputs to a diagnostic machine learning model. This viewpoint describes existing work in this emerging field and discusses ongoing challenges and opportunities with crowd-powered diagnostic systems, a nascent field of study. With the correct considerations, the addition of crowdsourcing to human-in-the-loop machine learning workflows for the prediction of complex and nuanced health conditions can accelerate screening, diagnostics, and ultimately access to care.

Digitally Diagnosing Multiple Developmental Delays Using Crowdsourcing Fused With Machine Learning: Protocol for a Human-in-the-Loop Machine Learning Study

BACKGROUND

A considerable number of minors in the United States are diagnosed with developmental or psychiatric conditions, potentially influenced by underdiagnosis factors such as cost, distance, and clinician availability. Despite the potential of digital phenotyping tools with machine learning (ML) approaches to expedite diagnoses and enhance diagnostic services for pediatric psychiatric conditions, existing methods face limitations because they use a limited set of social features for prediction tasks and focus on a single binary prediction, resulting in uncertain accuracies.

OBJECTIVE

This study aims to propose the development of a gamified web system for data collection, followed by a fusion of novel crowdsourcing algorithms with ML behavioral feature extraction approaches to simultaneously predict diagnoses of autism spectrum disorder and attention-deficit/hyperactivity disorder in a precise and specific manner.

METHODS

The proposed pipeline will consist of (1) gamified web applications to curate videos of social interactions adaptively based on the needs of the diagnostic system, (2) behavioral feature extraction techniques consisting of automated ML methods and novel crowdsourcing algorithms, and (3) the development of ML models that classify several conditions simultaneously and that adaptively request additional information based on uncertainties about the data.

RESULTS

A preliminary version of the web interface has been implemented, and a prior feature selection method has highlighted a core set of behavioral features that can be targeted through the proposed gamified approach.

CONCLUSIONS

The prospect for high reward stems from the possibility of creating the first artificial intelligence-powered tool that can identify complex social behaviors well enough to distinguish conditions with nuanced differentiators such as autism spectrum disorder and attention-deficit/hyperactivity disorder.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/52205.

Personalized Deep Learning for Substance Use in Hawaii: Protocol for a Passive Sensing and Ecological Momentary Assessment Study

BACKGROUND

Artificial intelligence (AI)-powered digital therapies that detect methamphetamine cravings via consumer devices have the potential to reduce health care disparities by providing remote and accessible care solutions to communities with limited care solutions, such as Native Hawaiian, Filipino, and Pacific Islander communities. However, Native Hawaiian, Filipino, and Pacific Islander communities are understudied with respect to digital therapeutics and AI health sensing despite using technology at the same rates as other racial groups.

OBJECTIVE

In this study, we aimed to understand the feasibility of continuous remote digital monitoring and ecological momentary assessments in Native Hawaiian, Filipino, and Pacific Islander communities in Hawaii by curating a novel data set of longitudinal Fitbit (Fitbit Inc) biosignals with the corresponding craving and substance use labels. We also aimed to develop personalized AI models that predict methamphetamine craving events in real time using wearable sensor data.

METHODS

We will develop personalized AI and machine learning models for methamphetamine use and craving prediction in 40 individuals from Native Hawaiian, Filipino, and Pacific Islander communities by curating a novel data set of real-time Fitbit biosensor readings and the corresponding participant annotations (ie, raw self-reported substance use data) of their methamphetamine use and cravings. In the process of collecting this data set, we will gain insights into cultural and other human factors that can challenge the proper acquisition of precise annotations. With the resulting data set, we will use self-supervised learning AI approaches, which are a new family of machine learning methods that allows a neural network to be trained without labels by being optimized to make predictions about the data. The inputs to the proposed AI models are Fitbit biosensor readings, and the outputs are predictions of methamphetamine use or craving. This paradigm is gaining increased attention in AI for health care.

RESULTS

To date, more than 40 individuals have expressed interest in participating in the study, and we have successfully recruited our first 5 participants with minimal logistical challenges and proper compliance. Several logistical challenges that the research team has encountered so far and the related implications are discussed.

CONCLUSIONS

We expect to develop models that significantly outperform traditional supervised methods by finetuning according to the data of a participant. Such methods will enable AI solutions that work with the limited data available from Native Hawaiian, Filipino, and Pacific Islander populations and that are inherently unbiased owing to their personalized nature. Such models can support future AI-powered digital therapeutics for substance abuse.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/46493.

Personalized Machine Learning using Passive Sensing and Ecological Momentary Assessments for Meth Users in Hawaii: A Research Protocol

Background

Artificial intelligence (AI)-powered digital therapies which detect meth cravings delivered on consumer devices have the potential to reduce these disparities by providing remote and accessible care solutions to Native Hawaiians, Filipinos, and Pacific Islanders (NHFPI) communities with limited care solutions. However, NHFPI are fully understudied with respect to digital therapeutics and AI health sensing despite using technology at the same rates as other races.

Objective

We seek to fulfill two research aims: (1) Understand the feasibility of continuous remote digital monitoring and ecological momentary assessments (EMAs) in NHFPI in Hawaii by curating a novel dataset of longitudinal FitBit biosignals with corresponding craving and substance use labels. (2) Develop personalized AI models which predict meth craving events in real time using wearable sensor data.

Methods

We will develop personalized AI/ML (artificial intelligence/machine learning) models for meth use and craving prediction in 40 NHFPI individuals by curating a novel dataset of real-time FitBit biosensor readings and corresponding participant annotations (i.e., raw self-reported substance use data) of their meth use and cravings. In the process of collecting this dataset, we will glean insights about cultural and other human factors which can challenge the proper acquisition of precise annotations. With the resulting dataset, we will employ self-supervised learning (SSL) AI approaches, which are a new family of ML methods that allow a neural network to be trained without labels by being optimized to make predictions about the data itself. The inputs to the proposed AI models are FitBit biosensor readings and the outputs are predictions of meth use or craving. This paradigm is gaining increased attention in AI for healthcare.

Conclusions

We expect to develop models which significantly outperform traditional supervised methods by fine-tuning to an individual subject's data. Such methods will enable AI solutions which work with the limited data available from NHFPI populations and which are inherently unbiased due to their personalized nature. Such models can support future AI-powered digital therapeutics for substance abuse.

A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autism

Autism spectrum disorder (autism) is a neurodevelopmental delay that affects at least 1 in 44 children. Like many neurological disorder phenotypes, the diagnostic features are observable, can be tracked over time, and can be managed or even eliminated through proper therapy and treatments. However, there are major bottlenecks in the diagnostic, therapeutic, and longitudinal tracking pipelines for autism and related neurodevelopmental delays, creating an opportunity for novel data science solutions to augment and transform existing workflows and provide increased access to services for affected families. Several efforts previously conducted by a multitude of research labs have spawned great progress toward improved digital diagnostics and digital therapies for children with autism. We review the literature on digital health methods for autism behavior quantification and beneficial therapies using data science. We describe both case-control studies and classification systems for digital phenotyping. We then discuss digital diagnostics and therapeutics that integrate machine learning models of autism-related behaviors, including the factors that must be addressed for translational use. Finally, we describe ongoing challenges and potential opportunities for the field of autism data science. Given the heterogeneous nature of autism and the complexities of the relevant behaviors, this review contains insights that are relevant to neurological behavior analysis and digital psychiatry more broadly.

A System of Emotion Recognition and Judgment and Its Application in Adaptive Interactive Game

A system of emotion recognition and judgment (SERJ) based on a set of optimal signal features is established, and an emotion adaptive interactive game (EAIG) is designed. The change in a player's emotion can be detected with the SERJ during the process of playing the game. A total of 10 subjects were selected to test the EAIG and SERJ. The results show that the SERJ and designed EAIG are effective. The game adapted itself by judging the corresponding special events triggered by a player's emotion and, as a result, enhanced the player's game experience. It was found that, in the process of playing the game, a player's perception of the change in emotion was different, and the test experience of a player had an effect on the test results. A SERJ that is based on a set of optimal signal features is better than a SERJ that is based on the conventional machine learning-based method.

Digitally Diagnosing Multiple Developmental Delays using Crowdsourcing fused with Machine Learning: A Research Protocol

Background

Roughly 17% percent of minors in the United States aged 3 through 17 years have a diagnosis of one or more developmental or psychiatric conditions, with the true prevalence likely being higher due to underdiagnosis in rural areas and for minority populations. Unfortunately, timely diagnostic services are inaccessible to a large portion of the United States and global population due to cost, distance, and clinician availability. Digital phenotyping tools have the potential to shorten the time-to-diagnosis and to bring diagnostic services to more people by enabling accessible evaluations. While automated machine learning (ML) approaches for detection of pediatric psychiatry conditions have garnered increased research attention in recent years, existing approaches use a limited set of social features for the prediction task and focus on a single binary prediction.

Objective

I propose the development of a gamified web system for data collection followed by a fusion of novel crowdsourcing algorithms with machine learning behavioral feature extraction approaches to simultaneously predict diagnoses of Autism Spectrum Disorder (ASD) and Attention-Deficit/Hyperactivity Disorder (ADHD) in a precise and specific manner.

Methods

The proposed pipeline will consist of: (1) a gamified web applications to curate videos of social interactions adaptively based on needs of the diagnostic system, (2) behavioral feature extraction techniques consisting of automated ML methods and novel crowdsourcing algorithms, and (3) development of ML models which classify several conditions simultaneously and which adaptively request additional information based on uncertainties about the data.

Conclusions

The prospective for high reward stems from the possibility of creating the first AI-powered tool which can identify complex social behaviors well enough to distinguish conditions with nuanced differentiators such as ASD and ADHD.

Technologies to support the diagnosis and/or treatment of neurodevelopmental disorders: A systematic review

In recent years, there has been a great interest in utilizing technology in mental health research. The rapid technological development has encouraged researchers to apply technology as a part of a diagnostic process or treatment of Neurodevelopmental Disorders (NDDs). With the large number of studies being published comes an urgent need to inform clinicians and researchers about the latest advances in this field. Here, we methodically explore and summarize findings from studies published between August 2019 and February 2022. A search strategy led to the identification of 4108 records from PubMed and APA PsycInfo databases. 221 quantitative studies were included, covering a wide range of technologies used for diagnosis and/or treatment of NDDs, with the biggest focus on Autism Spectrum Disorder (ASD). The most popular technologies included machine learning, functional magnetic resonance imaging, electroencephalogram, magnetic resonance imaging, and neurofeedback. The results of the review indicate that technology-based diagnosis and intervention for NDD population is promising. However, given a high risk of bias of many studies, more high-quality research is needed.

Enhancing Resilience in Family Caregivers Using an mHealth App

BACKGROUND

We previously developed a mobile health (mHealth) app (Roadmap) to promote the resilience of family caregivers during the acute phases of care in patients undergoing hematopoietic cell transplantation (HCT).

OBJECTIVE

This study explored users' perspectives on the uptake of Roadmap's multicomponent features and the app's utility in promoting resilience.

METHODS

Fifteen participants were randomized to the full version of the app that included resilience-building activities and the other 15 were randomized to the control version that included a limited view of the app (i.e., without any resilience-building activities). They were instructed to use the app for 120 days. Semistructured qualitative interviews were then conducted with users as part of an ongoing, larger Roadmap study (NCT04094844). During the interview, caregiver participants were asked about their overall experiences with the app, frequency of use, features used, facilitators of and barriers to use, and their perspectives on its utility in promoting resilience. Data were professionally transcribed, coded, and categorized through content analysis.

RESULTS

Interviews were conducted with 30 participants, which included 23 females and 7 males. The median age of the population was 58 years (range, 23-82). The four main themes that emerged included app use, ease of use, user experiences, and ability to foster resilience. The subthemes identified related to facilitators (convenience and not harmful), barriers (caregiver burden and being too overwhelmed during the acute phases of HCT care), resilience (optimism/positivity and self-care), and app design improvements (personalization and notifications/reminders).

CONCLUSION

The qualitative evaluation provided insights into which components were utilized and how one, or a combination of the multicomponent features, may be enhancing users' experiences. Lessons learned suggest that the Roadmap app contributed to promoting resilience during the acute phases of HCT care. Nonetheless, features that provided enhanced personalization may further improve longer-term engagement.

Sensory Integration Training and Social Sports Games Integrated Intervention for the Occupational Therapy of Children with Autism

This paper presents a research design for an integrated intervention using sensory integration training fused with social sports games for the treatment of children with autism. This study used a multiple baseline cross-subject design in a single-subject experiment, with structured play as the independent variable and expressive language skills of children with autism spectrum disorders as the dependent variable, with three phases of intervention: baseline, intervention period, and maintenance period. The expressive language ability was examined in terms of both oral expression and gestural expression, where the intervention effect of the oral expression was analyzed in terms of four components: the total number of words, the total number of sentences, average sentence length, and vocabulary complexity of oral expression, and the intervention effect of the gestural expression was analyzed in terms of changes in the frequency of children's gestural expression behaviors. For the categories classified by sensory integration ability, there are corresponding specific training programs that combine various physical exercises and play equipment to train the various abnormal functions of children with autism. Stereotyped behavior is a repetitive, self-imposed, and purposeless physical action, usually in the form of continuous and repetitive movements, sounds, and so on. 4 times a week, 25 minutes each time, the activity of recognizing pictures and familiar objects is carried out first, and then the children choose the structured game model and the initiative to build and take turns with the researchers to build. Stereotypic behaviors cause a great deal of distress in the lives of children with autism, and it is necessary to explore how to implement positive and effective interventions. Subjects' play abilities developed after receiving effective critical response training. The subjects' practice and symbolic play showed good immediate and maintenance intervention effectiveness; their associative and functional play showed no significant intervention effectiveness. The enhancement of the sensory integration skills of children with autism through sensory integration training resulted in a relative reduction of stereotypic behavior about the stimulus-seeking function, which had a positive effect on the intervention of stereotypic behavior.

Machine learning models using mobile game play accurately classify children with autism

Digitally-delivered healthcare is well suited to address current inequities in the delivery of care due to barriers of access to healthcare facilities. As the COVID-19 pandemic phases out, we have a unique opportunity to capitalize on the current familiarity with telemedicine approaches and continue to advocate for mainstream adoption of remote care delivery. In this paper, we specifically focus on the ability of GuessWhat? a smartphone-based charades-style gamified therapeutic intervention for autism spectrum disorder (ASD) to generate a signal that distinguishes children with ASD from neurotypical (NT) children. We demonstrate the feasibility of using "in-the-wild", naturalistic gameplay data to distinguish between ASD and NT by children by training a random forest classifier to discern the two classes (AU-ROC = 0.745, recall = 0.769). This performance demonstrates the potential for GuessWhat? to facilitate screening for ASD in historically difficult-to-reach communities. To further examine this potential, future work should expand the size of the training sample and interrogate differences in predictive ability by demographic.

Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods

BACKGROUND

Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process.

OBJECTIVE

In this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments.

METHODS

Here, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual's visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD.

RESULTS

Our results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD.

CONCLUSIONS

Ultimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.