Machine Learning to Study Social Interaction Difficulties in ASD

Sublime and extended reality experiences to enhance emotional wellbeing for autistic people: A state of the art review and narrative synthesis

BACKGROUND

Sublime is a centuries old concept of emergent experience arising from immense and threatening awareness provoked by overwhelming fear and dread when faced with an incomprehensible situation as is common to autistic people. Extended Reality (XR) technologies have been used since the mid-1990s, in regulating emotions, behaviour and supporting social skill development for autistic people.

AIMS

To understand utility of XR technologies in creating immersive experiences for autistic people to alleviate anxiety and the relationship to the sublime.

METHOD

A State of the Art literature review and narrative synthesis was conducted. PubMed, CINAHL, EMBASE, Cochrane Library, Scopus, Web of Science were searched with terms Autism AND Technology. In addition, fields of digital technologies and wellbeing, digital art and mental health, generative arts and the sublime were explored through web searches of grey literature, conversations with digital designers and explorations of extended reality platforms. No time limits were placed. Searches were done in English. Papers were screened and shortlisted using the inclusion criteria applied by two reviewers.

RESULTS

Fifty-eight papers/articles met the preliminary inclusion criteria for in-depth review of which 31 were found suitable for the narrative synthesis related to XR technologies and sublime experiences as related to autistic people. Narrative synthesis lent itself to four themes that is current utility of XR Technologies in autism, the impact of immersive experiences on Behavioural, phenomenological and biological markers of autistic people, the Benefits of increased sensory stimulation using XR on autism and an inquiry into the potential of the sublime for autism.

CONCLUSIONS

Mixed reality environments that experiment with a broad range of XR technologies including incorporating notions of the sublime, might be beneficial in reducing emotional dysregulation and improving social development in autistic people especially if co-designed with them.

Classifying autism in a clinical population based on motion synchrony: a proof-of-concept study using real-life diagnostic interviews

Predictive modeling strategies are increasingly studied as a means to overcome clinical bottlenecks in the diagnostic classification of autism spectrum disorder. However, while some findings are promising in the light of diagnostic marker research, many of these approaches lack the scalability for adequate and effective translation to everyday clinical practice. In this study, our aim was to explore the use of objective computer vision video analysis of real-world autism diagnostic interviews in a clinical sample of children and young individuals in the transition to adulthood to predict diagnosis. Specifically, we trained a support vector machine learning model on interpersonal synchrony data recorded in Autism Diagnostic Observation Schedule (ADOS-2) interviews of patient-clinician dyads. Our model was able to classify dyads involving an autistic patient (n = 56) with a balanced accuracy of 63.4% against dyads including a patient with other psychiatric diagnoses (n = 38). Further analyses revealed no significant associations between our classification metrics with clinical ratings. We argue that, given the above-chance performance of our classifier in a highly heterogeneous sample both in age and diagnosis, with few adjustments this highly scalable approach presents a viable route for future diagnostic marker research in autism.

Assisted Robots in Therapies for Children with Autism in Early Childhood

Children with autism spectrum disorder (ASD) have deficits that affect their social relationships, communication, and flexibility in reasoning. There are different types of treatment (pharmacological, educational, psychological, and rehabilitative). Currently, one way to address this problem is by using robotic systems to address the abilities that are altered in these children. The aim of this review will be to analyse the effectiveness of the incorporation of the different robotic systems currently existing in the treatment of children up to 10 years of age diagnosed with autism. A systematic review has been carried out in the PubMed, Scopus, Web of Science, and Dialnet databases, with the following descriptors: child, autism, and robot. The search yielded 578 papers, and nine were selected after the application of the PRISMA guideline. The quality of the studies was analysed with the PEDRo scale, and only those with a score between four and six were selected. From this study, the conclusion is that the use of robots, in general, improves children's behaviour in the short term, but longer-term experiences are necessary to achieve more conclusive results.

Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions

Autism spectrum disorder is characterized by impaired social communication and interaction. As a neurodevelopmental disorder typically diagnosed during childhood, diagnosis in adulthood is preceded by a resource-heavy clinical assessment period. The ongoing developments in digital phenotyping give rise to novel opportunities within the screening and diagnostic process. Our aim was to quantify multiple non-verbal social interaction characteristics in autism and build diagnostic classification models independent of clinical ratings. We analyzed videos of naturalistic social interactions in a sample including 28 autistic and 60 non-autistic adults paired in dyads and engaging in two conversational tasks. We used existing open-source computer vision algorithms for objective annotation to extract information based on the synchrony of movement and facial expression. These were subsequently used as features in a support vector machine learning model to predict whether an individual was part of an autistic or non-autistic interaction dyad. The two prediction models based on reciprocal adaptation in facial movements, as well as individual amounts of head and body motion and facial expressiveness showed the highest precision (balanced accuracies: 79.5% and 68.8%, respectively), followed by models based on reciprocal coordination of head (balanced accuracy: 62.1%) and body (balanced accuracy: 56.7%) motion, as well as intrapersonal coordination processes (balanced accuracy: 44.2%). Combinations of these models did not increase overall predictive performance. Our work highlights the distinctive nature of non-verbal behavior in autism and its utility for digital phenotyping-based classification. Future research needs to both explore the performance of different prediction algorithms to reveal underlying mechanisms and interactions, as well as investigate the prospective generalizability and robustness of these algorithms in routine clinical care.

Automated extraction of speech and turn-taking parameters in autism allows for diagnostic classification using a multivariable prediction model

Autism spectrum disorder (ASD) is diagnosed on the basis of speech and communication differences, amongst other symptoms. Since conversations are essential for building connections with others, it is important to understand the exact nature of differences between autistic and non-autistic verbal behaviour and evaluate the potential of these differences for diagnostics. In this study, we recorded dyadic conversations and used automated extraction of speech and interactional turn-taking features of 54 non-autistic and 26 autistic participants. The extracted speech and turn-taking parameters showed high potential as a diagnostic marker. A linear support vector machine was able to predict the dyad type with 76.2% balanced accuracy (sensitivity: 73.8%, specificity: 78.6%), suggesting that digitally assisted diagnostics could significantly enhance the current clinical diagnostic process due to their objectivity and scalability. In group comparisons on the individual and dyadic level, we found that autistic interaction partners talked slower and in a more monotonous manner than non-autistic interaction partners and that mixed dyads consisting of an autistic and a non-autistic participant had increased periods of silence, and the intensity, i.e. loudness, of their speech was more synchronous.

Classifying interpersonal synchronization states using a data-driven approach: implications for social interaction understanding

This study presents a data-driven approach to identifying interpersonal motor synchrony states by analyzing hand movements captured from a 3D depth camera. Utilizing a single frame from the experiment, an XGBoost machine learning model was employed to differentiate between spontaneous and intentional synchrony modes with nearly [Formula: see text] accuracy. Our findings demonstrate a consistent pattern across subjects, revealing that movement velocity tends to be slower in synchrony modes. These insights support the notion that the relationship between velocity and synchrony is influenced by the cognitive load required for the task, with slower movements leading to higher synchrony in tasks demanding higher cognitive load. This work not only contributes to the limited literature on algorithms for identifying interpersonal synchrony but also has potential implications for developing new metrics to assess real-time human social interactions, understanding social interaction, and diagnosing and developing treatment strategies for social deficits associated with conditions such as Autism Spectrum Disorder.

Predicting Autism from Head Movement Patterns during Naturalistic Social Interactions

Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized in part by difficulties in verbal and nonverbal social communication. Evidence indicates that autistic people, compared to neurotypical peers, exhibit differences in head movements, a key form of nonverbal communication. Despite the crucial role of head movements in social communication, research on this nonverbal cue is relatively scarce compared to other forms of nonverbal communication, such as facial expressions and gestures. There is a need for scalable, reliable, and accurate instruments for measuring head movements directly within the context of social interactions. In this study, we used computer vision and machine learning to examine the head movement patterns of neurotypical and autistic individuals during naturalistic, face-to-face conversations, at both the individual (monadic) and interpersonal (dyadic) levels. Our model predicts diagnostic status using dyadic head movement data with an accuracy of 80%, highlighting the value of head movement as a marker of social communication. The monadic data pipeline had lower accuracy (69.2%) compared to the dyadic approach, emphasizing the importance of studying back-and-forth social communication cues within a true social context. The proposed classifier is not intended for diagnostic purposes, and future research should replicate our findings in larger, more representative samples.

Digitally assisted diagnostics of autism spectrum disorder

Digital technologies have the potential to support psychiatric diagnostics and, in particular, differential diagnostics of autism spectrum disorder in the near future, making clinical decisions more objective, reliable and evidence-based while reducing clinical resources. Multimodal automatized measurement of symptoms at cognitive, behavioral, and neuronal levels combined with artificial intelligence applications offer promising strides toward personalized prognostics and treatment strategies. In addition, these new technologies could enable systematic and continuous assessment of longitudinal symptom development, beyond the usual scope of clinical practice. Early recognition of exacerbation and simplified, as well as detailed, progression control would become possible. Ultimately, digitally assisted diagnostics will advance early recognition. Nonetheless, digital technologies cannot and should not substitute clinical decision making that takes the comprehensive complexity of individual longitudinal and cross-section presentation of autism spectrum disorder into account. Yet, they might aid the clinician by objectifying decision processes and provide a welcome relief to resources in the clinical setting.

Machine Learning-Based Blood RNA Signature for Diagnosis of Autism Spectrum Disorder

Early diagnosis of autism spectrum disorder (ASD) is crucial for providing appropriate treatments and parental guidance from an early age. Yet, ASD diagnosis is a lengthy process, in part due to the lack of reliable biomarkers. We recently applied RNA-sequencing of peripheral blood samples from 73 American and Israeli children with ASD and 26 neurotypically developing (NT) children to identify 10 genes with dysregulated blood expression levels in children with ASD. Machine learning (ML) analyzes data by computerized analytical model building and may be applied to building diagnostic tools based on the optimization of large datasets. Here, we present several ML-generated models, based on RNA expression datasets collected during our recently published RNA-seq study, as tentative tools for ASD diagnosis. Using the random forest classifier, two of our proposed models yield an accuracy of 82% in distinguishing children with ASD and NT children. Our proof-of-concept study requires refinement and independent validation by studies with far larger cohorts of children with ASD and NT children and should thus be perceived as starting point for building more accurate ML-based tools. Eventually, such tools may potentially provide an unbiased means to support the early diagnosis of ASD.

Autism screening: an unsupervised machine learning approach

Early screening of autism spectrum disorders (ASD) is a key area of research in healthcare. Currently artificial intelligence (AI)-driven approaches are used to improve the process of autism diagnosis using computer-aided diagnosis (CAD) systems. One of the issues related to autism diagnosis and screening data is the reliance of the predictions primarily on scores provided by medical screening methods which can be biased depending on how the scores are calculated. We attempt to reduce this bias by assessing the performance of the predictions related to the screening process using a new model that consists of a Self-Organizing Map (SOM) with classification algorithms. The SOM is employed prior to the diagnostic process to derive a new class label using clusters learnt from the independent features; these clusters are related to communication, repetitive traits, and social traits in the input dataset. Then, the new clusters are compared with existing class labels in the dataset to refine and eliminate any inconsistencies. Lastly, the refined dataset is utilised to derive classification systems for autism diagnosis. The new model was evaluated against a real-life autism screening dataset that consists of over 2000 instances of cases and controls. The results based on the refined dataset show that the proposed method achieves significantly higher accuracy, precision, and recall for the classification models derived when compared to models derived from the original dataset.

Head Movement Patterns during Face-to-Face Conversations Vary with Age

Advances in computational behavior analysis have the potential to increase our understanding of behavioral patterns and developmental trajectories in neurotypical individuals, as well as in individuals with mental health conditions marked by motor, social, and emotional difficulties. This study focuses on investigating how head movement patterns during face-to-face conversations vary with age from childhood through adulthood. We rely on computer vision techniques due to their suitability for analysis of social behaviors in naturalistic settings, since video data capture can be unobtrusively embedded within conversations between two social partners. The methods in this work include unsupervised learning for movement pattern clustering, and supervised classification and regression as a function of age. The results demonstrate that 3-minute video recordings of head movements during conversations show patterns that distinguish between participants that are younger vs. older than 12 years with 78% accuracy. Additionally, we extract relevant patterns of head movement upon which the age distinction was determined by our models.

Large multicenter randomized trials in autism: key insights gained from the balovaptan clinical development program

BACKGROUND

Autism spectrum disorder (ASD) is a common and heterogeneous neurodevelopmental condition that is characterized by the core symptoms of social communication difficulties and restricted and repetitive behaviors. At present, there is an unmet medical need for therapies to ameliorate these core symptoms in order to improve quality of life of autistic individuals. However, several challenges are currently faced by the ASD community relating to the development of pharmacotherapies, namely in the conduct of clinical trials. Balovaptan is a V1a receptor antagonist that has been investigated to improve social communication difficulties in individuals with ASD. In this viewpoint, we draw upon our recent first-hand experiences of the balovaptan clinical development program to describe current challenges of ASD trials.

DISCUSSION POINTS

The balovaptan trials were conducted in a wide age range of individuals with ASD with the added complexities associated with international trials. When summarizing all three randomized trials of balovaptan, a placebo response was observed across several outcome measures. Placebo response was predicted by greater baseline symptom severity, online recruitment of participants, and less experienced or non-academic trial sites. We also highlight challenges relating to selection of outcome measures in ASD, the impact of baseline characteristics, and the role of expectation bias in influencing trial results.

CONCLUSION

Taken together, the balovaptan clinical development program has advanced our understanding of the key challenges facing ASD treatment research. The insights gained can be used to inform and improve the design of future clinical trials with the collective aim of developing efficacious therapies to support individuals with ASD.

A Machine Learning Approach in Autism Spectrum Disorders: From Sensory Processing to Behavior Problems

Atypical sensory processing described in autism spectrum disorders (ASDs) frequently cascade into behavioral alterations: isolation, aggression, indifference, anxious/depressed states, or attention problems. Predictive machine learning models might refine the statistical explorations of the associations between them by finding out how these dimensions are related. This study investigates whether behavior problems can be predicted using sensory processing abilities. Participants were 72 children and adolescents (21 females) diagnosed with ASD, aged between 6 and 14 years (M = 7.83 years; SD = 2.80 years). Parents of the participants were invited to answer the Sensory Profile 2 (SP2) and the Child Behavior Checklist (CBCL) questionnaires. A collection of 26 supervised machine learning regression models of different families was developed to predict the CBCL outcomes using the SP2 scores. The most reliable predictions were for the following outcomes: total problems (using the items in the SP2 touch scale as inputs), anxiety/depression (using avoiding quadrant), social problems (registration), and externalizing scales, revealing interesting relations between CBCL outcomes and SP2 scales. The prediction reliability on the remaining outcomes was “moderate to good” except somatic complaints and rule-breaking, where it was “bad to moderate.” Linear and ridge regression achieved the best prediction for a single outcome and globally, respectively, and gradient boosting machine achieved the best prediction in three outcomes. Results highlight the utility of several machine learning models in studying the predictive value of sensory processing impairments (with an early onset) on specific behavior alterations, providing evidences of relationship between sensory processing impairments and behavior problems in ASD.

A Virtual Reality Based System for the Screening and Classification of Autism

Autism - also known as Autism Spectrum Disorders or Autism Spectrum Conditions - is a neurodevelopmental condition characterized by repetitive behaviours and differences in communication and social interaction. As a consequence, many autistic individuals may struggle in everyday life, which sometimes manifests in depression, unemployment, or addiction. One crucial problem in patient support and treatment is the long waiting time to diagnosis, which was approximated to thirteen months on average. Yet, the earlier an intervention can take place the better the patient can be supported, which was identified as a crucial factor. We propose a system to support the screening of Autism Spectrum Disorders based on a virtual reality social interaction, namely a shopping experience, with an embodied agent. During this everyday interaction, behavioral responses are tracked and recorded. We analyze this behavior with machine learning approaches to classify participants from an autistic participant sample in comparison to a typically developed individuals control sample with high accuracy, demonstrating the feasibility of the approach. We believe that such tools can strongly impact the way mental disorders are assessed and may help to further find objective criteria and categorization.

Towards Robot-Assisted Therapy for Children With Autism—The Ontological Knowledge Models and Reinforcement Learning-Based Algorithms

Robots are more and more present in our lives, particularly in the health sector. In therapeutic centers, some therapists are beginning to explore various tools like video games, Internet exchanges, and robot-assisted therapy. These tools will be at the disposal of these professionals as additional resources that can support them to assist their patients intuitively and remotely. The humanoid robot can capture young children’s attention and then attract the attention of researchers. It can be considered as a play partner and can directly interact with children or without a third party’s presence. It can equally perform repetitive tasks that humans cannot achieve in the same way. Moreover, humanoid robots can assist a therapist by allowing him to teleoperated and interact from a distance. In this context, our research focuses on robot-assisted therapy and introduces a humanoid social robot in a pediatric hospital care unit. That will be performed by analyzing many aspects of the child’s behavior, such as verbal interactions, gestures and facial expressions, etc. Consequently, the robot can reproduce consistent experiences and actions for children with communication capacity restrictions. This work is done by applying a novel approach based on deep learning and reinforcement learning algorithms supported by an ontological knowledge base that contains relevant information and knowledge about patients, screening tests, and therapies. In this study, we realized a humanoid robot that will assist a therapist by equipping the robot NAO: 1) to detect whether a child is autistic or not using a convolutional neural network, 2) to recommend a set of therapies based on a selection algorithm using a correspondence matrix between screening test and therapies, and 2) to assist and monitor autistic children by executing tasks that require those therapies.

Annual Research Review: Translational machine learning for child and adolescent psychiatry

Children and adolescents could benefit from the use of predictive tools that facilitate personalized diagnoses, prognoses, and treatment selection. Such tools have not yet been deployed using traditional statistical methods, potentially due to the limitations of the paradigm and the need to leverage large amounts of digital data. This review will suggest that a machine learning approach could address these challenges and is designed to introduce new readers to the background, methods, and results in the field. A rationale is first introduced followed by an outline of fundamental elements of machine learning approaches. To provide an overview of the use of the techniques in child and adolescent literature, a scoping review of broad trends is then presented. Selected studies are also highlighted in order to draw attention to research areas that are closest to translation and studies that exhibit a high degree of experimental innovation. Limitations to the research, and machine learning approaches generally, are outlined in the penultimate section highlighting issues related to sample sizes, validation, clinical utility, and ethical challenges. Finally, future directions are discussed that could enhance the possibility of clinical implementation and address specific questions relevant to the child and adolescent psychiatry. The review gives a broad overview of the machine learning paradigm in order to highlight the benefits of a shift in perspective towards practically oriented statistical solutions that aim to improve clinical care of children and adolescents.

EYE-C: Eye-Contact Robust Detection and Analysis during Unconstrained Child-Therapist Interactions in the Clinical Setting of Autism Spectrum Disorders

The high level of heterogeneity in Autism Spectrum Disorder (ASD) and the lack of systematic measurements complicate predicting outcomes of early intervention and the identification of better-tailored treatment programs. Computational phenotyping may assist therapists in monitoring child behavior through quantitative measures and personalizing the intervention based on individual characteristics; still, real-world behavioral analysis is an ongoing challenge. For this purpose, we designed EYE-C, a system based on OpenPose and Gaze360 for fine-grained analysis of eye-contact episodes in unconstrained therapist-child interactions via a single video camera. The model was validated on video data varying in resolution and setting, achieving promising performance. We further tested EYE-C on a clinical sample of 62 preschoolers with ASD for spectrum stratification based on eye-contact features and age. By unsupervised clustering, three distinct sub-groups were identified, differentiated by eye-contact dynamics and a specific clinical phenotype. Overall, this study highlights the potential of Artificial Intelligence in categorizing atypical behavior and providing translational solutions that might assist clinical practice.

Brief Report: Specificity of Interpersonal Synchrony Deficits to Autism Spectrum Disorder and Its Potential for Digitally Assisted Diagnostics

Reliably diagnosing autism spectrum disorders (ASD) in adulthood poses a challenge to clinicians due to the absence of specific diagnostic markers. This study investigated the potential of interpersonal synchrony (IPS), which has been found to be reduced in ASD, to augment the diagnostic process. IPS was objectively assessed in videos of diagnostic interviews in a representative referral population from two specialized autism outpatient clinics. In contrast to the current screening tools that could not reliably differentiate, we found a significant reduction of IPS in interactions with individuals later diagnosed with ASD (n = 16) as opposed to those not receiving a diagnosis (n = 23). While these findings need to be validated in larger samples, they nevertheless underline the potential of digitally-enhanced diagnostic processes for ASD.

Automating autism assessment: What AI can bring to the diagnostic process

This paper examines the use of artificial intelligence (AI) for the diagnosis of autism spectrum disorder (ASD, hereafter autism). In so doing we examine some problems in existing diagnostic processes and criteria, including issues of bias and interpretation, and on concepts like the 'double empathy problem'. We then consider how novel applications of AI might contribute to these contexts. We're focussed specifically on adult diagnostic procedures as childhood diagnosis is already well covered in the literature.

Children ASD Evaluation Through Joint Analysis of EEG and Eye-Tracking Recordings With Graph Convolution Network

Recent advances in neuroscience indicate that analysis of bio-signals such as rest state electroencephalogram (EEG) and eye-tracking data can provide more reliable evaluation of children autism spectrum disorder (ASD) than traditional methods of behavior measurement relying on scales do. However, the effectiveness of the new approaches still lags behind the increasing requirement in clinical or educational practices as the “bio-marker” information carried by the bio-signal of a single-modality is likely insufficient or distorted. This study proposes an approach to joint analysis of EEG and eye-tracking for children ASD evaluation. The approach focuses on deep fusion of the features in two modalities as no explicit correlations between the original bio-signals are available, which also limits the performance of existing methods along this direction. First, the synchronization measures, information entropy, and time-frequency features of the multi-channel EEG are derived. Then a random forest applies to the eye-tracking recordings of the same subjects to single out the most significant features. A graph convolutional network (GCN) model then naturally fuses the two group of features to differentiate the children with ASD from the typically developed (TD) subjects. Experiments have been carried out on the two types of the bio-signals collected from 42 children (21 ASD and 21 TD subjects, 3–6 years old). The results indicate that (1) the proposed approach can achieve an accuracy of 95% in ASD detection, and (2) strong correlations exist between the two bio-signals collected even asynchronously, in particular the EEG synchronization against the face related/joint attentions in terms of covariance.

INTRApersonal Synchrony as Constituent of INTERpersonal Synchrony and Its Relevance for Autism Spectrum Disorder

INTERpersonal synchrony leads to increased empathy, rapport and understanding, enabling successful human-human interactions and reciprocal bonding. Research shows that individuals with Autism Spectrum Disorder (ASD) exhibit difficulties to INTERpersonally synchronize but underlying causes are yet unknown. In order to successfully synchronize with others, INTRApersonal synchronization of communicative signals appears to be a necessary prerequisite. We understand INTRApersonal synchrony as an implicit factor of INTERpersonal synchrony and therefore hypothesize that atypicalities of INTRApersonal synchrony may add to INTERpersonal synchrony problems in ASD and their interaction partners. In this perspective article, we first review evidence for INTERpersonal dissynchrony in ASD, with respect to different approaches and assessment methods. Second, we draft a theoretical conceptualization of INTRApersonal dissynchrony in ASD based on a temporal model of human interaction. We will outline literature indicating INTRApersonal dissynchrony in ASD, therefore highlighting findings of atypical timing functions and findings from clinical and behavioral studies that indicate peculiar motion patterns and communicative signal production in ASD. Third, we hypothesize that findings from these domains suggest an assessment and investigation of temporal parameters of social behavior in individuals with ASD. We will further propose specific goals of empirical approaches on INTRApersonal dissynchrony. Finally we present implications of research on INTRApersonal timing in ASD for diagnostic and therapeutic purposes, what in our opinion warrants the increase of research efforts in this domain.