Automatic Emotion Recognition in Children with Autism: A Systematic Literature Review

WS-BiTM: Integrating White Shark Optimization with Bi-LSTM for enhanced autism spectrum disorder diagnosis

Autism Spectrum Disorder (ASD) is a multifaceted neurodevelopmental condition marked by challenges in social communication, sensory processing, and behavioral regulation. The delayed diagnosis of ASD significantly impedes timely interventions, which can exacerbate symptom severity. With approximately 62 million individuals affected worldwide, the demand for efficient diagnostic tools is critical. This study introduces a novel framework that combines a White Shark Optimization (WSO)-based feature selection method with a Bidirectional Long Short-Term Memory (Bi-LSTM) classifier for enhanced autism classification. Utilizing the WSO technique, we identify key features from autism screening datasets, which markedly improves the model's predictive capabilities. The optimized feature set is then processed by the Bi-LSTM classifier, enhancing its efficiency in handling sequential data. We comprehensively address methodological challenges, including overfitting, generalization, interpretability, and computational efficiency. Furthermore, we conduct a comparative analysis against baseline algorithms such as Neural Networks, Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM) networks, while also employing Particle Swarm Optimization (PSO) for feature selection validation. We evaluate performance metrics, including accuracy, F1-score, specificity, precision, and sensitivity across three ASD datasets: Toddlers, Adults, and Children. Our results demonstrate that the WS-BiTM model significantly outperforms baseline methods, achieving accuracies of 97.6 %, 96.2 %, and 96.4 % on the respective datasets. Additionally, we implemented leave-one-dataset cross-validation and confirmed the statistical significance of our findings through a paired t-test, supplemented by an ablation study to detail the contributions of individual model components. These findings highlight the potential of the WS-BiTM model as a robust tool for ASD classification.

MCBERT: A multi-modal framework for the diagnosis of autism spectrum disorder

Within the domain of neurodevelopmental disorders, autism spectrum disorder (ASD) emerges as a distinctive neurological condition characterized by multifaceted challenges. The delayed identification of ASD poses a considerable hurdle in effectively managing its impact and mitigating its severity. Addressing these complexities requires a nuanced understanding of data modalities and the underlying patterns. Existing studies have focused on a single data modality for ASD diagnosis. Recently, there has been a significant shift towards multimodal architectures with deep learning strategies due to their ability to handle and incorporate complex data modalities. In this paper, we developed a novel multimodal ASD diagnosis architecture, referred to as Multi-Head CNN with BERT (MCBERT), which integrates bidirectional encoder representations from transformers (BERT) for meta-features and a multi-head convolutional neural network (MCNN) for the brain image modality. The MCNN incorporates two attention mechanisms to capture spatial (SAC) and channel (CAC) features. The outputs of BERT and MCNN are then fused and processed through a classification module to generate the final diagnosis. We employed the ABIDE-I dataset, a multimodal dataset, and conducted a leave-one-site-out classification to assess the model's effectiveness comprehensively. Experimental simulations demonstrate that the proposed architecture achieves a high accuracy of 93.4 %. Furthermore, the exploration of functional MRI data may provide a deeper understanding of the underlying characteristics of ASD.

AI technology to support adaptive functioning in neurodevelopmental conditions in everyday environments: a systematic review

Supports for adaptive functioning in individuals with neurodevelopmental conditions (NDCs) is of umost importance to long-term outcomes. Artificial intelligence (AI)-assistive technologies has enormous potential to offer efficient, cost-effective, and personalized solutions to address these challenges, particularly in everday environments. This systematic review examines the existing evidence for using AI-assistive technologies to support adaptive functioning in people with NDCs in everyday settings. Searches across six databases yielded 15 studies meeting inclusion criteria, focusing on robotics, phones/computers and virtual reality. Studies most frequently recruited children diagnosed with autism and targeted social skills (47%), daily living skills (26%), and communication (16%). Despite promising results, studies addressing broader transdiagnostic needs across different NDC populations are needed. There is also an urgent need to improve the quality of evidence-based research practices. This review concludes that AI holds enormous potential to support adaptive functioning for people with NDCs and for personalized health support. This review underscores the need for further research studies to advance AI technologies in this field.

Using facial reaction analysis and machine learning to objectively assess the taste of medicines in children

For orally administered drugs, palatability is key in ensuring patient acceptability and treatment compliance. Therefore, understanding children's taste sensitivity and preferences can support formulators in making paediatric medicines more acceptable. Presently, we explore if the application of computer-vision techniques to videos of children's reaction to gustatory taste strips can provide an objective assessment of palatability. Children aged 4 to 11 years old tasted four different flavoured strips: no taste, bitter, sweet, and sour. Data was collected at home, under the supervision of a guardian, with responses recorded using the Aparito Atom app and smartphone camera. Participants scored each strip on a 5-point hedonic scale. Facial landmarks were identified in the videos, and quantitative measures, such as changes around the eyes, nose, and mouth, were extracted to train models to classify strip taste and score. We received 197 videos and 256 self-reported scores from 64 participants. The hedonic scale elicited expected results: children like sweetness, dislike bitterness and have varying opinions for sourness. The findings revealed the complexity and variability of facial reactions and highlighted specific measures, such as eyebrow and mouth corner elevations, as significant indicators of palatability. This study capturing children's objective reactions to taste sensations holds promise in identifying palatable drug formulations and assessing patient acceptability of paediatric medicines. Moreover, collecting data in the home setting allows for natural behaviour, with minimal burden for participants.

Relationship of autistic traits and the severity of fear of the COVID-19 pandemic in the general population

Background

The aim of the study was to investigate the level of fear of the COVID-19 pandemic and to detect a possible correlation between the autistic traits and the level of fear and to learn about other factors that may affect the level of fear.

Methods

The study utilised a questionnaire and was conducted online in the period from 16.02.2021 to 11.06.2021. The test group consisted of 214 respondents with an average age of 23.78 years (95%CI: 22.48 - 25.08; max: 61, min: 14) from the general population. The study used The Autism-Spectrum Quotient (AQ) questionnaire to assess the degree of autistic traits in the general population and The Fear of COVID-19 Scale, which was used to assess the level of fear of COVID-19.

Results

Among the respondents, 9 people scored ≥32 on the AQ test and were considered to have a high degree of autistic traits. In multiple regression (R2 = 0.1, p<0.0001), a positive relationship between the severity of fear of COVID-19 and the autistic traits (p=0.01) and age (p<0.001) was obtained. Additionally, a second multiple regression (R2 = 0.1, p<0.000001) including the subscales of AQ was performed and a positive relationship between the severity of fear of COVID-19 and the difficulties in attention switching (p=0.0004) and age (p=0.00001) was obtained.

Conclusion

People with higher autistic traits present greater fear of the COVID-19 pandemic. We suggest that it might be caused by cognitive stiffness and disorders in emotions regulation, according to the literature. The elderly also present higher levels of fear. The other variables did not affect the level of fear of the COVID-19 pandemic.

In-the-Wild Affect Analysis of Children with ASD Using Heart Rate

Recognizing the affective state of children with autism spectrum disorder (ASD) in real-world settings poses challenges due to the varying head poses, illumination levels, occlusion and a lack of datasets annotated with emotions in in-the-wild scenarios. Understanding the emotional state of children with ASD is crucial for providing personalized interventions and support. Existing methods often rely on controlled lab environments, limiting their applicability to real-world scenarios. Hence, a framework that enables the recognition of affective states in children with ASD in uncontrolled settings is needed. This paper presents a framework for recognizing the affective state of children with ASD in an in-the-wild setting using heart rate (HR) information. More specifically, an algorithm is developed that can classify a participant's emotion as positive, negative, or neutral by analyzing the heart rate signal acquired from a smartwatch. The heart rate data are obtained in real time using a smartwatch application while the child learns to code a robot and interacts with an avatar. The avatar assists the child in developing communication skills and programming the robot. In this paper, we also present a semi-automated annotation technique based on facial expression recognition for the heart rate data. The HR signal is analyzed to extract features that capture the emotional state of the child. Additionally, in this paper, the performance of a raw HR-signal-based emotion classification algorithm is compared with a classification approach based on features extracted from HR signals using discrete wavelet transform (DWT). The experimental results demonstrate that the proposed method achieves comparable performance to state-of-the-art HR-based emotion recognition techniques, despite being conducted in an uncontrolled setting rather than a controlled lab environment. The framework presented in this paper contributes to the real-world affect analysis of children with ASD using HR information. By enabling emotion recognition in uncontrolled settings, this approach has the potential to improve the monitoring and understanding of the emotional well-being of children with ASD in their daily lives.

Assessing the construct validity of a theory of mind battery adapted to Tunisian school-aged children

Background

Theory of mind (ToM) refers to the ability to understand others' states of mind, desires, emotions, beliefs, and intentions to predict the content of their mental representations. Two major dimensions within ToM have been studied. The first is the type of inferred mental state, which can be cognitive or affective. The second comprises the types of processes involved according to their degree of complexity (first- and second-order false belief and advanced ToM). ToM acquisition is fundamental-a key component in the development of everyday human social interactions. ToM deficits have been reported in various neurodevelopmental disorders through various tools assessing disparate facets of social cognition. Nevertheless, Tunisian practitioners and researchers lack a linguistically and culturally appropriate psychometric tool for ToM assessment among school-aged children.

Objective

To assess the construct validity of a translated and adapted French ToM Battery for Arabic-speaking Tunisian school-aged children.

Methods

The focal ToM Battery was designed with neuropsychological and neurodevelopmental theory and composed of 10 subtests distributed evenly in three parts: Pre-conceptual, cognitive, and affective ToM. Translated and adapted to the Tunisian sociocultural context, this ToM battery was individually administered to 179 neurotypical Tunisian children (90 girls and 89 boys) aged 7-12 years.

Results

After controlling for the age effect, construct validity was empirically confirmed on two dimensions (cognitive and affective) via structural equation modeling (SEM) analysis, demonstrating that this solution has a good fit. The results confirmed that the age affected differentially the performance obtained on ToM tasks based on the two components of the battery.

Conclusion

Our findings confirm that the Tunisian version of the ToM Battery has robust construct validity for the assessment of cognitive and affective ToM in Tunisian school-aged children; hence, it could be adopted in clinical and research settings.

A Bimodal Emotion Recognition Approach through the Fusion of Electroencephalography and Facial Sequences

In recent years, human-computer interaction (HCI) systems have become increasingly popular. Some of these systems demand particular approaches for discriminating actual emotions through the use of better multimodal methods. In this work, a deep canonical correlation analysis (DCCA) based multimodal emotion recognition method is presented through the fusion of electroencephalography (EEG) and facial video clips. A two-stage framework is implemented, where the first stage extracts relevant features for emotion recognition using a single modality, while the second stage merges the highly correlated features from the two modalities and performs classification. Convolutional neural network (CNN) based Resnet50 and 1D-CNN (1-Dimensional CNN) have been utilized to extract features from facial video clips and EEG modalities, respectively. A DCCA-based approach was used to fuse highly correlated features, and three basic human emotion categories (happy, neutral, and sad) were classified using the SoftMax classifier. The proposed approach was investigated based on the publicly available datasets called MAHNOB-HCI and DEAP. Experimental results revealed an average accuracy of 93.86% and 91.54% on the MAHNOB-HCI and DEAP datasets, respectively. The competitiveness of the proposed framework and the justification for exclusivity in achieving this accuracy were evaluated by comparison with existing work.

Searching for Best Predictors of Paralinguistic Comprehension and Production of Emotions in Communication in Adults With Moderate Intellectual Disability

Paralinguistic comprehension and production of emotions in communication include the skills of recognizing and interpreting emotional states with the help of facial expressions, prosody and intonation. In the relevant scientific literature, the skills of paralinguistic comprehension and production of emotions in communication are related primarily to receptive language abilities, although some authors found also their correlations with intellectual abilities and acoustic features of the voice. Therefore, the aim of this study was to investigate which of the mentioned variables (receptive language ability, acoustic features of voice, intellectual ability, social-demographic), presents the most relevant predictor of paralinguistic comprehension and paralinguistic production of emotions in communication in adults with moderate intellectual disabilities (MID). The sample included 41 adults with MID, 20–49 years of age (M = 34.34, SD = 7.809), 29 of whom had MID of unknown etiology, while 12 had Down syndrome. All participants are native speakers of Serbian. Two subscales from The Assessment Battery for Communication – Paralinguistic comprehension of emotions in communication and Paralinguistic production of emotions in communication, were used to assess the examinees from the aspect of paralinguistic comprehension and production skills. For the graduation of examinees from the aspect of assumed predictor variables, the following instruments were used: Peabody Picture Vocabulary Test was used to assess receptive language abilities, Computerized Speech Lab (“Kay Elemetrics” Corp., model 4300) was used to assess acoustic features of voice, and Raven’s Progressive Matrices were used to assess intellectual ability. Hierarchical regression analysis was applied to investigate to which extent the proposed variables present an actual predictor variables for paralinguistic comprehension and production of emotions in communication as dependent variables. The results of this analysis showed that only receptive language skills had statistically significant predictive value for paralinguistic comprehension of emotions (β = 0.468, t = 2.236, p < 0.05), while the factor related to voice frequency and interruptions, form the domain of acoustic voice characteristics, displays predictive value for paralinguistic production of emotions (β = 0.280, t = 2.076, p < 0.05). Consequently, this study, in the adult population with MID, evidenced a greater importance of voice and language in relation to intellectual abilities in understanding and producing emotions.