Facial Expression Production in Autism: A Meta-Analysis

Sensorimotor regulation of facial expression - An untouched frontier

Facial expression is a critical form of nonverbal social communication which promotes emotional exchange and affiliation among humans. Facial expressions are generated via precise contraction of the facial muscles, guided by sensory feedback. While the neural pathways underlying facial motor control are well characterized in humans and primates, it remains unknown how tactile and proprioceptive information reaches these pathways to guide facial muscle contraction. Thus, despite the importance of facial expressions for social functioning, little is known about how they are generated as a unique sensorimotor behavior. In this review, we highlight current knowledge about sensory feedback from the face and how it is distinct from other body regions. We describe connectivity between the facial sensory and motor brain systems, and call attention to the other brain systems which influence facial expression behavior, including vision, gustation, emotion, and interoception. Finally, we petition for more research on the sensory basis of facial expressions, asserting that incomplete understanding of sensorimotor mechanisms is a barrier to addressing atypical facial expressivity in clinical populations.

Being facially expressive is socially advantageous

Individuals vary in how they move their faces in everyday social interactions. In a first large-scale study, we measured variation in dynamic facial behaviour during social interaction and examined dyadic outcomes and impression formation. In Study 1, we recorded semi-structured video calls with 52 participants interacting with a confederate across various everyday contexts. Video clips were rated by 176 independent participants. In Study 2, we examined video calls of 1315 participants engaging in unstructured video-call interactions. Facial expressivity indices were extracted using automated Facial Action Coding Scheme analysis and measures of personality and partner impressions were obtained by self-report. Facial expressivity varied considerably across participants, but little across contexts, social partners or time. In Study 1, more facially expressive participants were more well-liked, agreeable, and successful at negotiating (if also more agreeable). Participants who were more facially competent, readable, and perceived as readable were also more well-liked. In Study 2, we replicated the findings that facial expressivity was associated with agreeableness and liking by their social partner, and additionally found it to be associated with extraversion and neuroticism. Findings suggest that facial behaviour is a stable individual difference that proffers social advantages, pointing towards an affiliative, adaptive function.

Evaluation of the Psychometric Properties of the Social Communication Questionnaire in Rural Kenya

Children can be reliably diagnosed with autism as early as 3 years of age, and early interventions are initiated. There is often a significant gap between the age of onset of symptoms (2-3 years) and diagnosis (8-10 years) in Africa. We conducted a study to validate the Social Communication Questionnaire (SCQ) as a screening instrument in a rural setting in Kenya. The study was conducted along the Kenyan Coast. Study participants included 172 children with a neurodevelopmental disorder (NDD) diagnosis (84 of which were autism) and 112 controls. Internal consistency was evaluated through the use of Cronbach's alpha, confirmatory factor analysis (CFA) with maximum likelihood procedure to assess the conceptual model for the SCQ. Additionally, the sensitivity and specificity of cut-off scores using ROC analysis and item difficulties and discrimination quality using an IRT framework were also assessed. Factor analysis revealed an adequate fitting model for the three-factor DSM-IV-TR (root mean squared error of approximation (RMSEA) = 0.050; Comparative Fit Index (CFI) = 0.974; Tucker-Lewis Index (TLI) = 0.973) and two-factor DSM-5 factor structure (RMSEA = 0.050; CFI = 0.972; TLI = 0.974). The reliability coefficient alphas for the whole group for all items (Cronbach's α = 0.90) and all three domains (Cronbach's α = 0.68-0.84) were acceptable to excellent. The recommended cut-off score of 15 yielded 72% sensitivity and 100% specificity in the ASD group compared to the typically developing group. We provide early evidence of the adequate factor structure and good internal consistency of the SCQ. We also note that the recommended cut-off yielded sufficient predictive validity.

Mapping the differential impact of spontaneous and conversational laughter on brain and mind: an fMRI study in autism

Spontaneous and conversational laughter are important socio-emotional communicative signals. Neuroimaging findings suggest that non-autistic people engage in mentalizing to understand the meaning behind conversational laughter. Autistic people may thus face specific challenges in processing conversational laughter, due to their mentalizing difficulties. Using fMRI, we explored neural differences during implicit processing of these two types of laughter. Autistic and non-autistic adults passively listened to funny words, followed by spontaneous laughter, conversational laughter, or noise-vocoded vocalizations. Behaviourally, words plus spontaneous laughter were rated as funnier than words plus conversational laughter, and the groups did not differ. However, neuroimaging results showed that non-autistic adults exhibited greater medial prefrontal cortex activation while listening to words plus conversational laughter, than words plus genuine laughter, while autistic adults showed no difference in medial prefrontal cortex activity between these two laughter types. Our findings suggest a crucial role for the medial prefrontal cortex in understanding socio-emotionally ambiguous laughter via mentalizing. Our study also highlights the possibility that autistic people may face challenges in understanding the essence of the laughter we frequently encounter in everyday life, especially in processing conversational laughter that carries complex meaning and social ambiguity, potentially leading to social vulnerability. Therefore, we advocate for clearer communication with autistic people.

The relationship between autistic traits, expressiveness, readability and social perceptions

This study investigated the relationship between autistic traits, expressiveness, readability (both actual and perceived), social favourability, and likability. Sixty participants designated as 'targets' were video recorded in a range of social scenarios and their autistic traits were measured using the Autism Spectrum Quotient. The videos were then shown to 106 new participants designated 'perceivers', who were split into three groups to make judgments related to readability, expressiveness, and social favourability respectively. Mediation analyses revealed that autistic traits negatively impacted both perceived likeability and social favourability, mediated by lowered expressiveness. Autistic traits also directly impacted readability, which was not mediated by expressiveness. The findings show how the level of autistic traits of a target can influence how they are socially perceived by others.

Emotion recognition and social functioning in individuals with autism spectrum condition and intellectual disability

OBJECTIVE

Most previous studies have examined emotion recognition in autism spectrum condition (ASC) without intellectual disability (ID). However, ASC and ID co-occur to a high degree. The main aims of the study were to examine emotion recognition in individuals with ASC and co-occurring intellectual disability (ASC-ID) as compared to individuals with ID alone, and to investigate the relationship between emotion recognition and social functioning.

METHODS

The sample consisted of 30 adult participants with ASC-ID and a comparison group of 29 participants with ID. Emotion recognition was assessed by the facial emotions test, while. social functioning was assessed by the social responsiveness scale-second edition (SRS-2).

RESULTS

The accuracy of emotion recognition was significantly lower in individuals with ASC-ID compared to the control group with ID, especially when it came to identifying angry and fearful emotions. Participants with ASC-ID exhibited more pronounced difficulties in social functioning compared to those with ID, and there was a significant negative correlation between emotion recognition and social functioning. However, emotion recognition accounted for only 8% of the variability observed in social functioning.

CONCLUSION

Our data indicate severe difficulties in the social-perceptual domain and in everyday social functioning in individuals with ASC-ID.

Valence and Intensity of Emotional Expression in Autistic and Non-Autistic Toddlers

PURPOSE

Differences in emotional experience and expression have long been recognized as common in the presentation of autism, yet research examining emotional expression in early childhood is limited, with mixed findings. Understanding emotional reactivity and expression in autism in early life is an essential step towards uncovering the mechanisms of these risks and identifying targets for intervention.

METHODS

The present study examined emotional expression in autistic (N = 17) and non-autistic (N = 20) toddlers (mean age = 25.27; SD = 1.88) during emotion elicitation tasks aimed at eliciting joy, frustration, and unease. Video recorded tasks were coded in ten second intervals for emotional valence and intensity, and the following variables were computed: proportion of time in positive, neutral, and negative affect; maximum intensity of positive and negative affect; and range of affect (i.e., most negative to most positive intensity).

RESULTS

Autistic toddlers spent more time in neutral facial expressions, less time displaying positive affect, and had somewhat less intense positive emotional expression than non-autistic peers. Small differences were apparent in intensity of negative affect expression, while no differences emerged in duration of time spent in negative affect.

CONCLUSION

Findings emphasize that differences may be more apparent in duration, rather than intensity of emotional expression, and that it may be particularly important to examine periods of "neutral" affect in young autistic children. Future research should consider the best ways to understand emotional reactivity in this population considering their unique interests, challenges, and communication styles.

A comprehensive analysis towards exploring the promises of AI-related approaches in autism research

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition that presents challenges in communication, social interaction, repetitive behaviour, and limited interests. Detecting ASD at an early stage is crucial for timely interventions and an improved quality of life. In recent times, Artificial Intelligence (AI) has been increasingly used in ASD research. The rise in ASD diagnoses is due to the growing number of ASD cases and the recognition of the importance of early detection, which leads to better symptom management. This study explores the potential of AI in identifying early indicators of autism, aligning with the United Nations Sustainable Development Goals (SDGs) of Good Health and Well-being (Goal 3) and Peace, Justice, and Strong Institutions (Goal 16). The paper aims to provide a comprehensive overview of the current state-of-the-art AI-based autism classification by reviewing recent publications from the last decade. It covers various modalities such as Eye gaze, Facial Expression, Motor skill, MRI/fMRI, and EEG, and multi-modal approaches primarily grouped into behavioural and biological markers. The paper presents a timeline spanning from the history of ASD to recent developments in the field of AI. Additionally, the paper provides a category-wise detailed analysis of the AI-based application in ASD with a diagrammatic summarization to convey a holistic summary of different modalities. It also reports on the successes and challenges of applying AI for ASD detection while providing publicly available datasets. The paper paves the way for future scope and directions, providing a complete and systematic overview for researchers in the field of ASD.

The cross-race effect in automatic facial expression recognition violates measurement invariance

Emotion has been a subject undergoing intensive research in psychology and cognitive neuroscience over several decades. Recently, more and more studies of emotion have adopted automatic rather than manual methods of facial emotion recognition to analyze images or videos of human faces. Compared to manual methods, these computer-vision-based, automatic methods can help objectively and rapidly analyze a large amount of data. These automatic methods have also been validated and believed to be accurate in their judgments. However, these automatic methods often rely on statistical learning models (e.g., deep neural networks), which are intrinsically inductive and thus suffer from problems of induction. Specifically, the models that were trained primarily on Western faces may not generalize well to accurately judge Eastern faces, which can then jeopardize the measurement invariance of emotions in cross-cultural studies. To demonstrate such a possibility, the present study carries out a cross-racial validation of two popular facial emotion recognition systems-FaceReader and DeepFace-using two Western and two Eastern face datasets. Although both systems could achieve overall high accuracies in the judgments of emotion category on the Western datasets, they performed relatively poorly on the Eastern datasets, especially in recognition of negative emotions. While these results caution the use of these automatic methods of emotion recognition on non-Western faces, the results also suggest that the measurements of happiness outputted by these automatic methods are accurate and invariant across races and hence can still be utilized for cross-cultural studies of positive psychology.

Evidence for the adaptive parsing of non-communicative eye movements during joint attention interactions

During social interactions, the ability to detect and respond to gaze-based joint attention bids often involves the evaluation of non-communicative eye movements. However, very little is known about how much humans are able to track and parse spatial information from these non-communicative eye movements over time, and the extent to which this influences joint attention outcomes. This was investigated in the current study using an interactive computer-based joint attention game. Using a fully within-subjects design, we specifically examined whether participants were quicker to respond to communicative joint attention bids that followed predictive, as opposed to random or no, non-communicative gaze behaviour. Our results suggest that in complex, dynamic tasks, people adaptively use and dismiss non-communicative gaze information depending on whether it informs the locus of an upcoming joint attention bid. We also went further to examine the extent to which this ability to track dynamic spatial information was specific to processing gaze information. This was achieved by comparing performance to a closely matched non-social task where eye gaze cues were replaced with dynamic arrow stimuli. Whilst we found that people are also able to track and use dynamic non-social information from arrows, there was clear evidence for a relative advantage for tracking gaze cues during social interactions. The implications of these findings for social neuroscience and autism research are discussed.

Tübinger Training for Autism Spectrum Disorders (TüTASS): a structured group intervention on self-perception and social skills of children with autism spectrum disorder (ASD)

In autism spectrum disorders (ASD), social communication and stereotypical behaviour patterns affect all areas of life, and can result in a decrease of its quality. Previous research has shown promising results for the social skills of groups of children with ASD. Furthermore, a pilot study of the Tübingen Group Training for Autism Spectrum Disorders (TüTASS) has demonstrated that mindfulness-based elements achieve additional positive effects. To build on these findings, the TüTASS training was adapted and expanded. Indeed, the TüTASS currently includes 20 90-min appointments starting with the basic skills of emotions, body, and communication, which are then transferred to personal, family, peer, and school spheres. The appointments have a fixed, consistent structure and each includes a body awareness exercise. In this study, we evaluated the TÜTASS with 27 children with ASD. The results showed improvements in pre-post comparison in behaviours associated with autism, as well as in externalising and internalising behaviours as assessed by parent reports, participant self-reports, and independent raters in participating children. Furthermore, the perceived parent burden in relation to their children decreased, whereas the participants' self-rated quality of life increased. Overall, both the participants and their parents rated the TüTASS very positively in rating sheets and in free feedback. If replicated in larger controlled trials, TÜTASS training might be a useful treatment tool for groups of children with ASD.

Autism-associated gene shank3 is necessary for social contagion in zebrafish

BACKGROUND

Animal models enable targeting autism-associated genes, such as the shank3 gene, to assess their impact on behavioural phenotypes. However, this is often limited to simple behaviours relevant for social interaction. Social contagion is a complex phenotype forming the basis of human empathic behaviour and involves attention to the behaviour of others for recognizing and sharing their emotional or affective state. Thus, it is a form of social communication, which constitutes the most common developmental impairment across autism spectrum disorders (ASD).

METHODS

Here we describe the development of a zebrafish model that identifies the neurocognitive mechanisms by which shank3 mutation drives deficits in social contagion. We used a CRISPR-Cas9 technique to generate mutations to the shank3a gene, a zebrafish paralogue found to present greater orthology and functional conservation relative to the human gene. Mutants were first compared to wild types during a two-phase protocol that involves the observation of two conflicting states, distress and neutral, and the later recall and discrimination of others when no longer presenting such differences. Then, the whole-brain expression of different neuroplasticity markers was compared between genotypes and their contribution to cluster-specific phenotypic variation was assessed.

RESULTS

The shank3 mutation markedly reduced social contagion via deficits in attention contributing to difficulties in recognising affective states. Also, the mutation changed the expression of neuronal plasticity genes. However, only downregulated neuroligins clustered with shank3a expression under a combined synaptogenesis component that contributed specifically to variation in attention.

LIMITATIONS

While zebrafish are extremely useful in identifying the role of shank3 mutations to composite social behaviour, they are unlikely to represent the full complexity of socio-cognitive and communication deficits presented by human ASD pathology. Moreover, zebrafish cannot represent the scaling up of these deficits to higher-order empathic and prosocial phenotypes seen in humans.

CONCLUSIONS

We demonstrate a causal link between the zebrafish orthologue of an ASD-associated gene and the attentional control of affect recognition and consequent social contagion. This models autistic affect-communication pathology in zebrafish and reveals a genetic attention-deficit mechanism, addressing the ongoing debate for such mechanisms accounting for emotion recognition difficulties in autistic individuals.

Toward the detection of reduced emotion expression intensity: an autism sibling study

INTRODUCTION

Expressing emotions through spontaneous facial expression is an important nonverbal social communication skill. In our study, we aimed to demonstrate that both children with autism spectrum disorder (ASD) and the non-ASD siblings of children with ASD have deficits in this skill.

METHOD

In this study, we analyzed the six core facial emotion expressions of three distinct groups of children - those diagnosed with ASD (n = 60), non-ASD siblings (n = 60), and typically developed children (n = 60). To analyze facial expressions, we employed a computer vision program that uses machine learning algorithms to detect facial features and conducted an evidence-based task that involved assessing participants' ability to recognize facial emotion expressions.

RESULTS

Deficits in spontaneous emotion expression were shown in the children with ASD and in non-ASD siblings when compared with typically developed children. Interestingly, it was determined that these deficits were not related to the severity of the autism symptoms in the ASD group.

CONCLUSIONS

The results of the study suggest that computer-based automated analysis of facial expressions with contextual social scenes task holds potential for measuring limitations in the ability to express emotions, and they supplement the traditional clinical assessment of social phenotypical behavior deficits. This applies both to children with ASD and especially, to the non-ASD siblings of children with ASD. This study adds a novel approach to previous literature examining the emotion expression skills.

Facial mimicry and metacognitive judgments in emotion recognition are distinctly modulated by social anxiety and autistic traits

Facial mimicry as well as the accurate assessment of one's performance when judging others' emotional expressions have been suggested to inform successful emotion recognition. Differences in the integration of these two information sources might explain alterations in the perception of others' emotions in individuals with Social Anxiety Disorder and individuals on the autism spectrum. Using a non-clinical sample (N = 57), we examined the role of social anxiety and autistic traits in the link between facial mimicry, or confidence in one's performance, and emotion recognition. While participants were presented with videos of spontaneous emotional facial expressions, we measured their facial muscle activity, asked them to label the expressions and indicate their confidence in accurately labelling the expressions. Our results showed that confidence in emotion recognition was lower with higher social anxiety traits even though actual recognition was not related to social anxiety traits. Higher autistic traits, in contrast, were associated with worse recognition, and a weakened link between facial mimicry and performance. Consequently, high social anxiety traits might not affect emotion recognition itself, but the top-down evaluation of own abilities in emotion recognition contexts. High autistic traits, in contrast, may be related to lower integration of sensorimotor simulations, which promote emotion recognition.

Categorical versus dimensional structure of autism spectrum disorder: A multi-method investigation

Background

A key question for any psychopathological diagnosis is whether the condition is continuous or discontinuous with typical variation. The primary objective of this study was to use a multi-method approach to examine the broad latent categorical versus dimensional structure of autism spectrum disorder (ASD).

Method

Data were aggregated across seven independent samples of participants with ASD, other neurodevelopmental disorders (NDD), and non-ASD/NDD controls (aggregate Ns = 512-16,755; ages 1.5-22). Scores from four distinct phenotype measures formed composite "indicators" of the latent ASD construct. The primary indicator set included eye gaze metrics from seven distinct social stimulus paradigms. Logistic regressions were used to combine gaze metrics within/across paradigms, and derived predicted probabilities served as indicator values. Secondary indicator sets were constructed from clinical observation and parent-report measures of ASD symptoms. Indicator sets were submitted to taxometric- and latent class analyses.

Results

Across all indicator sets and analytic methods, there was strong support for categorical structure corresponding closely to ASD diagnosis. Consistent with notions of substantial phenotypic heterogeneity, the ASD category had a wide range of symptom severity. Despite the examination of a large sample with a wide range of IQs in both genders, males and children with lower IQ were over-represented in the ASD category, similar to observations in diagnosed cases.

Conclusions

Our findings provide strong support for categorical structure corresponding closely to ASD diagnosis. The present results bolster the use of well-diagnosed and representative ASD groups within etiologic and clinical research, motivating the ongoing search for major drivers of the ASD phenotype. Despite the categorical structure of ASD, quantitative symptom measurements appear more useful for examining relationships with other factors.

Primary school children rate children with autism negatively on looks, speech and speech content

Adults and adolescents form negative first impressions of ASD adults and children. We examined the first impression ratings of primary school children (6-9 years) of their ASD peers. 146 school children rated either silent videos, speech or transcribe speech from 14 actors (7 ASD, 7 TD). The ASD actors were rated more negatively than the typically developing actors on all three stimulus types. Children with ASD are likely to be judged more negatively than their peers at the very start of their formal education. Contrary to previous research, for primary school children, the content of the speech was judged as negatively as the delivery of the speech.

On the Evaluation of Engagement in Immersive Applications When Users Are on the Autism Spectrum

New generation wearable devices allow for the development of interactive environments tailored for Virtual Reality (VR)- and Augmented Reality (AR)-based treatment of Autism Spectrum Disorders (ASD). Experts agree on their potential; however, there is lack of consensus on how to perform trials and the need arises for evaluation frameworks, methods, and techniques appropriate for the ASD population. In this paper, we report on a study conducted with high-functioning ASD people in the 21-23 age range, with the objectives of (1) evaluating the engagement of two headsets offering distinct immersive experiences, (2) reasoning on the interpretation of engagement factors in the case of ASD people, and (3) translating results into general guidelines for the development of VR/AR-based ASD treatment. To this aim, we (1) designed two engagement evaluation frameworks based on behavioral observation measures, (2) set up two packages of reference immersive scenarios, (3) defined the association between metrics and scenarios, and (4) administered the scenarios in distinct sessions for the investigated headsets. Results show that the immersive experiences are engaging and that the apparent lack of success of some evaluation factors can become potential advantages within the framework of VR/AR-based ASD treatment design.

Digitally assisted diagnostics of autism spectrum disorder

Digital technologies have the potential to support psychiatric diagnostics and, in particular, differential diagnostics of autism spectrum disorder in the near future, making clinical decisions more objective, reliable and evidence-based while reducing clinical resources. Multimodal automatized measurement of symptoms at cognitive, behavioral, and neuronal levels combined with artificial intelligence applications offer promising strides toward personalized prognostics and treatment strategies. In addition, these new technologies could enable systematic and continuous assessment of longitudinal symptom development, beyond the usual scope of clinical practice. Early recognition of exacerbation and simplified, as well as detailed, progression control would become possible. Ultimately, digitally assisted diagnostics will advance early recognition. Nonetheless, digital technologies cannot and should not substitute clinical decision making that takes the comprehensive complexity of individual longitudinal and cross-section presentation of autism spectrum disorder into account. Yet, they might aid the clinician by objectifying decision processes and provide a welcome relief to resources in the clinical setting.

Specificity and sensitivity of the social communication questionnaire lifetime screening tool for autism spectrum disorder in a UK CAMHS service

INTRODUCTION

The Social Communication Questionnaire is used to identify children and young people (CYP) who may require formal ASD assessment. However, there is a paucity of research on its utility in Children and Adolescent Mental Health Services. This evaluation aimed to determine the sensitivity and specificity of the Social Communication Questionnaire (SCQ) in a UK, Midlands CAMHS service.

METHOD

Forty young people (mean age 13.75 years) were screened using the caregiver reported SCQ before completing 'gold standard' assessment.

RESULTS

The SCQ had a sensitivity of 80% and a specificity of 25.7%. ROC curve analysis indicated low diagnostic accuracy. Differences in predictive accuracy of SCQ and diagnostic standard were statistically significant (p < 0.0001).

CONCLUSION

This evaluation builds on previous research suggesting that the SCQ may not be an efficient screening tool in CAMHS settings.

Mapping the time course of overt emotion dysregulation, self-injurious behavior, and aggression in psychiatrically hospitalized autistic youth: A naturalistic study

Challenges with emotion dysregulation, self-injurious behavior (SIB), and aggression are common in autistic individuals. Prior research on the relationships between these behaviors is limited mainly to cross-sectional correlations of parent-report data. Understanding how emotion dysregulation, SIB, and aggression present and relate to one another in real-time could add to our understanding of the context and function of these behaviors. The present study examined the real-time occurrence and temporal relationships between these behaviors in 53 psychiatrically hospitalized autistic youth. Over 500 hours of behavioral observation occurred during everyday activities in the hospital. Start and stop times for instances of overt emotion dysregulation, SIB, and aggression were coded live using a custom mobile phone app. Results indicated large individual variability in the frequency and duration of these behaviors and their co-occurrence. Both SIB and aggression co-occurred with overt emotion dysregulation at above-chance levels, suggesting a role for emotional distress in the occurrence of these behaviors. However, there was substantial variability within and between individuals in co-occurrence, and SIB and aggression often (and for some individuals, almost always) occurred without overt emotion dysregulation. Relatedly, cross-recurrence quantitative analysis revealed that SIB and aggression preceded emotion dysregulation more often than emotion dysregulation preceded SIB and aggression. Future research, perhaps using ambulatory psychophysiological measures, is needed to understand whether emotion dysregulation may sometimes be present but not easily observed during SIB and aggression. LAY SUMMARY: This study provides insight into how overt emotion dysregulation (i.e., visible distress), aggression, and self-injury unfold in real-time for autistic individuals. Participants were 53 autistic youth staying in a psychiatric hospital. Research staff observed participants in everyday activities on the hospital unit and noted instances of aggression, self-injurious behavior, and emotion dysregulation. Results suggest that aggression and self-injury sometimes occur with visible signs of distress but also often occur without visible distress. In addition, observable distress was more common in the moments after these behaviors than in the moments before.

Identifying and describing subtypes of spontaneous empathic facial expression production in autistic adults

BACKGROUND

It is unclear whether atypical patterns of facial expression production metrics in autism reflect the dynamic and nuanced nature of facial expressions across people or a true diagnostic difference. Furthermore, the heterogeneity observed across autism symptomatology suggests a need for more adaptive and personalized social skills programs. Towards this goal, it would be useful to have a more concrete and empirical understanding of the different expressiveness profiles within the autistic population and how they differ from neurotypicals.

METHODS

We used automated facial coding and an unsupervised clustering approach to limit inter-individual variability in facial expression production that may have otherwise obscured group differences in previous studies, allowing an "apples-to-apples" comparison between autistic and neurotypical adults. Specifically, we applied k-means clustering to identify subtypes of facial expressiveness in an autism group (N = 27) and a neurotypical control group (N = 57) separately. The two most stable clusters from these analyses were then further characterized and compared based on their expressiveness and emotive congruence to emotionally charged stimuli.

RESULTS

Our main finding was that a subset of autistic adults in our sample show heightened spontaneous facial expressions irrespective of image valence. We did not find evidence for greater incongruous (i.e., inappropriate) facial expressions in autism. Finally, we found a negative trend between expressiveness and emotion recognition within the autism group.

CONCLUSION

The results from our previous study on self-reported empathy and current expressivity findings point to a higher degree of facial expressions recruited for emotional resonance in autism that may not always be adaptive (e.g., experiencing similar emotional resonance regardless of valence). These findings also build on previous work indicating that facial expression intensity is not diminished in autism and suggest the need for intervention programs to focus on emotion recognition and social skills in the context of both negative and positive emotions.

Test–Retest Reliability in Automated Emotional Facial Expression Analysis: Exploring FaceReader 8.0 on Data from Typically Developing Children and Children with Autism

Automated emotional facial expression analysis (AEFEA) is used widely in applied research, including the development of screening/diagnostic systems for atypical human neurodevelopmental conditions. The validity of AEFEA systems has been systematically studied, but their test–retest reliability has not been researched thus far. We explored the test–retest reliability of a specific AEFEA software, Noldus FaceReader 8.0 (FR8; by Noldus Information Technology). We collected intensity estimates for 8 repeated emotions through FR8 from facial video recordings of 60 children: 31 typically developing children and 29 children with autism spectrum disorder. Test–retest reliability was imperfect in 20% of cases, affecting a substantial proportion of data points; however, the test–retest differences were small. This shows that the test–retest reliability of FR8 is high but not perfect. A proportion of cases which initially failed to show perfect test–retest reliability reached it in a subsequent analysis by FR8. This suggests that repeated analyses by FR8 can, in some cases, lead to the “stabilization” of emotion intensity datasets. Under ANOVA, the test–retest differences did not influence the pattern of cross-emotion and cross-group effects and interactions. Our study does not question the validity of previous results gained by AEFEA technology, but it shows that further exploration of the test–retest reliability of AEFEA systems is desirable.

The Current View on the Paradox of Pain in Autism Spectrum Disorders

Autism spectrum disorder (ASD) is a neurodevelopmental disorder, which affects 1 in 44 children and may cause severe disabilities. Besides socio-communicational difficulties and repetitive behaviors, ASD also presents as atypical sensorimotor function and pain reactivity. While chronic pain is a frequent co-morbidity in autism, pain management in this population is often insufficient because of difficulties in pain evaluation, worsening their prognosis and perhaps driving higher mortality rates. Previous observations have tended to oversimplify the experience of pain in autism as being insensitive to painful stimuli. Various findings in the past 15 years have challenged and complicated this dogma. However, a relatively small number of studies investigates the physiological correlates of pain reactivity in ASD. We explore the possibility that atypical pain perception in people with ASD is mediated by alterations in pain perception, transmission, expression and modulation, and through interactions between these processes. These complex interactions may account for the great variability and sometimes contradictory findings from the studies. A growing body of evidence is challenging the idea of alterations in pain processing in ASD due to a single factor, and calls for an integrative view. We propose a model of the pain cycle that includes the interplay between the molecular and neurophysiological pathways of pain processing and it conscious appraisal that may interfere with pain reactivity and coping in autism. The role of social factors in pain-induced response is also discussed. Pain assessment in clinical care is mostly based on subjective rather than objective measures. This review clarifies the strong need for a consistent methodology, and describes innovative tools to cope with the heterogeneity of pain expression in ASD, enabling individualized assessment. Multiple measures, including self-reporting, informant reporting, clinician-assessed, and purely physiological metrics may provide more consistent results. An integrative view on the regulation of the pain cycle offers a more robust framework to characterize the experience of pain in autism.

Delineation of a Phenotype Caused by a KAT6B Missense Variant Not Resembling Say-Barber-Biesecker-Young-Simpson and Genitopatellar Syndromes

Say-Barber-Biesecker-Young-Simpson syndrome (SBBYSS) and genitopatellar syndrome (GPS) are caused by variants of lysine acetyltransferase 6B (KAT6B). These variants tend to occur in the terminal exons of KAT6B. Here, we report a patient with global developmental delay, intellectual disability, autistic behavior, muscular hypotonia, facial dysmorphism, and seizures caused by a novel missense variant in exon 7 of KAT6B. The patient showed a phenotype differing from those of SBBYSS and GPS. We also report patients with missense variants in the proximal exons of KAT6B showing dysmorphic features and autistic behavior not resembling the characteristics of SBBYSS and GPS. Missense variants in the proximal exons of KAT6B may have a dominant negative effect or cause gain of function, leading to unique phenotypes not resembling those of SBBYSS and GPS.

Application of Machine Learning Techniques to Detect the Children with Autism Spectrum Disorder

Early detection of autism spectrum disorder (ASD) is highly beneficial to the health sustainability of children. Existing detection methods depend on the assessment of experts, which are subjective and costly. In this study, we proposed a machine learning approach that fuses physiological data (electroencephalography, EEG) and behavioral data (eye fixation and facial expression) to detect children with ASD. Its implementation can improve detection efficiency and reduce costs. First, we used an innovative approach to extract features of eye fixation, facial expression, and EEG data. Then, a hybrid fusion approach based on a weighted naive Bayes algorithm was presented for multimodal data fusion with a classification accuracy of 87.50%. Results suggest that the machine learning classification approach in this study is effective for the early detection of ASD. Confusion matrices and graphs demonstrate that eye fixation, facial expression, and EEG have different discriminative powers for the detection of ASD and typically developing children, and EEG may be the most discriminative information. The physiological and behavioral data have important complementary characteristics. Thus, the machine learning approach proposed in this study, which combines the complementary information, can significantly improve classification accuracy.

Positive, negative, neutral-or unknown? The perceived valence of emotions expressed by young autistic children in a novel context suited to autism

Starting early in life, autistics are characterized as having atypical facial expressions, as well as decreased positive and increased negative affect. The literature on autistic facial expressions remains small, however, with disparate methods and results suggesting limited understanding of common autistic emotions. Furthermore, unlike non-autistics’ emotions, autistics’ emotions have been assessed without considering this population’s characteristics. In this study, the valence of young children’s facial expressions was thus rated as positive, negative, neutral, or “unknown”—a term for perceived emotions observers do not understand. Facial expressions were assessed using the Montreal Stimulating Play Situation, a context incorporating potential autistic interests. Comparing 37 autistic and 39 typical young (27–56 months) age-matched children, we found no group differences in expressed positive, negative, and neutral emotions. We did find differences in unknown emotions, which were unique to the autistic group. Preliminary data also showed that autistic children’s repetitive behaviors co-occurred with positive, neutral, and unknown emotions, but not with negative emotions. In a novel context that considers their characteristics, we did not find decreased positive or increased negative emotions in young autistic children. Instead, they uniquely expressed emotions perceived as unknown, showing the need to improve our understanding of their full emotional repertoire.

The Transdiagnostic Relevance of Self-Other Distinction to Psychiatry Spans Emotional, Cognitive and Motor Domains

Self-other distinction refers to the ability to distinguish between our own and other people's physical and mental states (actions, perceptions, emotions etc.). Both the right temporo-parietal junction and brain areas associated with the human mirror neuron system are likely to critically influence self-other distinction, given their respective contributions to theory of mind and embodied empathy. The degree of appropriate self-other distinction will vary according to the exact social situation, and how helpful it is to feel into, or remain detached from, another person's mental state. Indeed, the emotional resonance that we can share with others affords the gift of empathy, but over-sharing may pose a downside, leading to a range of difficulties from personal distress to paranoia, and perhaps even motor tics and compulsions. The aim of this perspective paper is to consider how evidence from behavioral and neurophysiological studies supports a role for problems with self-other distinction in a range of psychiatric symptoms spanning the emotional, cognitive and motor domains. The various signs and symptoms associated with problematic self-other distinction comprise both maladaptive and adaptive (compensatory) responses to dysfunction within a common underlying neuropsychological mechanism, compelling the adoption of more holistic transdiagnostic therapeutic approaches within Psychiatry.

Shifts in Behavioral Synchrony in Response to an Interaction Partner's Distress in Adolescents With and Without ASD

Individuals with Autism spectrum disorder (ASD) show altered behavioral synchrony and empathic behavior. Yet, an ecologically valid examination of these in adolescents is still required. We employed a social interaction sequence comprising a friendly conversation with a confederate, an expression of distress by her, and a return to friendly conversation. 44 male adolescents (22 with ASD) participated. Socio-emotional behaviors, coded by blind raters and through automated analyses of motion, were analyzed. Results indicated reduced synchrony in the ASD group. Moreover, whereas controls displayed a decrease in synchrony when facing another's distress, adolescents with ASD showed no such change. No group differences were found on empathic behavior. Findings imply gaps between verbal and non-verbal responses of adolescents with ASD to a distressed interaction partner.

Is the Combination of ADOS and ADI-R Necessary to Classify ASD? Rethinking the "Gold Standard" in Diagnosing ASD

Diagnosing autism spectrum disorder (ASD) requires extensive clinical expertise and training as well as a focus on differential diagnoses. The diagnostic process is particularly complex given symptom overlap with other mental disorders and high rates of co-occurring physical and mental health concerns. The aim of this study was to conduct a data-driven selection of the most relevant diagnostic information collected from a behavior observation and an anamnestic interview in two clinical samples of children/younger adolescents and adolescents/adults with suspected ASD. Via random forests, the present study discovered patterns of symptoms in the diagnostic data of 2310 participants (46% ASD, 54% non-ASD, age range 4-72 years) using data from the combined Autism Diagnostic Observation Schedule (ADOS) and Autism Diagnostic Interview-Revised (ADI-R) and ADOS data alone. Classifiers built on reduced subsets of diagnostic features yield satisfactory sensitivity and specificity values. For adolescents/adults specificity values were lower compared to those for children/younger adolescents. The models including ADOS and ADI-R data were mainly built on ADOS items and in the adolescent/adult sample the classifier including only ADOS items performed even better than the classifier including information from both instruments. Results suggest that reduced subsets of ADOS and ADI-R items may suffice to effectively differentiate ASD from other mental disorders. The imbalance of ADOS and ADI-R items included in the models leads to the assumption that, particularly in adolescents and adults, the ADI-R may play a lesser role than current behavior observations.

Review: Posed vs. Genuine Facial Emotion Recognition and Expression in Autism and Implications for Intervention

Different styles of social interaction are one of the core characteristics of autism spectrum disorder (ASD). Social differences among individuals with ASD often include difficulty in discerning the emotions of neurotypical people based on their facial expressions. This review first covers the rich body of literature studying differences in facial emotion recognition (FER) in those with ASD, including behavioral studies and neurological findings. In particular, we highlight subtle emotion recognition and various factors related to inconsistent findings in behavioral studies of FER in ASD. Then, we discuss the dual problem of FER – namely facial emotion expression (FEE) or the production of facial expressions of emotion. Despite being less studied, social interaction involves both the ability to recognize emotions and to produce appropriate facial expressions. How others perceive facial expressions of emotion in those with ASD has remained an under-researched area. Finally, we propose a method for teaching FER [FER teaching hierarchy (FERTH)] based on recent research investigating FER in ASD, considering the use of posed vs. genuine emotions and static vs. dynamic stimuli. We also propose two possible teaching approaches: (1) a standard method of teaching progressively from simple drawings and cartoon characters to more complex audio-visual video clips of genuine human expressions of emotion with context clues or (2) teaching in a field of images that includes posed and genuine emotions to improve generalizability before progressing to more complex audio-visual stimuli. Lastly, we advocate for autism interventionists to use FER stimuli developed primarily for research purposes to facilitate the incorporation of well-controlled stimuli to teach FER and bridge the gap between intervention and research in this area.

Measuring the Emergence of Specific Abilities in Young Children with Autism Spectrum Disorders: The Example of Early Hyperlexic Traits

The presence of a restricted interest in written materials, including an early ability to name and recognize letters and numbers, is regularly reported in preschoolers with autism spectrum disorders (ASDs). There is, however, scarce information on this early ability akin to emerging hyperlexic traits in preschoolers with ASD younger than 3 years old. Here, we defined a measure of early naming and recognition of letters and numbers in 155 preschoolers with ASD using a sliding window approach combined with a 90th percentile threshold criterion, and subsequently compared the profiles of children with ASD with and without early hyperlexic traits. Using this measure, we found that 9% of children with ASD showed early hyperlexic traits. The early ability to name and recognize letters and numbers was associated with a higher level of restricted and repetitive behaviors yet more social-oriented behaviors at baseline and with better expressive and written communication at baseline and one year later. This study contributes to a better definition of the profile of children with ASD with an early ability in letters and numbers akin to emerging hyperlexic traits, a skill that is associated with promising social strengths and language abilities in this subgroup of children.

The role of context in verbal humor processing in autism

Difficulties in processing humor have been associated with individuals with autism. The current study investigated whether humor comprehension and appreciation could be augmented in children with autism by providing contextual support suggesting that humor was to be expected. A verbally presented riddle task was used in which participants were assessed for their subjective ratings and comprehension of the materials. They were also filmed to record any smiling or laughing. Both riddles and control stimuli were presented with supporting verbal context and also without it. The results showed that (a) the greater subjective appreciation of riddles than of control stimuli was dependent on the provision of context for the participants with autism and that (b) context statistically equated these ratings of riddles between participants with autism and matched typically developing controls. However, context had no effect on comprehension or affective response. The results of the current study demonstrate that children with autism are, even in the most conservative interpretation, able to use verbal context to recognize verbal humor. This lays the foundation of possible interventions based on training sensitivity to context.

Brain mRNA Expression Associated with Cortical Volume Alterations in Autism Spectrum Disorder

Numerous studies report abnormal cerebral cortex volume (CCV) in autism spectrum disorder (ASD); however, genes related to CCV abnormalities in ASD remain largely unknown. Here, we identify genes associated with CCV alterations in ASD by performing spatial correlations between the gene expression of 6 donated brains and neuroimaging data from 1,404 ASD patients and 1,499 controls. Based on spatial correlations between gene expression and CCV differences from two independent meta-analyses and between gene expression and individual CCV distributions of 404 patients and 496 controls, we identify 417 genes associated with both CCV differences and individual CCV distributions. These genes are enriched for genetic association signals and genes downregulated in the ASD post-mortem brain. The expression patterns of these genes are correlated with brain activation patterns of language-related neural processes frequently impaired in ASD. These findings highlight a model whereby genetic risk impacts gene expression (downregulated), which leads to CCV alterations in ASD.

Déficits précoces de l'empathie et psychopathologie Early empathy deficits and psychopathology

Empathy is a complex social-cognitive ability that is best understood by distinguishing its emotional, cognitive and motivational dimensions, which from early childhood interact between the child and her/his social environment. To date, among the many factors that are acknowledged to influence the development of empathy, children's temperament and parenting behaviors have been identified as interacting in predicting the extent to which children demonstrate empathic responses. Recent studies in developmental social neuroscience cast light on the neural networks engaged in the development of each of the dimensions that constitute empathy, which are needed to navigate social interaction and establishing positive social relationships. Indeed, early deficits in empathic processes can lead to difficulties in socialization, particularly associated with reduced attention to others' emotions, especially when they are suffering, a lesser degree of remorse and guilt, and a greater tendency to ignore social norms or break the rules. Difficulties in socialization are particularly visible in two well-known developmental disorders: children with autism spectrum disorders (ASD) and children with conduct disorder and callous unemotional traits (CU). This paper provides a critical and selective review of recent empirical studies in psychopathology and developmental neuroscience by addressing the dimensions underlying empathy, specifically emotional sharing and caring for others. For children with ASD, some studies report that they pay less attention to another person in distress. However, functional neuroimaging studies conducted with ASD adolescents indicate that the emotional dimension appears to be preserved, but a lack of emotional self-regulation may impair them from experiencing empathic concern. Children with conduct disorder and CU traits clearly manifest a reduced autonomic nervous system response to others' distress or suffering. This may account for their disregard or contempt for others' well-being and social norms. Functional neuroimaging studies show that atypical patterns of brain activity at 15 months of age can predict later severe conduct disorder. Neural regions engaged in emotional processing such as the anterior cingulate cortex, insula and amygdala shown reduced activation to empathy-eliciting stimuli in children with CU. Finally, the genetic nature of CU traits is highlighted in several studies. We conclude by proposing several avenues for developmental research to identify biomarkers from an early age and by inviting to focus on psychological interventions with those populations accordingly.

Is Smiling the Key? Machine Learning Analytics Detect Subtle Patterns in Micro-Expressions of Infants with ASD

Time is a key factor to consider in Autism Spectrum Disorder. Detecting the condition as early as possible is crucial in terms of treatment success. Despite advances in the literature, it is still difficult to identify early markers able to effectively forecast the manifestation of symptoms. Artificial intelligence (AI) provides effective alternatives for behavior screening. To this end, we investigated facial expressions in 18 autistic and 15 typical infants during their first ecological interactions, between 6 and 12 months of age. We employed Openface, an AI-based software designed to systematically analyze facial micro-movements in images in order to extract the subtle dynamics of Social Smiles in unconstrained Home Videos. Reduced frequency and activation intensity of Social Smiles was computed for children with autism. Machine Learning models enabled us to map facial behavior consistently, exposing early differences hardly detectable by non-expert naked eye. This outcome contributes to enhancing the potential of AI as a supportive tool for the clinical framework.

Imitation and recognition of facial emotions in autism: a computer vision approach

Background

Imitation of facial expressions plays an important role in social functioning. However, little is known about the quality of facial imitation in individuals with autism and its relationship with defining difficulties in emotion recognition.

Methods

We investigated imitation and recognition of facial expressions in 37 individuals with autism spectrum conditions and 43 neurotypical controls. Using a novel computer-based face analysis, we measured instructed imitation of facial emotional expressions and related it to emotion recognition abilities.

Results

Individuals with autism imitated facial expressions if instructed to do so, but their imitation was both slower and less precise than that of neurotypical individuals. In both groups, a more precise imitation scaled positively with participants’ accuracy of emotion recognition.

Limitations

Given the study’s focus on adults with autism without intellectual impairment, it is unclear whether the results generalize to children with autism or individuals with intellectual disability. Further, the new automated facial analysis, despite being less intrusive than electromyography, might be less sensitive.

Conclusions

Group differences in emotion recognition, imitation and their interrelationships highlight potential for treatment of social interaction problems in individuals with autism.

Investigating automatic emotion processing in boys with autism via eye tracking and facial mimicry recordings

Difficulties in automatic emotion processing in individuals with autism spectrum disorder (ASD) might remain concealed in behavioral studies due to compensatory strategies. To gain more insight in the mechanisms underlying facial emotion recognition, we recorded eye tracking and facial mimicry data of 20 school-aged boys with ASD and 20 matched typically developing controls while performing an explicit emotion recognition task. Proportional looking times to specific face regions (eyes, nose, and mouth) and face exploration dynamics were analyzed. In addition, facial mimicry was assessed. Boys with ASD and controls were equally capable to recognize expressions and did not differ in proportional looking times, and number and duration of fixations. Yet, specific facial expressions elicited particular gaze patterns, especially within the control group. Both groups showed similar face scanning dynamics, although boys with ASD demonstrated smaller saccadic amplitudes. Regarding the facial mimicry, we found no emotion specific facial responses and no group differences in the responses to the displayed facial expressions. Our results indicate that boys with and without ASD employ similar eye gaze strategies to recognize facial expressions. Smaller saccadic amplitudes in boys with ASD might indicate a less exploratory face processing strategy. Yet, this slightly more persistent visual scanning behavior in boys with ASD does not imply less efficient emotion information processing, given the similar behavioral performance. Results on the facial mimicry data indicate similar facial responses to emotional faces in boys with and without ASD. LAY SUMMARY: We investigated (i) whether boys with and without autism apply different face exploration strategies when recognizing facial expressions and (ii) whether they mimic the displayed facial expression to a similar extent. We found that boys with and without ASD recognize facial expressions equally well, and that both groups show similar facial reactions to the displayed facial emotions. Yet, boys with ASD visually explored the faces slightly less than the boys without ASD.

Facial Expression Production and Recognition in Autism Spectrum Disorders: A Shifting Landscape

Social "difficulties" associated with ASD may be a product of neurotypical-autistic differences in emotion expression and recognition. Research suggests that neurotypical and autistic individuals exhibit expressive differences, with autistic individuals displaying less frequent expressions that are rated lower in quality by non-autistic raters. Autistic individuals have difficulties recognizing neurotypical facial expressions; neurotypical individuals have difficulties recognizing autistic expressions. However, findings are mixed. Task-related factors (e.g., intensity of stimuli) and participant characteristics (e.g., age, IQ, comorbid diagnoses) may contribute to the mixed findings. The authors conclude by highlighting important areas for future research and the clinical implications of the discussed findings.

Digital Behavioral Phenotyping Detects Atypical Pattern of Facial Expression in Toddlers with Autism

Commonly used screening tools for autism spectrum disorder (ASD) generally rely on subjective caregiver questionnaires. While behavioral observation is more objective, it is also expensive, time-consuming, and requires significant expertise to perform. As such, there remains a critical need to develop feasible, scalable, and reliable tools that can characterize ASD risk behaviors. This study assessed the utility of a tablet-based behavioral assessment for eliciting and detecting one type of risk behavior, namely, patterns of facial expression, in 104 toddlers (ASD N = 22) and evaluated whether such patterns differentiated toddlers with and without ASD. The assessment consisted of the child sitting on his/her caregiver's lap and watching brief movies shown on a smart tablet while the embedded camera recorded the child's facial expressions. Computer vision analysis (CVA) automatically detected and tracked facial landmarks, which were used to estimate head position and facial expressions (Positive, Neutral, All Other). Using CVA, specific points throughout the movies were identified that reliably differentiate between children with and without ASD based on their patterns of facial movement and expressions (area under the curves for individual movies ranging from 0.62 to 0.73). During these instances, children with ASD more frequently displayed Neutral expressions compared to children without ASD, who had more All Other expressions. The frequency of All Other expressions was driven by non-ASD children more often displaying raised eyebrows and an open mouth, characteristic of engagement/interest. Preliminary results suggest computational coding of facial movements and expressions via a tablet-based assessment can detect differences in affective expression, one of the early, core features of ASD. LAY SUMMARY: This study tested the use of a tablet in the behavioral assessment of young children with autism. Children watched a series of developmentally appropriate movies and their facial expressions were recorded using the camera embedded in the tablet. Results suggest that computational assessments of facial expressions may be useful in early detection of symptoms of autism.

New Technologies as Promising Tools for Assessing Facial Emotion Expressions Impairments in ASD: A Systematic Review

The ability to recognize and express emotions from facial expressions are essential for successful social interactions. Facial Emotion Recognition (FER) and Facial Emotion Expressions (FEEs), both of which seem to be impaired in Autism Spectrum Disorders (ASD) and contribute to socio-communicative difficulties, participate in the diagnostic criteria for ASD. Only a few studies have focused on FEEs processing and the rare behavioral studies of FEEs in ASD have yielded mixed results. Here, we review studies comparing the production of FEEs between participants with ASD and non-ASD control subjects, with a particular focus on the use of automatic facial expression analysis software. A systematic literature search in accordance with the PRISMA statement identified 20 reports published up to August 2020 concerning the use of new technologies to evaluate both spontaneous and voluntary FEEs in participants with ASD. Overall, the results highlight the importance of considering socio-demographic factors and psychiatric co-morbidities which may explain the previous inconsistent findings, particularly regarding quantitative data on spontaneous facial expressions. There is also reported evidence for an inadequacy of FEEs in individuals with ASD in relation to expected emotion, with a lower quality and coordination of facial muscular movements. Spatial and kinematic approaches to characterizing the synchrony, symmetry and complexity of facial muscle movements thus offer clues to identifying and exploring promising new diagnostic targets. These findings have allowed hypothesizing that there may be mismatches between mental representations and the production of FEEs themselves in ASD. Such considerations are in line with the Facial Feedback Hypothesis deficit in ASD as part of the Broken Mirror Theory, with the results suggesting impairments of neural sensory-motor systems involved in processing emotional information and ensuring embodied representations of emotions, which are the basis of human empathy. In conclusion, new technologies are promising tools for evaluating the production of FEEs in individuals with ASD, and controlled studies involving larger samples of patients and where possible confounding factors are considered, should be conducted in order to better understand and counter the difficulties in global emotional processing in ASD.

Neurotypical individuals fail to understand action vitality form in children with autism spectrum disorder

Action vitality forms are highly pervasive aspects of daily life and have been widely assumed to be critical for basic social interactions. Previous evidence indicates that ASD children express their own vitality forms in a way that is motorically dissimilar to TD children. Here we demonstrate that this motor dissimilarity prevents neurotypical adults from recognizing vitality forms, when observing ASD children acting gently or rudely. Although ASD children differentiate these vitality forms, neurotypical adults were remarkably inaccurate in identifying them. This indicates that difficulty in social interaction for ASD individuals should not be entirely ascribed to their lack of understanding others, as standardly assumed. The failure of neurotypical individuals to understand them plays a critical role too.

Any defects of sociality in individuals diagnosed with autism spectrum disorder (ASD) are standardly explained in terms of those individuals’ putative impairments in a variety of cognitive functions. Recently, however, the need for a bidirectional approach to social interaction has been emphasized. Such an approach highlights differences in basic ways of acting between ASD and neurotypical individuals which would prevent them from understanding each other. Here we pursue this approach by focusing on basic action features reflecting the agent’s mood and affective states. These are action features Stern named “vitality forms,” and which are widely assumed to substantiate core social interactions [D. N. Stern, The Interpersonal World of the Infant (1985); D. N. Stern, Forms of Vitality Exploring Dynamic Experience in Psychology, Arts, Psychotherapy, and Development (2010)]. Previously we demonstrated that, although ASD and typically developing (TD) children alike differentiate vitality forms when performing actions, ASD children express them in a way that is motorically dissimilar to TD children. To assess whether this motor dissimilarity may have consequences for vitality form recognition, we asked neurotypical participants to identify the vitality form of different types of action performed by ASD or TD children. We found that participants exhibited remarkable inaccuracy in identifying ASD children’s vitality forms. Interestingly, their performance did not benefit from information feedback. This indicates that how people act matters for understanding others and for being understood by them. Because vitality forms pervade every aspect of daily life, our findings promise to open the way to a deeper comprehension of the bidirectional difficulties for both ASD and neurotypical individuals in interacting with one another.

Effects of being watched on eye gaze and facial displays of typical and autistic individuals during conversation

Communication with others relies on coordinated exchanges of social signals, such as eye gaze and facial displays. However, this can only happen when partners are able to see each other. Although previous studies report that autistic individuals have difficulties in planning eye gaze and making facial displays during conversation, evidence from real-life dyadic tasks is scarce and mixed. Across two studies, here we investigate how eye gaze and facial displays of typical and high-functioning autistic individuals are modulated by the belief in being seen and potential to show true gaze direction. Participants were recorded with an eye-tracking and video-camera system while they completed a structured Q&A task with a confederate under three social contexts: pre-recorded video, video-call and face-to-face. Typical participants gazed less to the confederate and produced more facial displays when they were being watched and when they were speaking. Contrary to our hypotheses, eye gaze and facial motion patterns in autistic participants were overall similar to the typical group. This suggests that high-functioning autistic participants are able to use eye gaze and facial displays as social signals. Future studies will need to investigate to what extent this reflects spontaneous behaviour or the use of compensation strategies.

Computer Vision Analysis of Reduced Interpersonal Affect Coordination in Youth With Autism Spectrum Disorder

Atypical social-emotional reciprocity is a core feature of autism spectrum disorder (ASD) but can be difficult to operationalize. One measurable manifestation of reciprocity may be interpersonal coordination, the tendency to align the form and timing of one's behaviors (including facial affect) with others. Interpersonal affect coordination facilitates sharing and understanding of emotional cues, and there is evidence that it is reduced in ASD. However, most research has not measured this process in true social contexts, due in part to a lack of tools for measuring dynamic facial expressions over the course of an interaction. Automated facial analysis via computer vision provides an efficient, granular, objective method for measuring naturally occurring facial affect and coordination. Youth with ASD and matched typically developing youth participated in cooperative conversations with their mothers and unfamiliar adults. Time-synchronized videos were analyzed with an open-source computer vision toolkit for automated facial analysis, for the presence and intensity of facial movements associated with positive affect. Both youth and adult conversation partners exhibited less positive affect during conversations when the youth partner had ASD. Youth with ASD also engaged in less affect coordination over the course of conversations. When considered dimensionally across youth with and without ASD, affect coordination significantly predicted scores on rating scales of autism-related social atypicality, adaptive social skills, and empathy. Findings suggest that affect coordination is an important interpersonal process with implications for broader social-emotional functioning. This preliminary study introduces a promising novel method for quantifying moment-to-moment facial expression and emotional reciprocity during natural interactions. LAY SUMMARY: This study introduces a novel, automated method for measuring social-emotional reciprocity during natural conversations, which may improve assessment of this core autism diagnostic behavior. We used computerized methods to measure facial affect and the degree of affect coordination between conversation partners. Youth with autism displayed reduced affect coordination, and reduced affect coordination predicted lower scores on measures of broader social-emotional skills.

Facial Expression Production and Recognition in Autism Spectrum Disorders: A Shifting Landscape

Social "difficulties" associated with ASD may be a product of neurotypical-autistic differences in emotion expression and recognition. Research suggests that neurotypical and autistic individuals exhibit expressive differences, with autistic individuals displaying less frequent expressions that are rated lower in quality by non-autistic raters. Autistic individuals have difficulties recognizing neurotypical facial expressions; neurotypical individuals have difficulties recognizing autistic expressions. However, findings are mixed. Task-related factors (e.g., intensity of stimuli) and participant characteristics (e.g., age, IQ, comorbid diagnoses) may contribute to the mixed findings. The authors conclude by highlighting important areas for future research and the clinical implications of the discussed findings.

Automated recognition of spontaneous facial expression in individuals with autism spectrum disorder: parsing response variability

BACKGROUND

Reduction or differences in facial expression are a core diagnostic feature of autism spectrum disorder (ASD), yet evidence regarding the extent of this discrepancy is limited and inconsistent. Use of automated facial expression detection technology enables accurate and efficient tracking of facial expressions that has potential to identify individual response differences.

METHODS

Children and adults with ASD (N = 124) and typically developing (TD, N = 41) were shown short clips of "funny videos." Using automated facial analysis software, we investigated differences between ASD and TD groups and within the ASD group in evidence of facial action unit (AU) activation related to the expression of positive facial expression, in particular, a smile.

RESULTS

Individuals with ASD on average showed less evidence of facial AUs (AU12, AU6) relating to positive facial expression, compared to the TD group (p < .05, r = - 0.17). Using Gaussian mixture model for clustering, we identified two distinct distributions within the ASD group, which were then compared to the TD group. One subgroup (n = 35), termed "over-responsive," expressed more intense positive facial expressions in response to the videos than the TD group (p < .001, r = 0.31). The second subgroup (n = 89), ("under-responsive"), displayed fewer, less intense positive facial expressions in response to videos than the TD group (p < .001; r = - 0.36). The over-responsive subgroup differed from the under-responsive subgroup in age and caregiver-reported impulsivity (p < .05, r = 0.21). Reduced expression in the under-responsive, but not the over-responsive group, was related to caregiver-reported social withdrawal (p < .01, r = - 0.3).

LIMITATIONS

This exploratory study does not account for multiple comparisons, and future work will have to ascertain the strength and reproducibility of all results. Reduced displays of positive facial expressions do not mean individuals with ASD do not experience positive emotions.

CONCLUSIONS

Individuals with ASD differed from the TD group in their facial expressions of positive emotion in response to "funny videos." Identification of subgroups based on response may help in parsing heterogeneity in ASD and enable targeting of treatment based on subtypes.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02299700. Registration date: November 24, 2014.

Towards the automatic detection of social biomarkers in autism spectrum disorder: introducing the simulated interaction task (SIT)

Social interaction deficits are evident in many psychiatric conditions and specifically in autism spectrum disorder (ASD), but hard to assess objectively. We present a digital tool to automatically quantify biomarkers of social interaction deficits: the simulated interaction task (SIT), which entails a standardized 7-min simulated dialog via video and the automated analysis of facial expressions, gaze behavior, and voice characteristics. In a study with 37 adults with ASD without intellectual disability and 43 healthy controls, we show the potential of the tool as a diagnostic instrument and for better description of ASD-associated social phenotypes. Using machine-learning tools, we detected individuals with ASD with an accuracy of 73%, sensitivity of 67%, and specificity of 79%, based on their facial expressions and vocal characteristics alone. Especially reduced social smiling and facial mimicry as well as a higher voice fundamental frequency and harmony-to-noise-ratio were characteristic for individuals with ASD. The time-effective and cost-effective computer-based analysis outperformed a majority vote and performed equal to clinical expert ratings.

Children with autism spectrum disorder produce more ambiguous and less socially meaningful facial expressions: an experimental study using random forest classifiers

Background

Computer vision combined with human annotation could offer a novel method for exploring facial expression (FE) dynamics in children with autism spectrum disorder (ASD).

Methods

We recruited 157 children with typical development (TD) and 36 children with ASD in Paris and Nice to perform two experimental tasks to produce FEs with emotional valence. FEs were explored by judging ratings and by random forest (RF) classifiers. To do so, we located a set of 49 facial landmarks in the task videos, we generated a set of geometric and appearance features and we used RF classifiers to explore how children with ASD differed from TD children when producing FEs.

Results

Using multivariate models including other factors known to predict FEs (age, gender, intellectual quotient, emotion subtype, cultural background), ratings from expert raters showed that children with ASD had more difficulty producing FEs than TD children. In addition, when we explored how RF classifiers performed, we found that classification tasks, except for those for sadness, were highly accurate and that RF classifiers needed more facial landmarks to achieve the best classification for children with ASD. Confusion matrices showed that when RF classifiers were tested in children with ASD, anger was often confounded with happiness.

Limitations

The sample size of the group of children with ASD was lower than that of the group of TD children. By using several control calculations, we tried to compensate for this limitation.

Conclusion

Children with ASD have more difficulty producing socially meaningful FEs. The computer vision methods we used to explore FE dynamics also highlight that the production of FEs in children with ASD carries more ambiguity.

Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production

The computational analysis of facial expressions is an emerging research topic that could overcome the limitations of human perception and get quick and objective outcomes in the assessment of neurodevelopmental disorders (e.g., Autism Spectrum Disorders, ASD). Unfortunately, there have been only a few attempts to quantify facial expression production and most of the scientific literature aims at the easier task of recognizing if either a facial expression is present or not. Some attempts to face this challenging task exist but they do not provide a comprehensive study based on the comparison between human and automatic outcomes in quantifying children’s ability to produce basic emotions. Furthermore, these works do not exploit the latest solutions in computer vision and machine learning. Finally, they generally focus only on a homogeneous (in terms of cognitive capabilities) group of individuals. To fill this gap, in this paper some advanced computer vision and machine learning strategies are integrated into a framework aimed to computationally analyze how both ASD and typically developing children produce facial expressions. The framework locates and tracks a number of landmarks (virtual electromyography sensors) with the aim of monitoring facial muscle movements involved in facial expression production. The output of these virtual sensors is then fused to model the individual ability to produce facial expressions. Gathered computational outcomes have been correlated with the evaluation provided by psychologists and evidence has been given that shows how the proposed framework could be effectively exploited to deeply analyze the emotional competence of ASD children to produce facial expressions.

In the face of stress: Interpreting individual differences in stress-induced facial expressions

Stress is an inevitable part of life that can profoundly impact social and emotional functioning, contributing to the development of psychiatric disease. One key component of emotion and social processing is facial expressions, which humans can readily detect and react to even without conscious awareness. Facial expressions have been the focus of philosophic and scientific interest for centuries. Historically, facial expressions have been relegated to peripheral indices of fixed emotion states. More recently, affective neuroscience has undergone a conceptual revolution, resulting in novel interpretations of these muscle movements. Here, we review the role of facial expressions according to the leading affective neuroscience theories, including constructed emotion and social-motivation accounts. We specifically highlight recent data (Mayo et al, 2018) demonstrating the way in which stress shapes facial expressions and how this is influenced by individual factors. In particular, we focus on the consequence of genetic variation within the endocannabinoid system, a neuromodulatory system implicated in stress and emotion, and its impact on stress-induced facial muscle activity. In a re-analysis of this dataset, we highlight how gender may also influence these processes, conceptualized as variation in the "fight-or-flight" or "tend-and-befriend" behavioral responses to stress. We speculate on how these interpretations may contribute to a broader understanding of facial expressions, discuss the potential use of facial expressions as a trans-diagnostic marker of psychiatric disease, and suggest future work necessary to resolve outstanding questions.