Initial eye gaze to faces and its functional consequence on face identification abilities in autism spectrum disorder

Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods

BACKGROUND

Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process.

OBJECTIVE

In this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments.

METHODS

Here, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual's visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD.

RESULTS

Our results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD.

CONCLUSIONS

Ultimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.

Peripheral facial features guiding eye movements and reducing fixational variability

Face processing is a fast and efficient process due to its evolutionary and social importance. A majority of people direct their first eye movement to a featureless point just below the eyes that maximizes accuracy in recognizing a person's identity and gender. Yet, the exact properties or features of the face that guide the first eye movements and reduce fixational variability are unknown. Here, we manipulated the presence of the facial features and the spatial configuration of features to investigate their effect on the location and variability of first and second fixations to peripherally presented faces. Our results showed that observers can utilize the face outline, individual facial features, and feature spatial configuration to guide the first eye movements to their preferred point of fixation. The eyes have a preferential role in guiding the first eye movements and reducing fixation variability. Eliminating the eyes or altering their position had the greatest influence on the location and variability of fixations and resulted in the largest detriment to face identification performance. The other internal features (nose and mouth) also contribute to reducing fixation variability. A subsequent experiment measuring detection of single features showed that the eyes have the highest detectability (relative to other features) in the visual periphery providing a strong sensory signal to guide the oculomotor system. Together, the results suggest a flexible multiple-cue approach that might be a robust solution to cope with how the varying eccentricities in the real world influence the ability to resolve individual feature properties and the preferential role of the eyes.

Alexithymia explains atypical spatiotemporal dynamics of eye gaze in autism

Recognition of emotional facial expressions is considered to be atypical in autism. This difficulty is thought to be due to the way that facial expressions are visually explored. Evidence for atypical visual exploration of emotional faces in autism is, however, equivocal. We propose that, where observed, atypical visual exploration of emotional facial expressions is due to alexithymia, a distinct but frequently co-occurring condition. In this eye-tracking study we tested the alexithymia hypothesis using a number of recent methodological advances to study eye gaze during several emotion processing tasks (emotion recognition, intensity judgements, free gaze), in 25 adults with, and 45 without, autism. A multilevel polynomial modelling strategy was used to describe the spatiotemporal dynamics of eye gaze to emotional facial expressions. Converging evidence from traditional and novel analysis methods revealed that atypical gaze to the eyes is best predicted by alexithymia in both autistic and non-autistic individuals. Information theoretic analyses also revealed differential effects of task on gaze patterns as a function of alexithymia, but not autism. These findings highlight factors underlying atypical emotion processing in autistic individuals, with wide-ranging implications for emotion research.

Social Attention Deficits in Children With Autism Spectrum Disorder: Task Dependence of Objects vs. Faces Observation Bias

Social attention deficits represent a central impairment of patients suffering from autism spectrum disorder (ASD), but the nature of such deficits remains controversial. We compared visual attention regarding social (faces) vs. non-social stimuli (objects), in an ecological diagnostic context, in 46 children and adolescents divided in two groups: ASD (N = 23) and typical neurodevelopment (TD) (N = 23), matched for chronological age and intellectual performance. Eye-tracking measures of visual scanning, while exploring and describing scenes from three different tasks from the Autism Diagnostic Observation Schedule (ADOS), were analyzed: "Description of a Picture," "Cartoons," and "Telling a Story from a Book." Our analyses revealed a three-way interaction between Group, Task, and Social vs. Object Stimuli. We found a striking main effect of group and a task dependence of attentional allocation: while the TD attended first and longer to faces, ASD participants became similar to TD when they were asked to look at pictures while telling a story. Our results suggest that social attention allocation is task dependent, raising the question whether spontaneous attention deficits can be rescued by guiding goal-directed actions.

Autism spectrum heterogeneity: fact or artifact?

The current diagnostic practices are linked to a 20-fold increase in the reported prevalence of ASD over the last 30 years. Fragmenting the autism phenotype into dimensional "autistic traits" results in the alleged recognition of autism-like symptoms in any psychiatric or neurodevelopemental condition and in individuals decreasingly distant from the typical population, and prematurely dismisses the relevance of a diagnostic threshold. Non-specific socio-communicative and repetitive DSM 5 criteria, combined with four quantitative specifiers as well as all their possible combinations, render limitless variety of presentations consistent with the categorical diagnosis of ASD. We propose several remedies to this problem: maintain a line of research on prototypical autism; limit the heterogeneity compatible with a categorical diagnosis to situations with a phenotypic overlap and a validated etiological link with prototypical autism; reintroduce the qualitative properties of autism presentations and of current dimensional specifiers, language, intelligence, comorbidity, and severity in the criteria used to diagnose autism in replacement of quantitative "social" and "repetitive" criteria; use these qualitative features combined with the clinical intuition of experts and machine-learning algorithms to differentiate coherent subgroups in today's autism spectrum; study these subgroups separately, and then compare them; and question the autistic nature of "autistic traits".