Predicting ASD Diagnosis in Children with Synthetic and Image-based Eye Gaze Data

Enhancing Autism Detection Through Gaze Analysis Using Eye Tracking Sensors and Data Attribution with Distillation in Deep Neural Networks

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition characterized by differences in social communication and repetitive behaviors, often associated with atypical visual attention patterns. In this paper, the Gaze-Based Autism Classifier (GBAC) is proposed, which is a Deep Neural Network model that leverages both data distillation and data attribution techniques to enhance ASD classification accuracy and explainability. Using data sampled by eye tracking sensors, the model identifies unique gaze behaviors linked to ASD and applies an explainability technique called TracIn for data attribution by computing self-influence scores to filter out noisy or anomalous training samples. This refinement process significantly improves both accuracy and computational efficiency, achieving a test accuracy of 94.35% while using only 77% of the dataset, showing that the proposed GBAC outperforms the same model trained on the full dataset and random sample reductions, as well as the benchmarks. Additionally, the data attribution analysis provides insights into the most influential training examples, offering a deeper understanding of how gaze patterns correlate with ASD-specific characteristics. These results underscore the potential of integrating explainable artificial intelligence into neurodevelopmental disorder diagnostics, advancing clinical research by providing deeper insights into the visual attention patterns associated with ASD.

Early identification of autism spectrum disorder based on machine learning with eye-tracking data

BACKGROUND

Early identification of autism spectrum disorder (ASD) improves long-term outcomes, yet significant diagnostic delays persist.

METHODS

A retrospective cohort of 449 children (ASD: 246, typically developing [TD]: 203) was used for model development. Eye-movement data were collected from the participants watching videos that featured eye-tracking paradigms for assessing social and non-social cognition. Five machine learning algorithms, namely random forest, support vector machine, logistic regression, artificial neural network, and extreme gradient boosting, were trained to classify children with ASD and TD. The best-performing algorithm was selected to build the final model which was further evaluated in a prospective cohort of 80 children. The Shapley values interpreted important eye-tracking features.

RESULTS

Random forest outperformed other algorithms during model development and achieved an area under the curve of 0.849 (< 3 years: 0.832, ≥ 3 years: 0.868) on the external validation set. Of the ten most important eye-tracking features, three measured social cognition, and the rest were related to non-social cognition. A deterioration in model performance was observed using only the social or non-social cognition-related eye-tracking features.

LIMITATIONS

The sample size of this study, although larger than that of existing studies of ASD based on eye-tracking data, was still relatively small compared to the number of features.

CONCLUSIONS

Machine learning models based on eye-tracking data have the potential to be cost- and time-efficient digital tools for the early identification of ASD. Eye-tracking phenotypes related to social and non-social cognition play an important role in distinguishing children with ASD from TD children.

A Review on Autism Spectrum Disorder Screening by Artificial Intelligence Methods

PURPOSE

With the increasing prevalence of autism spectrum disorders (ASD), the importance of early screening and diagnosis has been subject to considerable discussion. Given the subtle differences between ASD children and typically developing children during the early stages of development, it is imperative to investigate the utilization of automatic recognition methods powered by artificial intelligence. We aim to summarize the research work on this topic and sort out the markers that can be used for identification.

METHODS

We searched the papers published in the Web of Science, PubMed, Scopus, Medline, SpringerLink, Wiley Online Library, and EBSCO databases from 1st January 2013 to 13th November 2023, and 43 articles were included.

RESULTS

These articles mainly divided recognition markers into five categories: gaze behaviors, facial expressions, motor movements, voice features, and task performance. Based on the above markers, the accuracy of artificial intelligence screening ranged from 62.13 to 100%, the sensitivity ranged from 69.67 to 100%, the specificity ranged from 54 to 100%.

CONCLUSION

Therefore, artificial intelligence recognition holds promise as a tool for identifying children with ASD. However, it still needs to continually enhance the screening model and improve accuracy through multimodal screening, thereby facilitating timely intervention and treatment.

A review of machine learning in scanpath analysis for passive gaze-based interaction

The scanpath is an important concept in eye tracking. It refers to a person's eye movements over a period of time, commonly represented as a series of alternating fixations and saccades. Machine learning has been increasingly used for the automatic interpretation of scanpaths over the past few years, particularly in research on passive gaze-based interaction, i.e., interfaces that implicitly observe and interpret human eye movements, with the goal of improving the interaction. This literature review investigates research on machine learning applications in scanpath analysis for passive gaze-based interaction between 2012 and 2022, starting from 2,425 publications and focussing on 77 publications. We provide insights on research domains and common learning tasks in passive gaze-based interaction and present common machine learning practices from data collection and preparation to model selection and evaluation. We discuss commonly followed practices and identify gaps and challenges, especially concerning emerging machine learning topics, to guide future research in the field.

Using visual attention estimation on videos for automated prediction of autism spectrum disorder and symptom severity in preschool children

Atypical visual attention in individuals with autism spectrum disorders (ASD) has been utilised as a unique diagnosis criterion in previous research. This paper presents a novel approach to the automatic and quantitative screening of ASD as well as symptom severity prediction in preschool children. We develop a novel computational pipeline that extracts learned features from a dynamic visual stimulus to classify ASD children and predict the level of ASD-related symptoms. Experimental results demonstrate promising performance that is superior to using handcrafted features and machine learning algorithms, in terms of evaluation metrics used in diagnostic tests. Using a leave-one-out cross-validation approach, we obtained an accuracy of 94.59%, a sensitivity of 100%, a specificity of 76.47% and an area under the receiver operating characteristic curve (AUC) of 96% for ASD classification. In addition, we obtained an accuracy of 94.74%, a sensitivity of 87.50%, a specificity of 100% and an AUC of 99% for ASD symptom severity prediction.

Prediction Model for Sensory Perception Abnormality in Autism Spectrum Disorder

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by heterogeneous clinical phenotypes. Patients often experience abnormal sensory perception, which may further affect the ASD core phenotype, significantly and adversely affecting their quality of life. However, biomarkers for the diagnosis of ASD sensory perception abnormality are currently elusive. We sought to identify potential biomarkers related to ASD sensory perception abnormality to construct a prediction model that could facilitate the early identification of and screening for ASD. Differentially expressed genes in ASD were obtained from the Gene Expression Omnibus database and were screened for genes related to sensory perception abnormality. After enrichment analysis, the random forest method was used to identify disease-characteristic genes. A prediction model was constructed with an artificial neural network. Finally, the results were validated using data from the dorsal root ganglion, cerebral cortex, and striatum of the BTBR T+ Itpr3tf/J (BTBR) ASD mouse model. A total of 1869 differentially expressed genes in ASD were screened, among which 16 genes related to sensory perception abnormality were identified. According to enrichment analysis, these 16 genes were mainly related to actin, cholesterol metabolism, and tight junctions. Using random forest, 15 disease-characteristic genes were screened for model construction. The area under the curve of the training set validation result was 0.999, and for the model function validation, the result was 0.711, indicating high accuracy. The validation of BTBR mice confirmed the reliability of using these disease-characteristic genes for prediction of ASD. In conclusion, we developed a highly accurate model for predicting ASD sensory perception abnormality from 15 disease-characteristic genes. This model provides a new method for the early identification and diagnosis of ASD sensory perception abnormality.

Developing inclusive lateral layouts for students with dyslexia - Chinese reading materials as an example

BACKGROUND/AIM

Students with learning disabilities have difficulties in reading abilities; however, their IQ is no less than that of ordinary students of the same age. This study investigated and developed three articles as the author and schoolteachers developed reading materials. Article A is with a standard layout; Article B is with keywords of various font sizes, and Article C is with a related illustration.

METHODS

Data of eye movements and reading tests from thirty students wherein 15 participants have dyslexia were collected. An eye-tracking methodology was employed to assess the dyslexics' students reading patterns and behavior.

RESULTS

ANOVA analysis shows differences in reading test performance among students for Article A with usual layout [F (1, 28) = 133.16, p = 0.000], but no significant differences for the other two articles. Based on the gaze map analysis, Article C (with illustration) can improve the reading completeness of the dyslexic students (eight out of fifteen dyslexic students had completed the reading during our experiment) than Article A and Article B.

CONCLUSION

The results affirm that special layouts and narrative writing styles can improve the reading attention of students with dyslexia. This study's results and conclusions can reference future teaching materials or lesson preparation using lateral layouts for people with dyslexia.

Machine learning based on eye-tracking data to identify Autism Spectrum Disorder: A systematic review and meta-analysis

BACKGROUND

Machine learning has been widely used to identify Autism Spectrum Disorder (ASD) based on eye-tracking, but its accuracy is uncertain. We aimed to summarize the available evidence on the performances of machine learning algorithms in classifying ASD and typically developing (TD) individuals based on eye-tracking data.

METHODS

We searched Medline, Embase, Web of Science, Scopus, Cochrane Library, IEEE Xplore Digital Library, Wan Fang Database, China National Knowledge Infrastructure, Chinese BioMedical Literature Database, VIP Database for Chinese Technical Periodicals, from database inception to December 24, 2021. Studies using machine learning methods to classify ASD and TD individuals based on eye-tracking technologies were included. We extracted the data on study population, model performances, algorithms of machine learning, and paradigms of eye-tracking. This study is registered with PROSPERO, CRD42022296037.

RESULTS

261 articles were identified, of which 24 studies with sample sizes ranging from 28 to 141 were included (n = 1396 individuals). Machine learning based on eye-tracking yielded the pooled classified accuracy of 81 % (I² = 73 %), specificity of 79 % (I² = 61 %), and sensitivity of 84 % (I² = 61 %) in classifying ASD and TD individuals. In subgroup analysis, the accuracy was 88 % (95 % CI: 85-91 %), 79 % (95 % CI: 72-84 %), 71 % (95 % CI: 59-91 %) for preschool-aged, school-aged, and adolescent-adult group. Eye-tracking stimuli and machine learning algorithms varied widely across studies, with social, static, and active stimuli and Support Vector Machine and Random Forest most commonly reported. Regarding the model performance evaluation, 15 studies reported their final results on validation datasets, four based on testing datasets, and five did not report whether they used validation datasets. Most studies failed to report the information on eye-tracking hardware and the implementation process.

CONCLUSION

Using eye-tracking data, machine learning has shown potential in identifying ASD individuals with high accuracy, especially in preschool-aged children. However, the heterogeneity between studies, the absence of test set-based performance evaluations, the small sample size, and the non-standardized implementation of eye-tracking might deteriorate the reliability of results. Further well-designed and well-executed studies with comprehensive and transparent reporting are needed to determine the optimal eye-tracking paradigms and machine learning algorithms.

Identifying Visual Attention Features Accurately Discerning Between Autism and Typically Developing: a Deep Learning Framework

Atypical visual attention is a hallmark of autism spectrum disorder (ASD). Identifying the attention features accurately discerning between people with ASD and typically developing (TD) at the individual level remains a challenge. In this study, we developed a new systematic framework combining high accuracy deep learning classification, deep learning segmentation, image ablation and a direct measurement of classification ability to identify the discriminative features for autism identification. Our two-stream model achieved the state-of-the-art performance with a classification accuracy of 0.95. Using this framework, two new categories of features, Food & drink and Outdoor-objects, were identified as discriminative attention features, in addition to the previously reported features including Center-object and Human-faces, etc. Altered attention to the new categories helps to understand related atypical behaviors in ASD. Importantly, the area under curve (AUC) based on the combined top-9 features identified in this study was 0.92, allowing an accurate classification at the individual level. We also obtained a small but informative dataset of 12 images with an AUC of 0.86, suggesting a potentially efficient approach for the clinical diagnosis of ASD. Together, our deep learning framework based on VGG-16 provides a novel and powerful tool to recognize and understand abnormal visual attention in ASD, which will, in turn, facilitate the identification of biomarkers for ASD.

The Classification of Abnormal Hand Movement to Aid in Autism Detection: Machine Learning Study

Background

A formal autism diagnosis can be an inefficient and lengthy process. Families may wait several months or longer before receiving a diagnosis for their child despite evidence that earlier intervention leads to better treatment outcomes. Digital technologies that detect the presence of behaviors related to autism can scale access to pediatric diagnoses. A strong indicator of the presence of autism is self-stimulatory behaviors such as hand flapping.

Objective

This study aims to demonstrate the feasibility of deep learning technologies for the detection of hand flapping from unstructured home videos as a first step toward validation of whether statistical models coupled with digital technologies can be leveraged to aid in the automatic behavioral analysis of autism. To support the widespread sharing of such home videos, we explored privacy-preserving modifications to the input space via conversion of each video to hand landmark coordinates and measured the performance of corresponding time series classifiers.

Methods

We used the Self-Stimulatory Behavior Dataset (SSBD) that contains 75 videos of hand flapping, head banging, and spinning exhibited by children. From this data set, we extracted 100 hand flapping videos and 100 control videos, each between 2 to 5 seconds in duration. We evaluated five separate feature representations: four privacy-preserved subsets of hand landmarks detected by MediaPipe and one feature representation obtained from the output of the penultimate layer of a MobileNetV2 model fine-tuned on the SSBD. We fed these feature vectors into a long short-term memory network that predicted the presence of hand flapping in each video clip.

Results

The highest-performing model used MobileNetV2 to extract features and achieved a test F1 score of 84 (SD 3.7; precision 89.6, SD 4.3 and recall 80.4, SD 6) using 5-fold cross-validation for 100 random seeds on the SSBD data (500 total distinct folds). Of the models we trained on privacy-preserved data, the model trained with all hand landmarks reached an F1 score of 66.6 (SD 3.35). Another such model trained with a select 6 landmarks reached an F1 score of 68.3 (SD 3.6). A privacy-preserved model trained using a single landmark at the base of the hands and a model trained with the average of the locations of all the hand landmarks reached an F1 score of 64.9 (SD 6.5) and 64.2 (SD 6.8), respectively.

Conclusions

We created five lightweight neural networks that can detect hand flapping from unstructured videos. Training a long short-term memory network with convolutional feature vectors outperformed training with feature vectors of hand coordinates and used almost 900,000 fewer model parameters. This study provides the first step toward developing precise deep learning methods for activity detection of autism-related behaviors.