1
|
Wei Q, Dong W, Yu D, Wang K, Yang T, Xiao Y, Long D, Xiong H, Chen J, Xu X, Li T. Early identification of autism spectrum disorder based on machine learning with eye-tracking data. J Affect Disord 2024; 358:326-334. [PMID: 38615846 DOI: 10.1016/j.jad.2024.04.049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 02/15/2024] [Accepted: 04/11/2024] [Indexed: 04/16/2024]
Abstract
BACKGROUND Early identification of autism spectrum disorder (ASD) improves long-term outcomes, yet significant diagnostic delays persist. METHODS A retrospective cohort of 449 children (ASD: 246, typically developing [TD]: 203) was used for model development. Eye-movement data were collected from the participants watching videos that featured eye-tracking paradigms for assessing social and non-social cognition. Five machine learning algorithms, namely random forest, support vector machine, logistic regression, artificial neural network, and extreme gradient boosting, were trained to classify children with ASD and TD. The best-performing algorithm was selected to build the final model which was further evaluated in a prospective cohort of 80 children. The Shapley values interpreted important eye-tracking features. RESULTS Random forest outperformed other algorithms during model development and achieved an area under the curve of 0.849 (< 3 years: 0.832, ≥ 3 years: 0.868) on the external validation set. Of the ten most important eye-tracking features, three measured social cognition, and the rest were related to non-social cognition. A deterioration in model performance was observed using only the social or non-social cognition-related eye-tracking features. LIMITATIONS The sample size of this study, although larger than that of existing studies of ASD based on eye-tracking data, was still relatively small compared to the number of features. CONCLUSIONS Machine learning models based on eye-tracking data have the potential to be cost- and time-efficient digital tools for the early identification of ASD. Eye-tracking phenotypes related to social and non-social cognition play an important role in distinguishing children with ASD from TD children.
Collapse
Affiliation(s)
- Qiuhong Wei
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China; College of Medical Informatics, Medical Data Science Academy, Chongqing Engineering Research Center for Clinical Big-data and Drug Evaluation, Chongqing Medical University, Chongqing, China
| | - Wenxin Dong
- College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China; Big Data Center for Children's Medical Care, Children's Hospital of Chongqing Medical University, Chongqing, China
| | - Dongchuan Yu
- Key Laboratory of Child Development and Learning Science (Ministry of Education), Research Center for Learning Science, School of Biological Sciences & Medical Engineering, Southeast University, Nanjing, Jiangsu, China
| | - Ke Wang
- Big Data Center for Children's Medical Care, Children's Hospital of Chongqing Medical University, Chongqing, China
| | - Ting Yang
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China
| | - Yuanjie Xiao
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China
| | - Dan Long
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China
| | - Haiyi Xiong
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China
| | - Jie Chen
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China
| | - Ximing Xu
- Big Data Center for Children's Medical Care, Children's Hospital of Chongqing Medical University, Chongqing, China.
| | - Tingyu Li
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, Chongqing Key Laboratory of Child Neurodevelopment and Cognitive Disorders, Chongqing, China.
| |
Collapse
|
2
|
Mohamed Selim A, Barz M, Bhatti OS, Alam HMT, Sonntag D. A review of machine learning in scanpath analysis for passive gaze-based interaction. Front Artif Intell 2024; 7:1391745. [PMID: 38903158 PMCID: PMC11188426 DOI: 10.3389/frai.2024.1391745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Accepted: 05/15/2024] [Indexed: 06/22/2024] Open
Abstract
The scanpath is an important concept in eye tracking. It refers to a person's eye movements over a period of time, commonly represented as a series of alternating fixations and saccades. Machine learning has been increasingly used for the automatic interpretation of scanpaths over the past few years, particularly in research on passive gaze-based interaction, i.e., interfaces that implicitly observe and interpret human eye movements, with the goal of improving the interaction. This literature review investigates research on machine learning applications in scanpath analysis for passive gaze-based interaction between 2012 and 2022, starting from 2,425 publications and focussing on 77 publications. We provide insights on research domains and common learning tasks in passive gaze-based interaction and present common machine learning practices from data collection and preparation to model selection and evaluation. We discuss commonly followed practices and identify gaps and challenges, especially concerning emerging machine learning topics, to guide future research in the field.
Collapse
Affiliation(s)
- Abdulrahman Mohamed Selim
- German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Saarbrücken, Germany
| | - Michael Barz
- German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Saarbrücken, Germany
- Applied Artificial Intelligence, University of Oldenburg, Oldenburg, Germany
| | - Omair Shahzad Bhatti
- German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Saarbrücken, Germany
| | - Hasan Md Tusfiqur Alam
- German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Saarbrücken, Germany
| | - Daniel Sonntag
- German Research Center for Artificial Intelligence (DFKI), Interactive Machine Learning Department, Saarbrücken, Germany
- Applied Artificial Intelligence, University of Oldenburg, Oldenburg, Germany
| |
Collapse
|
3
|
Wei Q, Cao H, Shi Y, Xu X, Li T. Machine learning based on eye-tracking data to identify Autism Spectrum Disorder: A systematic review and meta-analysis. J Biomed Inform 2023; 137:104254. [PMID: 36509416 DOI: 10.1016/j.jbi.2022.104254] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Revised: 11/15/2022] [Accepted: 11/17/2022] [Indexed: 12/13/2022]
Abstract
BACKGROUND Machine learning has been widely used to identify Autism Spectrum Disorder (ASD) based on eye-tracking, but its accuracy is uncertain. We aimed to summarize the available evidence on the performances of machine learning algorithms in classifying ASD and typically developing (TD) individuals based on eye-tracking data. METHODS We searched Medline, Embase, Web of Science, Scopus, Cochrane Library, IEEE Xplore Digital Library, Wan Fang Database, China National Knowledge Infrastructure, Chinese BioMedical Literature Database, VIP Database for Chinese Technical Periodicals, from database inception to December 24, 2021. Studies using machine learning methods to classify ASD and TD individuals based on eye-tracking technologies were included. We extracted the data on study population, model performances, algorithms of machine learning, and paradigms of eye-tracking. This study is registered with PROSPERO, CRD42022296037. RESULTS 261 articles were identified, of which 24 studies with sample sizes ranging from 28 to 141 were included (n = 1396 individuals). Machine learning based on eye-tracking yielded the pooled classified accuracy of 81 % (I2 = 73 %), specificity of 79 % (I2 = 61 %), and sensitivity of 84 % (I2 = 61 %) in classifying ASD and TD individuals. In subgroup analysis, the accuracy was 88 % (95 % CI: 85-91 %), 79 % (95 % CI: 72-84 %), 71 % (95 % CI: 59-91 %) for preschool-aged, school-aged, and adolescent-adult group. Eye-tracking stimuli and machine learning algorithms varied widely across studies, with social, static, and active stimuli and Support Vector Machine and Random Forest most commonly reported. Regarding the model performance evaluation, 15 studies reported their final results on validation datasets, four based on testing datasets, and five did not report whether they used validation datasets. Most studies failed to report the information on eye-tracking hardware and the implementation process. CONCLUSION Using eye-tracking data, machine learning has shown potential in identifying ASD individuals with high accuracy, especially in preschool-aged children. However, the heterogeneity between studies, the absence of test set-based performance evaluations, the small sample size, and the non-standardized implementation of eye-tracking might deteriorate the reliability of results. Further well-designed and well-executed studies with comprehensive and transparent reporting are needed to determine the optimal eye-tracking paradigms and machine learning algorithms.
Collapse
Affiliation(s)
- Qiuhong Wei
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, China International Science and Technology Cooperation Base of Child Development and Critical Disorders, Chongqing Key Laboratory of Childhood Nutrition and Health, Chongqing, China
| | - Huiling Cao
- Department of Neonatology, Children's Hospital of Chongqing Medical University, Chongqing, China
| | - Yuan Shi
- Department of Neonatology, Children's Hospital of Chongqing Medical University, Chongqing, China
| | - Ximing Xu
- Big Data Center for Children's Medical Care, Children's Hospital of Chongqing Medical University, Chongqing, China.
| | - Tingyu Li
- Children Nutrition Research Center, Children's Hospital of Chongqing Medical University, National Clinical Research Center for Child Health and Disorders, Ministry of Education Key Laboratory of Child Development and Disorders, China International Science and Technology Cooperation Base of Child Development and Critical Disorders, Chongqing Key Laboratory of Childhood Nutrition and Health, Chongqing, China.
| |
Collapse
|
4
|
Quantifying the Predictability of Visual Scanpaths Using Active Information Storage. ENTROPY 2021; 23:e23020167. [PMID: 33573069 PMCID: PMC7912697 DOI: 10.3390/e23020167] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Revised: 01/22/2021] [Accepted: 01/23/2021] [Indexed: 12/27/2022]
Abstract
Entropy-based measures are an important tool for studying human gaze behavior under various conditions. In particular, gaze transition entropy (GTE) is a popular method to quantify the predictability of a visual scanpath as the entropy of transitions between fixations and has been shown to correlate with changes in task demand or changes in observer state. Measuring scanpath predictability is thus a promising approach to identifying viewers' cognitive states in behavioral experiments or gaze-based applications. However, GTE does not account for temporal dependencies beyond two consecutive fixations and may thus underestimate the actual predictability of the current fixation given past gaze behavior. Instead, we propose to quantify scanpath predictability by estimating the active information storage (AIS), which can account for dependencies spanning multiple fixations. AIS is calculated as the mutual information between a processes' multivariate past state and its next value. It is thus able to measure how much information a sequence of past fixations provides about the next fixation, hence covering a longer temporal horizon. Applying the proposed approach, we were able to distinguish between induced observer states based on estimated AIS, providing first evidence that AIS may be used in the inference of user states to improve human-machine interaction.
Collapse
|
5
|
Kowallik AE, Schweinberger SR. Sensor-Based Technology for Social Information Processing in Autism: A Review. SENSORS (BASEL, SWITZERLAND) 2019; 19:E4787. [PMID: 31689906 PMCID: PMC6864871 DOI: 10.3390/s19214787] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 10/11/2019] [Revised: 10/29/2019] [Accepted: 10/30/2019] [Indexed: 11/16/2022]
Abstract
The prevalence of autism spectrum disorders (ASD) has increased strongly over the past decades, and so has the demand for adequate behavioral assessment and support for persons affected by ASD. Here we provide a review on original research that used sensor technology for an objective assessment of social behavior, either with the aim to assist the assessment of autism or with the aim to use this technology for intervention and support of people with autism. Considering rapid technological progress, we focus (1) on studies published within the last 10 years (2009-2019), (2) on contact- and irritation-free sensor technology that does not constrain natural movement and interaction, and (3) on sensory input from the face, the voice, or body movements. We conclude that sensor technology has already demonstrated its great potential for improving both behavioral assessment and interventions in autism spectrum disorders. We also discuss selected examples for recent theoretical questions related to the understanding of psychological changes and potentials in autism. In addition to its applied potential, we argue that sensor technology-when implemented by appropriate interdisciplinary teams-may even contribute to such theoretical issues in understanding autism.
Collapse
Affiliation(s)
- Andrea E Kowallik
- Early Support and Counselling Center Jena, Herbert Feuchte Stiftungsverbund, 07743 Jena, Germany.
- Social Potential in Autism Research Unit, Friedrich Schiller University, 07743 Jena, Germany.
- Department of General Psychology and Cognitive Neuroscience, Friedrich Schiller University Jena, Am Steiger 3/Haus 1, 07743 Jena, Germany.
| | - Stefan R Schweinberger
- Early Support and Counselling Center Jena, Herbert Feuchte Stiftungsverbund, 07743 Jena, Germany.
- Social Potential in Autism Research Unit, Friedrich Schiller University, 07743 Jena, Germany.
- Department of General Psychology and Cognitive Neuroscience, Friedrich Schiller University Jena, Am Steiger 3/Haus 1, 07743 Jena, Germany.
- Michael Stifel Center Jena for Data-Driven and Simulation Science, Friedrich Schiller University, 07743 Jena, Germany.
- Swiss Center for Affective Science, University of Geneva, 1202 Geneva, Switzerland.
| |
Collapse
|