1
|
El Hmimdi AE, Palpanas T, Kapoula Z. Efficient diagnostic classification of diverse pathologies through contextual eye movement data analysis with a novel hybrid architecture. Sci Rep 2024; 14:21461. [PMID: 39271749 PMCID: PMC11399410 DOI: 10.1038/s41598-024-68056-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Accepted: 07/19/2024] [Indexed: 09/15/2024] Open
Abstract
The analysis of eye movements has proven valuable for understanding brain function and the neuropathology of various disorders. This research aims to utilize eye movement data analysis as a screening tool for differentiation between eight different groups of pathologies, including scholar, neurologic, and postural disorders. Leveraging a dataset from 20 clinical centers, all employing AIDEAL and REMOBI eye movement technologies this study extends prior research by considering a multi-annotation setting, incorporating information from recordings from saccade and vergence eye movement tests, and using contextual information (e.g. target signals and latency of the eye movement relative to the target and confidence level of the quality of eye movement recording) to improve accuracy while reducing noise interference. Additionally, we introduce a novel hybrid architecture that combines the weight-sharing feature of convolution layers with the long-range capabilities of the transformer architecture to improve model efficiency and reduce the computation cost by a factor of 3.36, while still being competitive in terms of macro F1 score. Evaluated on two diverse datasets, our method demonstrates promising results, the most powerful discrimination being Attention & Neurologic; with a macro F1 score of up to 78.8%; disorder. The results indicate the effectiveness of our approach in classifying eye movement data from different pathologies and different clinical centers accurately, thus enabling the creation of an assistant tool in the future.
Collapse
Affiliation(s)
- Alae Eddine El Hmimdi
- Orasis Eye Analytics and Rehabilitation, Paris, France
- Laboratoire d'Informatique Paris Descartes,LIPADE, French University Institute (IUF) Universitá de Paris, 45 Rue Des Saints-Peres, 75006, Paris, France
| | - Themis Palpanas
- Laboratoire d'Informatique Paris Descartes,LIPADE, French University Institute (IUF) Universitá de Paris, 45 Rue Des Saints-Peres, 75006, Paris, France
| | - Zoi Kapoula
- Orasis Eye Analytics and Rehabilitation, Paris, France.
- Laboratoire d'Informatique Paris Descartes,LIPADE, French University Institute (IUF) Universitá de Paris, 45 Rue Des Saints-Peres, 75006, Paris, France.
| |
Collapse
|
2
|
Žugelj N, Peterlin L, Muznik U, Klobučar P, Jaki Mekjavić P, Vidović Valentinčić N, Fakin A. Face Recognition Characteristics in Patients with Age-Related Macular Degeneration Determined Using a Virtual Reality Headset with Eye Tracking. J Clin Med 2024; 13:636. [PMID: 38276142 PMCID: PMC10816606 DOI: 10.3390/jcm13020636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2023] [Revised: 12/28/2023] [Accepted: 01/05/2024] [Indexed: 01/27/2024] Open
Abstract
BACKGROUND AND OBJECTIVES Face recognition is one of the most serious disabilities of patients with age-related macular degeneration (AMD). Our purpose was to study face recognition using a novel method incorporating virtual reality (VR) and eye tracking. MATERIALS AND METHODS Eighteen patients with AMD (seven male; median age 83 years; 89% with bilateral advanced AMD) and nineteen healthy controls (five male; median age 68 years) underwent the face recognition test IC FACES (Synthesius, Ljubljna, Slovenia) on a VR headset with built-in eye tracking sensors. Analysis included recognition accuracy, recognition time and fixation patterns. Additionally, a screening test for dementia and imaging with fundus autofluorescence and optical coherence tomography was performed. RESULTS AMD patients had significantly lower face recognition accuracy (42% vs. 92%; p < 0.001) and longer recognition time (median 4.0 vs. 2.0 s; p < 0.001) in comparison to controls. Both parameters were significantly worse in patients with lower visual acuity. In both groups, eye-tracking data revealed the two classical characteristics of the face recognition process, i.e., fixations clustering mainly in the nose-eyes-mouth triangle and starting observation in the nasal area. CONCLUSIONS The study demonstrates usability of a VR headset with eye tracking for studying visual perception in real-world situations which could be applicable in the design of clinical studies.
Collapse
Affiliation(s)
- Nina Žugelj
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
- Faculty of Medicine, University of Ljubljana, 1000 Ljubljana, Slovenia
| | - Lara Peterlin
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
- Faculty of Medicine, University of Ljubljana, 1000 Ljubljana, Slovenia
| | - Urša Muznik
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
| | - Pia Klobučar
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
| | - Polona Jaki Mekjavić
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
- Faculty of Medicine, University of Ljubljana, 1000 Ljubljana, Slovenia
| | - Nataša Vidović Valentinčić
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
- Faculty of Medicine, University of Ljubljana, 1000 Ljubljana, Slovenia
| | - Ana Fakin
- Eye Hospital, University Medical Centre Ljubljana, 1000 Ljubljana, Slovenia; (N.Ž.); (L.P.); (U.M.); (P.K.); (P.J.M.); (N.V.V.)
- Faculty of Medicine, University of Ljubljana, 1000 Ljubljana, Slovenia
| |
Collapse
|
3
|
Dunn JD, Varela VPL, Nicholls VI, Papinutto M, White D, Miellet S. Face-Information Sampling in Super-Recognizers. Psychol Sci 2022; 33:1615-1630. [PMID: 36044042 DOI: 10.1177/09567976221096320] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Perceptual processes underlying individual differences in face-recognition ability remain poorly understood. We compared visual sampling of 37 adult super-recognizers-individuals with superior face-recognition ability-with that of 68 typical adult viewers by measuring gaze position as they learned and recognized unfamiliar faces. In both phases, participants viewed faces through "spotlight" apertures that varied in size, with face information restricted in real time around their point of fixation. We found higher accuracy in super-recognizers at all aperture sizes-showing that their superiority does not rely on global sampling of face information but is also evident when they are forced to adopt piecemeal sampling. Additionally, super-recognizers made more fixations, focused less on eye region, and distributed their gaze more than typical viewers. These differences were most apparent when learning faces and were consistent with trends we observed across the broader ability spectrum, suggesting that they are reflective of factors that vary dimensionally in the broader population.
Collapse
Affiliation(s)
- James D Dunn
- School of Psychology, University of New South Wales
| | | | - Victoria I Nicholls
- Faculty of Science & Technology, Bournemouth University.,Department of Psychology, University of Cambridge
| | | | - David White
- School of Psychology, University of New South Wales
| | | |
Collapse
|
4
|
Oakes LM, DeBolt MC, Beckner AG, Voss AT, Cantrell LM. Infant Eye Gaze While Viewing Dynamic Faces. Brain Sci 2021; 11:brainsci11020231. [PMID: 33673342 PMCID: PMC7918747 DOI: 10.3390/brainsci11020231] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Revised: 02/05/2021] [Accepted: 02/08/2021] [Indexed: 01/24/2023] Open
Abstract
Research using eye tracking methods has revealed that when viewing faces, between 6 to 10 months of age, infants begin to shift visual attention from the eye region to the mouth region. Moreover, this shift varies with stimulus characteristics and infants’ experience with faces and languages. The current study examined the eye movements of a racially diverse sample of 98 infants between 7.5 and 10.5 months of age as they viewed movies of White and Asian American women reciting a nursery rhyme (the auditory component of the movies was replaced with music to eliminate the influence of the speech on infants’ looking behavior). Using an analytic approach inspired by the multiverse analysis approach, several measures from infants’ eye gaze were examined to identify patterns that were robust across different analyses. Although in general infants preferred the lower regions of the faces, i.e., the region containing the mouth, this preference depended on the stimulus characteristics and was stronger for infants whose typical experience included faces of more races and for infants who were exposed to multiple languages. These results show how we can leverage the richness of eye tracking data with infants to add to our understanding of the factors that influence infants’ visual exploration of faces.
Collapse
Affiliation(s)
- Lisa M. Oakes
- Department of Psychology, University of California, Davis, CA 95616, USA; (M.C.D); (A.G.B.)
- Center for Mind and Brain, University of California, Davis, CA 95618, USA;
- Correspondence:
| | - Michaela C. DeBolt
- Department of Psychology, University of California, Davis, CA 95616, USA; (M.C.D); (A.G.B.)
- Center for Mind and Brain, University of California, Davis, CA 95618, USA;
| | - Aaron G. Beckner
- Department of Psychology, University of California, Davis, CA 95616, USA; (M.C.D); (A.G.B.)
- Center for Mind and Brain, University of California, Davis, CA 95618, USA;
| | - Annika T. Voss
- Center for Mind and Brain, University of California, Davis, CA 95618, USA;
| | - Lisa M. Cantrell
- Department of Child and Adolescent Development, California State University, Sacramento, CA 95819, USA;
| |
Collapse
|
5
|
Meso AI, De Vai RL, Mahabeer A, Hills PJ. Evidence of inverted gravity-driven variation in predictive sensorimotor function. Eur J Neurosci 2020; 52:4803-4823. [PMID: 32730682 DOI: 10.1111/ejn.14926] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2020] [Revised: 07/01/2020] [Accepted: 07/20/2020] [Indexed: 11/30/2022]
Abstract
We move our eyes to place the fovea into the part of a viewed scene currently of interest. Recent evidence suggests that each human has signature patterns of eye movements like handwriting which depend on their sensitivity, allocation of attention and experience. Use of implicit knowledge of how earth's gravity influences object motion has been shown to aid dynamic perception. We used a projected ball-tracking task with a plain background offering no context cues to probe the effect of acquired experience about physical laws of gravitation on performance differences of 44 participants under a simulated gravity and an atypical (upward) antigravity condition. Performance measured by the unsigned difference between instantaneous eye and stimulus positions (RMSE) was consistently worse in the antigravity condition. In the vertical RMSE, participants took about 200 ms longer to improve to the best performance for antigravity compared to gravity trials. The antigravity condition produced a divergence of individual performance which was correlated with levels of questionnaire-based quantified traits of schizotypy but not control traits. Grouping participants by high or low traits revealed a negative relationship between schizotypy trait level and both initiation and maintenance of tracking, a result consistent with trait-related impoverished sensory prediction. The findings confirm for the first time that where cues enabling exact estimation of acceleration are unavailable, knowledge of gravity contributes to dynamic prediction improving motion processing. With acceleration expectations violated, we demonstrate that antigravity tracking could act as a multivariate diagnostic window into predictive brain function.
Collapse
Affiliation(s)
- Andrew Isaac Meso
- Neuroimaging Department, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.,Institut de Neuroscience de la Timone, Team Invibe, CNRS & Aix-Marseille Université, Marseille, 13005, France.,Psychology & Interdisciplinary Neuroscience Group, Bournemouth University, Poole, UK
| | - Robert L De Vai
- Psychology & Interdisciplinary Neuroscience Group, Bournemouth University, Poole, UK
| | - Ashakee Mahabeer
- Psychology & Interdisciplinary Neuroscience Group, Bournemouth University, Poole, UK
| | - Peter J Hills
- Psychology & Interdisciplinary Neuroscience Group, Bournemouth University, Poole, UK
| |
Collapse
|
6
|
Wegner-Clemens K, Rennig J, Beauchamp MS. A relationship between Autism-Spectrum Quotient and face viewing behavior in 98 participants. PLoS One 2020; 15:e0230866. [PMID: 32352984 PMCID: PMC7192493 DOI: 10.1371/journal.pone.0230866] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2019] [Accepted: 03/10/2020] [Indexed: 01/18/2023] Open
Abstract
Faces are one of the most important stimuli that we encounter, but humans vary dramatically in their behavior when viewing a face: some individuals preferentially fixate the eyes, others fixate the mouth, and still others show an intermediate pattern. The determinants of these large individual differences are unknown. However, individuals with Autism Spectrum Disorder (ASD) spend less time fixating the eyes of a viewed face than controls, suggesting the hypothesis that autistic traits in healthy adults might explain individual differences in face viewing behavior. Autistic traits were measured in 98 healthy adults recruited from an academic setting using the Autism-Spectrum Quotient, a validated 50-statement questionnaire. Fixations were measured using a video-based eye tracker while participants viewed two different types of audiovisual movies: short videos of talker speaking single syllables and longer videos of talkers speaking sentences in a social context. For both types of movies, there was a positive correlation between Autism-Spectrum Quotient score and percent of time fixating the lower half of the face that explained from 4% to 10% of the variance in individual face viewing behavior. This effect suggests that in healthy adults, autistic traits are one of many factors that contribute to individual differences in face viewing behavior.
Collapse
Affiliation(s)
- Kira Wegner-Clemens
- Department of Neurosurgery and Core for Advanced MRI, Baylor College of Medicine, Houston, Texas, United States of America
| | - Johannes Rennig
- Department of Neurosurgery and Core for Advanced MRI, Baylor College of Medicine, Houston, Texas, United States of America
| | - Michael S. Beauchamp
- Department of Neurosurgery and Core for Advanced MRI, Baylor College of Medicine, Houston, Texas, United States of America
| |
Collapse
|