1
|
El Hmimdi AE, Palpanas T, Kapoula Z. Efficient diagnostic classification of diverse pathologies through contextual eye movement data analysis with a novel hybrid architecture. Sci Rep 2024; 14:21461. [PMID: 39271749 PMCID: PMC11399410 DOI: 10.1038/s41598-024-68056-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Accepted: 07/19/2024] [Indexed: 09/15/2024] Open
Abstract
The analysis of eye movements has proven valuable for understanding brain function and the neuropathology of various disorders. This research aims to utilize eye movement data analysis as a screening tool for differentiation between eight different groups of pathologies, including scholar, neurologic, and postural disorders. Leveraging a dataset from 20 clinical centers, all employing AIDEAL and REMOBI eye movement technologies this study extends prior research by considering a multi-annotation setting, incorporating information from recordings from saccade and vergence eye movement tests, and using contextual information (e.g. target signals and latency of the eye movement relative to the target and confidence level of the quality of eye movement recording) to improve accuracy while reducing noise interference. Additionally, we introduce a novel hybrid architecture that combines the weight-sharing feature of convolution layers with the long-range capabilities of the transformer architecture to improve model efficiency and reduce the computation cost by a factor of 3.36, while still being competitive in terms of macro F1 score. Evaluated on two diverse datasets, our method demonstrates promising results, the most powerful discrimination being Attention & Neurologic; with a macro F1 score of up to 78.8%; disorder. The results indicate the effectiveness of our approach in classifying eye movement data from different pathologies and different clinical centers accurately, thus enabling the creation of an assistant tool in the future.
Collapse
Affiliation(s)
- Alae Eddine El Hmimdi
- Orasis Eye Analytics and Rehabilitation, Paris, France
- Laboratoire d'Informatique Paris Descartes,LIPADE, French University Institute (IUF) Universitá de Paris, 45 Rue Des Saints-Peres, 75006, Paris, France
| | - Themis Palpanas
- Laboratoire d'Informatique Paris Descartes,LIPADE, French University Institute (IUF) Universitá de Paris, 45 Rue Des Saints-Peres, 75006, Paris, France
| | - Zoi Kapoula
- Orasis Eye Analytics and Rehabilitation, Paris, France.
- Laboratoire d'Informatique Paris Descartes,LIPADE, French University Institute (IUF) Universitá de Paris, 45 Rue Des Saints-Peres, 75006, Paris, France.
| |
Collapse
|
2
|
Werth R. Revealing the Causes of Dyslexia through a Differential Diagnosis, a Short-Term Effective Treatment and an Appropriate Conceptual Framework. Diagnostics (Basel) 2024; 14:1965. [PMID: 39272749 PMCID: PMC11393927 DOI: 10.3390/diagnostics14171965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2024] [Revised: 08/27/2024] [Accepted: 09/02/2024] [Indexed: 09/15/2024] Open
Abstract
Various different impairments and their interactions can cause reading problems referred to as "dyslexia". Since reading requires the interaction of many abilities, the impairment of each of these abilities can result in dyslexia. Therefore, the diagnosis must differentiate various kinds of dyslexia. The diagnosis of a certain kind of dyslexia cannot be delimited to the investigation and description of symptoms but must also include the investigation of the causes of each kind of dyslexia. For this purpose, a scientifically unequivocal concept of causation and appropriate methods are needed to distinguish them from co-existing impairments that have no causal influence on reading performance. The results of applying these methods cannot be adequately accounted for by a non-scientific, intuitive understanding of necessary and sufficient conditions and causation. The methods suitable for revealing the causes of dyslexia are described in detail, and the results of applying these methods in experiments, in which 356 children with developmental dyslexia participated, are reviewed. Since the concepts of "necessary" and "sufficient" conditions and "causation" proposed in the philosophy of science are not suitable for describing causes of dyslexia and their interaction, they are replaced by a more detailed, experimentally based conceptual framework that provides an accurate description of the conditions required for correct reading and the causes of dyslexia.
Collapse
Affiliation(s)
- Reinhard Werth
- Institute for Social Pediatrics and Adolescent Medicine, Ludwig-Maximilians-University of Munich, Haydnstr. 5, D-80336 Munich, Germany
| |
Collapse
|
3
|
Werth R. Dyslexia Due to Visual Impairments. Biomedicines 2023; 11:2559. [PMID: 37760998 PMCID: PMC10526907 DOI: 10.3390/biomedicines11092559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 08/28/2023] [Accepted: 09/01/2023] [Indexed: 09/29/2023] Open
Abstract
Reading involves many different abilities that are necessary or sufficient conditions for fluent and flawless reading. The absence of one necessary or of all sufficient conditions is a cause of dyslexia. The present study investigates whether too short fixation times and an impaired ability to recognize a string of letters simultaneously are causes of dyslexia. The frequency and types of reading mistakes were investigated in a tachistoscopic pseudoword experiment with 100 children with dyslexia to test the impact of too short fixation times and the attempts of children with dyslexia to recognize more letters simultaneously than they can when reading pseudowords. The experiment demonstrates that all types of reading mistakes disappear when the fixation time increases and/or the number of letters that the children try to recognize simultaneously is reduced. The results cannot be interpreted as being due to altered visual crowding, impaired attention, or impaired phonological awareness, but can be regarded as an effect of impaired temporal summation and a dysfunction in the ventral stream of the visual system.
Collapse
Affiliation(s)
- Reinhard Werth
- Institute for Social Pediatrics and Adolescent Medicine, Ludwig-Maximilians-University of Munich, Haydnstr. 5, D-80336 München, Germany
| |
Collapse
|
4
|
Justino J, Kolinsky R. Eye movements during reading in beginning and skilled readers: Impact of reading level or physiological maturation? Acta Psychol (Amst) 2023; 236:103927. [PMID: 37126894 DOI: 10.1016/j.actpsy.2023.103927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 04/19/2023] [Accepted: 04/24/2023] [Indexed: 05/03/2023] Open
Abstract
We begin by presenting and examining relevant data in the literature on eye movements in reading, from childhood to adulthood. In particular, we discuss the differences found in eye movements during reading between children in different age groups and with different reading levels and skilled adult readers in terms of word recognition and sentence processing. We then critically discuss two hypotheses that account for the differences between children and adults' eye movement during reading: one being reading age itself - the changes in eye movement patterns in reading are regulated by the level of reading proficiency and its automatization - and the other being the role of maturation of oculomotor control and, consequently, its possible changes in eye movement patterns during reading. Finally, we list gaps in the research field and suggest that future research will benefit from investigating eye movements during reading in ex-illiterate adults who are in the process of learning to read in order to isolate both reading and maturational factors.
Collapse
Affiliation(s)
- Julia Justino
- Fonds de la Recherche Scientifique - FNRS (FRS-FNRS), Belgium; Unité de Recherche en Neurosciences Cognitives (Unescog/CRCN), Université Libre de Bruxelles, Belgium.
| | - Régine Kolinsky
- Fonds de la Recherche Scientifique - FNRS (FRS-FNRS), Belgium; Unité de Recherche en Neurosciences Cognitives (Unescog/CRCN), Université Libre de Bruxelles, Belgium.
| |
Collapse
|
5
|
Werth R. Dyslexia: Causes and Concomitant Impairments. Brain Sci 2023; 13:brainsci13030472. [PMID: 36979282 PMCID: PMC10046374 DOI: 10.3390/brainsci13030472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Revised: 02/07/2023] [Accepted: 03/06/2023] [Indexed: 03/12/2023] Open
Abstract
In recent decades, theories have been presented to explain the nature of dyslexia, but the causes of dyslexia remained unclear. Although the investigation of the causes of dyslexia presupposes a clear understanding of the concept of cause, such an understanding is missing. The present paper proposes the absence of at least one necessary condition or the absence of all sufficient conditions as causes for impaired reading. The causes of impaired reading include: an incorrect fixation location, too short a fixation time, the attempt to recognize too many letters simultaneously, too large saccade amplitudes, and too short verbal reaction times. It is assumed that a longer required fixation time in dyslexic readers results from a functional impairment of areas V1, V2, and V3 that require more time to complete temporal summation. These areas and areas that receive input from them, such as the fusiform gyrus, are assumed to be impaired in their ability to simultaneously process a string of letters. When these impairments are compensated by a new reading strategy, reading ability improves immediately.
Collapse
Affiliation(s)
- Reinhard Werth
- Institute for Social Pediatrics and Adolescent Medicine, Ludwig-Maximilians-University of Munich, Haydnstr. 5, D-80336 München, Germany
| |
Collapse
|
6
|
El Hmimdi AE, Ward LM, Palpanas T, Sainte Fare Garnot V, Kapoula Z. Predicting Dyslexia in Adolescents from Eye Movements during Free Painting Viewing. Brain Sci 2022; 12:1031. [PMID: 36009094 PMCID: PMC9405842 DOI: 10.3390/brainsci12081031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 07/04/2022] [Accepted: 07/05/2022] [Indexed: 02/04/2023] Open
Abstract
It is known that dyslexics present eye movement abnormalities. Previously, we have shown that eye movement abnormalities during reading or during saccade and vergence testing can predict dyslexia successfully. The current study further examines this issue focusing on eye movements during free exploration of paintings; the dataset was provided by a study in our laboratory carried by Ward and Kapoula. Machine learning (ML) classifiers were applied to eye movement features extracted by the software AIDEAL: a velocity threshold analysis reporting amplitude speed and disconjugacy of horizontal saccades. In addition, a new feature was introduced that concerns only the very short periods during which the eyes were moving, one to the left the other to the right; such periods occurred mostly during fixations between saccades; we calculated a global index of the frequency of such disconjugacy segments, of their duration and their amplitude. Such continuous evaluation of disconjugacy throughout the time series of eye movements differs from the disconjugacy feature that describes inequality of the saccade amplitude between the two eyes. The results show that both AIDEAL features, and the Disconjugacy Global Index (DGI) enable successful categorization of dyslexics from non-dyslexics, at least when applying this analysis to the specific paintings used in the present study. We suggest that this high power of predictability arises from both the content of the paintings selected and the physiologic relevance of eye movement features extracted by the AIDEAL and the DGI.
Collapse
Affiliation(s)
- Alae Eddine El Hmimdi
- Orasis Eye Analytics and Rehabilitation, CNRS Spinoff up, 12 rue Lacretelle, 75015 Paris, France; (A.E.E.H.); (V.S.F.G.)
- LIPADE, French University Institute (IUF), Laboratoire d’Informatique Paris Descartes, University of Paris, 45 Rue Des Saints-Pères, 75006 Paris, France;
| | - Lindsey M Ward
- IRIS Lab, Neurophysiology of Binocular Motor Control and Vision, CNRS UAR 2022, University of Paris, 45 rue des Saints Pères, 75006 Paris, France;
| | - Themis Palpanas
- LIPADE, French University Institute (IUF), Laboratoire d’Informatique Paris Descartes, University of Paris, 45 Rue Des Saints-Pères, 75006 Paris, France;
| | - Vivien Sainte Fare Garnot
- Orasis Eye Analytics and Rehabilitation, CNRS Spinoff up, 12 rue Lacretelle, 75015 Paris, France; (A.E.E.H.); (V.S.F.G.)
| | - Zoï Kapoula
- Orasis Eye Analytics and Rehabilitation, CNRS Spinoff up, 12 rue Lacretelle, 75015 Paris, France; (A.E.E.H.); (V.S.F.G.)
- IRIS Lab, Neurophysiology of Binocular Motor Control and Vision, CNRS UAR 2022, University of Paris, 45 rue des Saints Pères, 75006 Paris, France;
| |
Collapse
|
7
|
Ward LM, Kapoula Z. Creativity, Eye-Movement Abnormalities, and Aesthetic Appreciation of Magritte’s Paintings. Brain Sci 2022; 12:brainsci12081028. [PMID: 36009091 PMCID: PMC9406106 DOI: 10.3390/brainsci12081028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2022] [Revised: 07/19/2022] [Accepted: 07/26/2022] [Indexed: 11/18/2022] Open
Abstract
Dyslexic children have been shown to be more creative than their non-dyslexic counterparts. They have also been shown to have an abnormal oculomotor profile while viewing targets in free space, making vergence or saccadic eye movements while reading or when viewing Op art. They show a slower deceleration of their eye movements and a difficulty in coordinating their two eyes to obtain single fused vision in depth. Interestingly, their abnormal oculo-motor profile is exacerbated while reading more difficult texts. Given these differences, we postulate that dyslexics’ increased creativity may be related to their different eye movement control affecting how they perceive the world. Therefore, we decided to measure adolescent dyslexics’ creativity, oculomotor profile, and subjective responses while they viewed three paintings by Magritte. These were chosen to stimulate the perception of hidden conceptual spaces or stimulate conflict between the perception of the figural and textural content. For the first time to our knowledge, dyslexic adolescents were demonstrated to be more creative in terms of flexibility and fluidity than their non-dyslexic peers. Subjectively, while viewing the Magritte paintings, dyslexics reported fewer conceptual spaces and fewer hidden words than their non-dyslexic peers; thus, they confabulated less than non-dyslexics. Dyslexics also demonstrated an abnormal oculomotor profile similar to those that we have shown when reading, viewing randomized targets, and while perceiving illusions of depth in Op art paintings, in that they demonstrated difficulty with disconjugation and abnormalities in their eye velocity profiles. We propose there may be a link between dyslexic increased creativity and their eye movement abnormalities. Similar to reading nonsense text, we propose that Magritte’s contradictory paintings exacerbate dyslexics’ eye movement abnormalities. These eye movement abnormalities while viewing these particular paintings might provide a physiological signature suggesting a contribution of their unusual eye control to their higher creativity scores.
Collapse
|
8
|
Ward LM, Kapoula Z. Disconjugate Eye Movements in Dyslexic Adolescents While Viewing Op Art: A Creative Handicap? Brain Sci 2022; 12:835. [PMID: 35884642 PMCID: PMC9312852 DOI: 10.3390/brainsci12070835] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 06/09/2022] [Accepted: 06/15/2022] [Indexed: 11/16/2022] Open
Abstract
Op art was created, in part, to produce illusions of movement. Given that dyslexics have been shown to have impaired visuo-postural axis deficits, it may be possible that dyslexics see illusions different than their non-dyslexic peers. To test this theory, we measured eye movement and posture in 47 dyslexic (18 female, 29 male; mean age 15.4) and 44 non dyslexic (22 female, 22 male; mean age 14.8) adolescents while they viewed three works of art by Op artist Bridget Riley. They then responded to a questionnaire about how they felt while viewing the artworks. Dyslexics demonstrated significantly slower saccades in terms of average velocity that was particularly disturbed in paintings that manipulated depth. Subjectively, dyslexics felt much more destabilized compared to their peers; however, there was not a significant difference in objective postural measurements between the two groups. The sensation of destabilization was positively correlated with appreciation in non-dyslexic adolescents. These subjective results suggest that dyslexics may be more sensitive to movement in depth, which could be related to the instability in vergence movements. Whereas this instability represents a hinderance in relation to reading, it could be an advantage while viewing paintings such as these.
Collapse
Affiliation(s)
| | - Zoi Kapoula
- IRIS Laboratory, Neurophysiology of Binocular Motor Control and Vision, CNRS UAR 2022 Neurosciences, UFR Biomedical, University of Paris, 45 Rue des Saints Pères, 75006 Paris, France;
| |
Collapse
|
9
|
El Hmimdi AE, Ward LM, Palpanas T, Kapoula Z. Predicting Dyslexia and Reading Speed in Adolescents from Eye Movements in Reading and Non-Reading Tasks: A Machine Learning Approach. Brain Sci 2021; 11:brainsci11101337. [PMID: 34679400 PMCID: PMC8534067 DOI: 10.3390/brainsci11101337] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Revised: 08/09/2021] [Accepted: 08/13/2021] [Indexed: 11/16/2022] Open
Abstract
There is evidence that abnormalities in eye movements exist during reading in dyslexic individuals. A few recent studies applied Machine Learning (ML) classifiers to such eye movement data to predict dyslexia. A general problem with these studies is that eye movement data sets are limited to reading saccades and fixations that are confounded by reading difficulty, e.g., it is unclear whether abnormalities are the consequence or the cause of reading difficulty. Recently, Ward and Kapoula used LED targets (with the REMOBI & AIDEAL method) to demonstrate abnormalities of large saccades and vergence eye movements in depth demonstrating intrinsic eye movement problems independent from reading in dyslexia. In another study, binocular eye movements were studied while reading two texts: one using the "Alouette" text, which has no meaning and requires word decoding, the other using a meaningful text. It was found the Alouette text exacerbates eye movement abnormalities in dyslexics. In this paper, we more precisely quantify the quality of such eye movement descriptors for dyslexia detection. We use the descriptors produced in the four different setups as input to multiple classifiers and compare their generalization performances. Our results demonstrate that eye movement data from the Alouette test predicts dyslexia with an accuracy of 81.25%; similarly, we were able to predict dyslexia with an accuracy of 81.25% when using data from saccades to LED targets on the Remobi device and 77.3% when using vergence movements to LED targets. Noticeably, eye movement data from the meaningful text produced the lowest accuracy (70.2%). In a subsequent analysis, ML algorithms were applied to predict reading speed based on eye movement descriptors extracted from the meaningful reading, then from Remobi saccade and vergence tests. Remobi vergence eye movement descriptors can predict reading speed even better than eye movement descriptors from the meaningful reading test.
Collapse
Affiliation(s)
- Alae Eddine El Hmimdi
- Orasis Eye Analytics & Rehabilitation, CNRS Spinoff up, 12 Rue Lacretelle, 75015 Paris, France;
- LIPADE, French University Institute (IUF) Laboratoire d’Informatique Paris Descartes, Université de Paris, 45 Rue Des Saints-Peres, 75006 Paris, France;
| | - Lindsey M Ward
- IRIS Lab, Neurophysiology of Binocular Motor Control and Vision, CNRS UAR 2022 Neurosciences, UFR Biomedical, University of Paris, 45 Rue des Saints Pères, 75006 Paris, France;
| | - Themis Palpanas
- LIPADE, French University Institute (IUF) Laboratoire d’Informatique Paris Descartes, Université de Paris, 45 Rue Des Saints-Peres, 75006 Paris, France;
| | - Zoï Kapoula
- Orasis Eye Analytics & Rehabilitation, CNRS Spinoff up, 12 Rue Lacretelle, 75015 Paris, France;
- IRIS Lab, Neurophysiology of Binocular Motor Control and Vision, CNRS UAR 2022 Neurosciences, UFR Biomedical, University of Paris, 45 Rue des Saints Pères, 75006 Paris, France;
- Correspondence: ; Tel.: +33-142-864-039
| |
Collapse
|