1
|
Kim NY, He J, Wu Q, Dai N, Kohlhoff K, Turner J, Paul LK, Kennedy DP, Adolphs R, Navalpakkam V. Smartphone-based gaze estimation for in-home autism research. Autism Res 2024; 17:1140-1148. [PMID: 38660935 DOI: 10.1002/aur.3140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 03/26/2024] [Indexed: 04/26/2024]
Abstract
Atypical gaze patterns are a promising biomarker of autism spectrum disorder. To measure gaze accurately, however, it typically requires highly controlled studies in the laboratory using specialized equipment that is often expensive, thereby limiting the scalability of these approaches. Here we test whether a recently developed smartphone-based gaze estimation method could overcome such limitations and take advantage of the ubiquity of smartphones. As a proof-of-principle, we measured gaze while a small sample of well-assessed autistic participants and controls watched videos on a smartphone, both in the laboratory (with lab personnel) and in remote home settings (alone). We demonstrate that gaze data can be efficiently collected, in-home and longitudinally by participants themselves, with sufficiently high accuracy (gaze estimation error below 1° visual angle on average) for quantitative, feature-based analysis. Using this approach, we show that autistic individuals have reduced gaze time on human faces and longer gaze time on non-social features in the background, thereby reproducing established findings in autism using just smartphones and no additional hardware. Our approach provides a foundation for scaling future research with larger and more representative participant groups at vastly reduced cost, also enabling better inclusion of underserved communities.
Collapse
Affiliation(s)
- Na Yeon Kim
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA
| | - Junfeng He
- Google Research, Mountain View, California, USA
| | - Qianying Wu
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA
| | - Na Dai
- Google Research, Mountain View, California, USA
| | | | - Jasmin Turner
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA
| | - Lynn K Paul
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA
| | - Daniel P Kennedy
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, Indiana, USA
| | - Ralph Adolphs
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, California, USA
- Chen Neuroscience Institute, California Institute of Technology, Pasadena, California, USA
| | | |
Collapse
|
2
|
Keles U, Dubois J, Le KJM, Tyszka JM, Kahn DA, Reed CM, Chung JM, Mamelak AN, Adolphs R, Rutishauser U. Multimodal single-neuron, intracranial EEG, and fMRI brain responses during movie watching in human patients. Sci Data 2024; 11:214. [PMID: 38365977 PMCID: PMC10873379 DOI: 10.1038/s41597-024-03029-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 01/31/2024] [Indexed: 02/18/2024] Open
Abstract
We present a multimodal dataset of intracranial recordings, fMRI, and eye tracking in 20 participants during movie watching. Recordings consist of single neurons, local field potential, and intracranial EEG activity acquired from depth electrodes targeting the amygdala, hippocampus, and medial frontal cortex implanted for monitoring of epileptic seizures. Participants watched an 8-min long excerpt from the video "Bang! You're Dead" and performed a recognition memory test for movie content. 3 T fMRI activity was recorded prior to surgery in 11 of these participants while performing the same task. This NWB- and BIDS-formatted dataset includes spike times, field potential activity, behavior, eye tracking, electrode locations, demographics, and functional and structural MRI scans. For technical validation, we provide signal quality metrics, assess eye tracking quality, behavior, the tuning of cells and high-frequency broadband power field potentials to familiarity and event boundaries, and show brain-wide inter-subject correlations for fMRI. This dataset will facilitate the investigation of brain activity during movie watching, recognition memory, and the neural basis of the fMRI-BOLD signal.
Collapse
Affiliation(s)
- Umit Keles
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Julien Dubois
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Kevin J M Le
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - J Michael Tyszka
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - David A Kahn
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Chrystal M Reed
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Jeffrey M Chung
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Adam N Mamelak
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ralph Adolphs
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
| | - Ueli Rutishauser
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Center for Neural Science and Medicine, Department of Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
| |
Collapse
|
3
|
Bleimeister I, Avni I, Granovetter M, Meiri G, Ilan M, Michaelovski A, Menashe I, Behrmann M, Dinstein I. Idiosyncratic pupil regulation in autistic children. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.10.575072. [PMID: 38260528 PMCID: PMC10802609 DOI: 10.1101/2024.01.10.575072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Recent neuroimaging and eye tracking studies have suggested that children with autism spectrum disorder (ASD) may exhibit more variable and idiosyncratic brain responses and eye movements than typically developing (TD) children. Here we extended this research for the first time to pupillometry recordings. We successfully completed pupillometry recordings with 103 children (66 with ASD), 4.5-years-old on average, who viewed three 90 second movies, twice. We extracted their pupillary time-course for each movie, capturing their stimulus evoked pupillary responses. We then computed the correlation between the time-course of each child and those of all others in their group. This yielded an average inter-subject correlation value per child, representing how similar their pupillary responses were to all others in their group. ASD participants exhibited significantly weaker inter-subject correlations than TD participants, reliably across all three movies. Differences across groups were largest in responses to a naturalistic movie containing footage of a social interaction between two TD children. This measure enabled classification of ASD and TD children with a sensitivity of 0.82 and specificity of 0.73 when trained and tested on independent datasets. Using the largest ASD pupillometry dataset to date, we demonstrate the utility of a new technique for measuring the idiosyncrasy of pupil regulation, which can be completed even by young children with co-occurring intellectual disability. These findings reveal that a considerable subgroup of ASD children have significantly more unstable, idiosyncratic pupil regulation than TD children, indicative of more variable, weakly regulated, underlying neural activity.
Collapse
Affiliation(s)
- Isabel Bleimeister
- Psychology Department, Ben Gurion University of the Negev, Beer Sheva, Israel 84105
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
| | - Inbar Avni
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
- Cognitive and Brain Sciences Department, Ben Gurion University of the Negev, Beer Sheva, Israel
| | - Michael Granovetter
- Department of Ophthalmology, University of Pittsburgh, Pittsburgh, U.S.A 15213
| | - Gal Meiri
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
- Pre-school Psychiatry Unit, Soroka Medical Center, Beer Sheva, Israel 84105
| | - Michal Ilan
- Psychology Department, Ben Gurion University of the Negev, Beer Sheva, Israel 84105
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
- Pre-school Psychiatry Unit, Soroka Medical Center, Beer Sheva, Israel 84105
| | - Analya Michaelovski
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
- Child Development Institute, Soroka Medical Center, Beer Sheva, Israel 84105
| | - Idan Menashe
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
- Public Health Department, Ben-Gurion University, Beer Sheva, Israel 84105
| | - Marlene Behrmann
- Department of Ophthalmology, University of Pittsburgh, Pittsburgh, U.S.A 15213
| | - Ilan Dinstein
- Psychology Department, Ben Gurion University of the Negev, Beer Sheva, Israel 84105
- Azrieli National Centre for Autism and Neurodevelopment Research, Ben Gurion University of the Negev, Beer Sheva, Israel
- Cognitive and Brain Sciences Department, Ben Gurion University of the Negev, Beer Sheva, Israel
| |
Collapse
|
4
|
Bloch C, Viswanathan S, Tepest R, Jording M, Falter-Wagner CM, Vogeley K. Differentiated, rather than shared, strategies for time-coordinated action in social and non-social domains in autistic individuals. Cortex 2023; 166:207-232. [PMID: 37393703 DOI: 10.1016/j.cortex.2023.05.008] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Revised: 05/15/2023] [Accepted: 05/19/2023] [Indexed: 07/04/2023]
Abstract
Autism spectrum disorder (ASD) is a neurodevelopmental condition with a highly heterogeneous adult phenotype that includes social and non-social behavioral characteristics. The link between the characteristics assignable to the different domains remains unresolved. One possibility is that social and non-social behaviors in autism are modulated by a common underlying deficit. However, here we report evidence supporting an alternative concept that is individual-centered rather than deficit-centered. Individuals are assumed to have a distinctive style in the strategies they adopt to perform social and non-social tasks with these styles presumably being structured differently between autistic individuals and typically-developed (TD) individuals. We tested this hypothesis for the execution of time-coordinated (synchronized) actions. Participants performed (i) a social task that required synchronized gaze and pointing actions to interact with another person, and (ii) a non-social task that required finger-tapping actions synchronized to periodic stimuli at different time-scales and sensory modalities. In both tasks, synchronization behavior differed between ASD and TD groups. However, a principal component analysis of individual behaviors across tasks revealed associations between social and non-social features for the TD persons but such cross-domain associations were strikingly absent for autistic individuals. The highly differentiated strategies between domains in ASD are inconsistent with a general synchronization deficit and instead highlight the individualized developmental heterogeneity in the acquisition of domain-specific behaviors. We propose a cognitive model to help disentangle individual-centered from deficit-centered effects in other domains. Our findings reinforce the importance to identify individually differentiated phenotypes to personalize autism therapies.
Collapse
Affiliation(s)
- Carola Bloch
- Department of Psychiatry and Psychotherapy, LMU University Hospital, LMU Munich, Munich, Germany; Department of Psychiatry, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany.
| | - Shivakumar Viswanathan
- Cognitive Neuroscience, Institute of Neuroscience and Medicine (INM-3), Forschungszentrum Jülich, Jülich, Germany
| | - Ralf Tepest
- Department of Psychiatry, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| | - Mathis Jording
- Cognitive Neuroscience, Institute of Neuroscience and Medicine (INM-3), Forschungszentrum Jülich, Jülich, Germany
| | | | - Kai Vogeley
- Department of Psychiatry, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany; Cognitive Neuroscience, Institute of Neuroscience and Medicine (INM-3), Forschungszentrum Jülich, Jülich, Germany
| |
Collapse
|