1
|
Fu X, Franchak JM, MacNeill LA, Gunther KE, Borjon JI, Yurkovic-Harding J, Harding S, Bradshaw J, Pérez-Edgar KE. Implementing mobile eye tracking in psychological research: A practical guide. Behav Res Methods 2024:10.3758/s13428-024-02473-6. [PMID: 39147949 DOI: 10.3758/s13428-024-02473-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/20/2024] [Indexed: 08/17/2024]
Abstract
Eye tracking provides direct, temporally and spatially sensitive measures of eye gaze. It can capture visual attention patterns from infancy through adulthood. However, commonly used screen-based eye tracking (SET) paradigms are limited in their depiction of how individuals process information as they interact with the environment in "real life". Mobile eye tracking (MET) records participant-perspective gaze in the context of active behavior. Recent technological developments in MET hardware enable researchers to capture egocentric vision as early as infancy and across the lifespan. However, challenges remain in MET data collection, processing, and analysis. The present paper aims to provide an introduction and practical guide to starting researchers in the field to facilitate the use of MET in psychological research with a wide range of age groups. First, we provide a general introduction to MET. Next, we briefly review MET studies in adults and children that provide new insights into attention and its roles in cognitive and socioemotional functioning. We then discuss technical issues relating to MET data collection and provide guidelines for data quality inspection, gaze annotations, data visualization, and statistical analyses. Lastly, we conclude by discussing the future directions of MET implementation. Open-source programs for MET data quality inspection, data visualization, and analysis are shared publicly.
Collapse
Affiliation(s)
- Xiaoxue Fu
- Department of Psychology, University of South Carolina, Columbia, SC, USA.
| | - John M Franchak
- Department of Psychology, University of California Riverside, Riverside, CA, USA
| | - Leigha A MacNeill
- Department of Medical Social Sciences, Northwestern University, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Institute for Innovations in Developmental Sciences, Northwestern University, Evanston, IL, USA
| | - Kelley E Gunther
- Neuroscience and Cognitive Science Program, University of Maryland, College Park, MD, USA
| | - Jeremy I Borjon
- Department of Psychology, University of Houston, Houston, TX, USA
- Texas Institute for Measurement, Evaluation, and Statistics, University of Houston, Houston, TX, USA
- Texas Center for Learning Disorders, University of Houston, Houston, TX, USA
| | | | - Samuel Harding
- Department of Psychology, University of South Carolina, Columbia, SC, USA
| | - Jessica Bradshaw
- Department of Psychology, University of South Carolina, Columbia, SC, USA
| | - Koraly E Pérez-Edgar
- Department of Psychology, The Pennsylvania State University, University Park, PA, USA
| |
Collapse
|
2
|
Grasso-Cladera A, Bremer M, Ladouce S, Parada F. A systematic review of mobile brain/body imaging studies using the P300 event-related potentials to investigate cognition beyond the laboratory. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024; 24:631-659. [PMID: 38834886 DOI: 10.3758/s13415-024-01190-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/15/2024] [Indexed: 06/06/2024]
Abstract
The P300 ERP component, related to the onset of task-relevant or infrequent stimuli, has been widely used in the Mobile Brain/Body Imaging (MoBI) literature. This systematic review evaluates the quality and breadth of P300 MoBI studies, revealing a maturing field with well-designed research yet grappling with standardization and global representation challenges. While affirming the reliability of measuring P300 ERP components in mobile settings, the review identifies significant hurdles in standardizing data cleaning and processing techniques, impacting comparability and reproducibility. Geographical disparities emerge, with studies predominantly in the Global North and a dearth of research from the Global South, emphasizing the need for broader inclusivity to counter the WEIRD bias in psychology. Collaborative projects and mobile EEG systems showcase the feasibility of reaching diverse populations, which is essential to advance precision psychiatry and to integrate varied data streams. Methodologically, a trend toward ecological validity is noted, shifting from lab-based to real-world settings with portable EEG system advancements. Future hardware developments are expected to balance signal quality and sensor intrusiveness, enriching data collection in everyday contexts. Innovative methodologies reflect a move toward more natural experimental settings, prompting critical questions about the applicability of traditional ERP markers, such as the P300 outside structured paradigms. The review concludes by highlighting the crucial role of integrating mobile technologies, physiological sensors, and machine learning to advance cognitive neuroscience. It advocates for an operational definition of ecological validity to bridge the gap between controlled experiments and the complexity of embodied cognitive experiences, enhancing both theoretical understanding and practical application in study design.
Collapse
Affiliation(s)
| | - Marko Bremer
- Facultad de Psicología, Centro de Estudios en Neurociencia Humana y Neuropsicología (CENHN), Diego Portales University, Santiago, Chile
- Facultad de Psicología, Programa de Magíster en Neurociencia Social, Diego Portales University, Santiago, Chile
| | - Simon Ladouce
- Department Brain and Cognition, Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | - Francisco Parada
- Facultad de Psicología, Centro de Estudios en Neurociencia Humana y Neuropsicología (CENHN), Diego Portales University, Santiago, Chile.
| |
Collapse
|
3
|
Wohltjen S, Wheatley T. Interpersonal eye-tracking reveals the dynamics of interacting minds. Front Hum Neurosci 2024; 18:1356680. [PMID: 38532792 PMCID: PMC10963423 DOI: 10.3389/fnhum.2024.1356680] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2023] [Accepted: 02/20/2024] [Indexed: 03/28/2024] Open
Abstract
The human eye is a rich source of information about where, when, and how we attend. Our gaze paths indicate where and what captures our attention, while changes in pupil size can signal surprise, revealing our expectations. Similarly, the pattern of our blinks suggests levels of alertness and when our attention shifts between external engagement and internal thought. During interactions with others, these cues reveal how we coordinate and share our mental states. To leverage these insights effectively, we need accurate, timely methods to observe these cues as they naturally unfold. Advances in eye-tracking technology now enable real-time observation of these cues, shedding light on mutual cognitive processes that foster shared understanding, collaborative thought, and social connection. This brief review highlights these advances and the new opportunities they present for future research.
Collapse
Affiliation(s)
- Sophie Wohltjen
- Department of Psychology, University of Wisconsin–Madison, Madison, WI, United States
| | - Thalia Wheatley
- Department of Psychological and Brain Sciences, Consortium for Interacting Minds, Dartmouth College, Hanover, NH, United States
- Santa Fe Institute, Santa Fe, NM, United States
| |
Collapse
|
4
|
Dilbeck MD, Gentry TN, Economides JR, Horton JC. Quotidian Profile of Vergence Angle in Ambulatory Subjects Monitored With Wearable Eye Tracking Glasses. Transl Vis Sci Technol 2023; 12:17. [PMID: 36780142 PMCID: PMC9927788 DOI: 10.1167/tvst.12.2.17] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/14/2023] Open
Abstract
Purpose Wearable eye trackers record gaze position as ambulatory subjects navigate their environment. Tobii Pro Glasses 3 were tested to assess their accuracy and precision in the measurement of vergence angle. Methods Four subjects wore the eye tracking glasses, with their head stabilized, while fixating at a series of distances corresponding to vergence demands of: 0.25, 0.50, 1, 2, 4, 8, 16, and 32°. After these laboratory trials were completed, 10 subjects wore the glasses for a prolonged period while carrying out their customary daily pursuits. A vergence profile was compiled for each subject and compared with interpupillary distance. Results In the laboratory, the eye tracking glasses were comparable in accuracy to remote video eye trackers, outputting a mean vergence value within 1° of demand at all angles except 32°. In ambulatory subjects, the glasses were less accurate, due to tracking interruptions and measurement errors, partly mitigated by the application of data filters. Nonetheless, a useful record of vergence behavior was obtained in every subject. Vergence profiles often had a bimodal distribution, reflecting a preponderance of activities at near (mobile phone and computer) or far (driving and walking). As expected, vergence angle correlated with interpupillary distance. Conclusions Wearable eye tracking glasses make it possible to compile a nearly continuous record of vergence angle over hours, which can be correlated with the corresponding visual scene viewed by ambulatory subjects. Translational Relevance This technology provides new insight into the diversity of human ocular motor behavior and may become useful for the diagnosis of disorders that affect vergence function such as: convergence insufficiency, Parkinson disease, and strabismus.
Collapse
Affiliation(s)
- Mikayla D. Dilbeck
- Program in Neuroscience, Department of Ophthalmology, University of California, San Francisco, San Francisco, CA, USA
| | - Thomas N. Gentry
- Program in Neuroscience, Department of Ophthalmology, University of California, San Francisco, San Francisco, CA, USA
| | - John R. Economides
- Program in Neuroscience, Department of Ophthalmology, University of California, San Francisco, San Francisco, CA, USA
| | - Jonathan C. Horton
- Program in Neuroscience, Department of Ophthalmology, University of California, San Francisco, San Francisco, CA, USA
| |
Collapse
|
5
|
Ghiani A, Van Hout LR, Driessen JG, Brenner E. Where do people look when walking up and down familiar staircases? J Vis 2023; 23:7. [PMID: 36633872 PMCID: PMC9840440 DOI: 10.1167/jov.23.1.7] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023] Open
Abstract
Many activities in daily life do not impose strict requirements on gaze. We investigated gaze when walking up and down staircases within one's own house. We anticipated that using a variety of staircases in different environments and not informing participants that stair climbing was the focus of investigation might provide a description of gaze behavior that is closer to that used in our daily life than doing so under circumstances in which the focus is explicitly and exclusively directed at the stairs. We analyzed several measures, including the order in which participants fixated the steps. We confirmed that people often look at the steps sequentially, but found that they often made fixations back to steps they had already fixated. They also regularly skipped looking at several steps to fixate further ahead. On average, they directed their gaze at about half the steps. They looked further ahead when ascending than when descending staircases. Overall, the results are similar to those found under highly constrained laboratory conditions, although we do report some differences. One such difference is a tendency to fixate fewer steps. Another is that participants fixated steps that were less far ahead when descending staircases. We also introduced some new analyses that may help understand gaze behavior during stair climbing.
Collapse
Affiliation(s)
- Andrea Ghiani
- Department of Movement Sciences, Vrije Universiteit Amsterdam, The Netherlands.,
| | - Liz R. Van Hout
- Department of Movement Sciences, Vrije Universiteit Amsterdam, The Netherlands,
| | - Joost G. Driessen
- Department of Movement Sciences, Vrije Universiteit Amsterdam, The Netherlands,
| | - Eli Brenner
- Department of Movement Sciences, Vrije Universiteit Amsterdam, The Netherlands.,
| |
Collapse
|
6
|
Eye contact avoidance in crowds: A large wearable eye-tracking study. Atten Percept Psychophys 2022; 84:2623-2640. [PMID: 35996058 PMCID: PMC9630249 DOI: 10.3758/s13414-022-02541-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/08/2022] [Indexed: 11/30/2022]
Abstract
Eye contact is essential for human interactions. We investigated whether humans are able to avoid eye contact while navigating crowds. At a science festival, we fitted 62 participants with a wearable eye tracker and instructed them to walk a route. Half of the participants were further instructed to avoid eye contact. We report that humans can flexibly allocate their gaze while navigating crowds and avoid eye contact primarily by orienting their head and eyes towards the floor. We discuss implications for crowd navigation and gaze behavior. In addition, we address a number of issues encountered in such field studies with regard to data quality, control of the environment, and participant adherence to instructions. We stress that methodological innovation and scientific progress are strongly interrelated.
Collapse
|
7
|
Beyond screen time: Using head-mounted eye tracking to study natural behavior. ADVANCES IN CHILD DEVELOPMENT AND BEHAVIOR 2022; 62:61-91. [PMID: 35249686 DOI: 10.1016/bs.acdb.2021.11.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Head-mounted eye tracking is a new method that allows researchers to catch a glimpse of what infants and children see during naturalistic activities. In this chapter, we review how mobile, wearable eye trackers improve the construct validity of important developmental constructs, such as visual object experiences and social attention, in ways that would be impossible using screen-based eye tracking. Head-mounted eye tracking improves ecological validity by allowing researchers to present more realistic and complex visual scenes, create more interactive experimental situations, and examine how the body influences what infants and children see. As with any new method, there are difficulties to overcome. Accordingly, we identify what aspects of head-mounted eye-tracking study design affect the measurement quality, interpretability of the results, and efficiency of gathering data. Moreover, we provide a summary of best practices aimed at allowing researchers to make well-informed decisions about whether and how to apply head-mounted eye tracking to their own research questions.
Collapse
|
8
|
Janssen TW, Grammer JK, Bleichner MG, Bulgarelli C, Davidesco I, Dikker S, Jasińska KK, Siugzdaite R, Vassena E, Vatakis A, Zion‐Golumbic E, van Atteveldt N. Opportunities and Limitations of Mobile Neuroimaging Technologies in Educational Neuroscience. MIND, BRAIN AND EDUCATION : THE OFFICIAL JOURNAL OF THE INTERNATIONAL MIND, BRAIN, AND EDUCATION SOCIETY 2021; 15:354-370. [PMID: 35875415 PMCID: PMC9292610 DOI: 10.1111/mbe.12302] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Revised: 06/18/2021] [Accepted: 09/01/2021] [Indexed: 05/20/2023]
Abstract
As the field of educational neuroscience continues to grow, questions have emerged regarding the ecological validity and applicability of this research to educational practice. Recent advances in mobile neuroimaging technologies have made it possible to conduct neuroscientific studies directly in naturalistic learning environments. We propose that embedding mobile neuroimaging research in a cycle (Matusz, Dikker, Huth, & Perrodin, 2019), involving lab-based, seminaturalistic, and fully naturalistic experiments, is well suited for addressing educational questions. With this review, we take a cautious approach, by discussing the valuable insights that can be gained from mobile neuroimaging technology, including electroencephalography and functional near-infrared spectroscopy, as well as the challenges posed by bringing neuroscientific methods into the classroom. Research paradigms used alongside mobile neuroimaging technology vary considerably. To illustrate this point, studies are discussed with increasingly naturalistic designs. We conclude with several ethical considerations that should be taken into account in this unique area of research.
Collapse
Affiliation(s)
- Tieme W.P. Janssen
- Department of Clinical, Neuro‐ & Developmental Psychology, Vrije Universiteit
| | - Jennie K. Grammer
- Graduate School of Education and Information Studies, University of California Los Angeles
| | | | - Chiara Bulgarelli
- Centre for Brain and Cognitive Development, Birkbeck University of London
| | - Ido Davidesco
- Department of Educational Psychology, University of Connecticut
| | | | - Kaja K. Jasińska
- Department of Applied Psychology and Human Development, University of Toronto
| | | | - Eliana Vassena
- Donders Institute for Brain, Cognition and Behaviour, Radboud University
| | | | | | | |
Collapse
|
9
|
Snow JC, Culham JC. The Treachery of Images: How Realism Influences Brain and Behavior. Trends Cogn Sci 2021; 25:506-519. [PMID: 33775583 PMCID: PMC10149139 DOI: 10.1016/j.tics.2021.02.008] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2020] [Revised: 02/08/2021] [Accepted: 02/22/2021] [Indexed: 10/21/2022]
Abstract
Although the cognitive sciences aim to ultimately understand behavior and brain function in the real world, for historical and practical reasons, the field has relied heavily on artificial stimuli, typically pictures. We review a growing body of evidence that both behavior and brain function differ between image proxies and real, tangible objects. We also propose a new framework for immersive neuroscience to combine two approaches: (i) the traditional build-up approach of gradually combining simplified stimuli, tasks, and processes; and (ii) a newer tear-down approach that begins with reality and compelling simulations such as virtual reality to determine which elements critically affect behavior and brain processing.
Collapse
Affiliation(s)
- Jacqueline C Snow
- Department of Psychology, University of Nevada Reno, Reno, NV 89557, USA
| | - Jody C Culham
- Department of Psychology, University of Western Ontario, London, Ontario, N6A 5C2, Canada; Brain and Mind Institute, Western Interdisciplinary Research Building, University of Western Ontario, London, Ontario, N6A 3K7, Canada.
| |
Collapse
|
10
|
Abstract
There is a long history of interest in looking behavior during human interaction. With the advance of (wearable) video-based eye trackers, it has become possible to measure gaze during many different interactions. We outline the different types of eye-tracking setups that currently exist to investigate gaze during interaction. The setups differ mainly with regard to the nature of the eye-tracking signal (head- or world-centered) and the freedom of movement allowed for the participants. These features place constraints on the research questions that can be answered about human interaction. We end with a decision tree to help researchers judge the appropriateness of specific setups.
Collapse
|
11
|
Ziv G, Lidor R, Zach S, Brams S, Helsen WF. Gaze Behavior of Referees in Sport-A Review. Front Sports Act Living 2020; 2:572891. [PMID: 33345134 PMCID: PMC7739781 DOI: 10.3389/fspor.2020.572891] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 09/08/2020] [Indexed: 11/13/2022] Open
Abstract
The purpose of this review was to examine the literature on gaze behavior in referees. A literature search found only 12 relevant studies. Five of those studies were conducted on referees in association football (soccer), three on judges in gymnastics, one on softball umpires, and one each on referees in team handball, rugby, and ice hockey. Seven studies reported differences in gaze behavior between referees of a higher skill level and those of a lower skill level, while four studies found no differences. In addition, five studies reported differences between referees of different skill levels in both gaze behavior and performance, while four studies found differences in performance only. A number of methodological concerns arise from the current review. Among them are the lack of studies conducted in ecologically valid conditions, the lack of studies on peripheral vision, and the lack of data on referees who are working together as teams. Based on this review, we conclude that additional research is needed to clarify the relationships between gaze behavior and performance in refereeing. Practitioners who work with referees should be cautious when adopting gaze training strategies to improve selective attention, since the data on their effectiveness are scarce and sometimes contradictory.
Collapse
Affiliation(s)
- Gal Ziv
- The Academic College at Wingate, Wingate Institute, Netanya, Israel
| | - Ronnie Lidor
- The Academic College at Wingate, Wingate Institute, Netanya, Israel
| | - Sima Zach
- The Academic College at Wingate, Wingate Institute, Netanya, Israel
| | - Stephanie Brams
- Movement Control & Neuroplasticity Research Group, Department of Movement Sciences, KU Leuven, Leuven, Belgium
| | - Werner F. Helsen
- Movement Control & Neuroplasticity Research Group, Department of Movement Sciences, KU Leuven, Leuven, Belgium
| |
Collapse
|