1
|
Cai Y, Strauch C, Van der Stigchel S, Naber M. Open-DPSM: An open-source toolkit for modeling pupil size changes to dynamic visual inputs. Behav Res Methods 2024; 56:5605-5621. [PMID: 38082113 PMCID: PMC11335788 DOI: 10.3758/s13428-023-02292-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/07/2023] [Indexed: 08/21/2024]
Abstract
Pupil size change is a widely adopted, sensitive indicator for sensory and cognitive processes. However, the interpretation of these changes is complicated by the influence of multiple low-level effects, such as brightness or contrast changes, posing challenges to applying pupillometry outside of extremely controlled settings. Building on and extending previous models, we here introduce Open Dynamic Pupil Size Modeling (Open-DPSM), an open-source toolkit to model pupil size changes to dynamically changing visual inputs using a convolution approach. Open-DPSM incorporates three key steps: (1) Modeling pupillary responses to both luminance and contrast changes; (2) Weighing of the distinct contributions of visual events across the visual field on pupil size change; and (3) Incorporating gaze-contingent visual event extraction and modeling. These steps improve the prediction of pupil size changes beyond the here-evaluated benchmarks. Open-DPSM provides Python functions, as well as a graphical user interface (GUI), enabling the extension of its applications to versatile scenarios and adaptations to individualized needs. By obtaining a predicted pupil trace using video and eye-tracking data, users can mitigate the effects of low-level features by subtracting the predicted trace or assess the efficacy of the low-level feature manipulations a priori by comparing estimated traces across conditions.
Collapse
Affiliation(s)
- Yuqing Cai
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands.
| | - Christoph Strauch
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands
| | - Stefan Van der Stigchel
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands
| | - Marnix Naber
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands
| |
Collapse
|
2
|
Fink L, Simola J, Tavano A, Lange E, Wallot S, Laeng B. From pre-processing to advanced dynamic modeling of pupil data. Behav Res Methods 2024; 56:1376-1412. [PMID: 37351785 PMCID: PMC10991010 DOI: 10.3758/s13428-023-02098-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/20/2023] [Indexed: 06/24/2023]
Abstract
The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.
Collapse
Affiliation(s)
- Lauren Fink
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany.
- Department of Psychology, Neuroscience & Behavior, McMaster University, 1280 Main St. West, Hamilton, Ontario, L8S 4L8, Canada.
| | - Jaana Simola
- Helsinki Collegium for Advanced Studies, University of Helsinki, Helsinki, Finland
- Department of Education, University of Helsinki, Helsinki, Finland
| | - Alessandro Tavano
- Department of Cognitive Neuropsychology, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Elke Lange
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany
| | - Sebastian Wallot
- Department of Literature, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- Institute for Sustainability Education and Psychologyy, Leuphana University, Lüneburg, Germany
| | - Bruno Laeng
- Department of Psychology, University of Oslo, Oslo, Norway
- RITMO Centre for Interdisciplinary studies in Rhythm, Time, and Motion, University of Oslo, Oslo, Norway
| |
Collapse
|
3
|
Fitzgerald LP, DeDe G, Shen J. Effects of linguistic context and noise type on speech comprehension. Front Psychol 2024; 15:1345619. [PMID: 38375107 PMCID: PMC10875108 DOI: 10.3389/fpsyg.2024.1345619] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 01/17/2024] [Indexed: 02/21/2024] Open
Abstract
Introduction Understanding speech in background noise is an effortful endeavor. When acoustic challenges arise, linguistic context may help us fill in perceptual gaps. However, more knowledge is needed regarding how different types of background noise affect our ability to construct meaning from perceptually complex speech input. Additionally, there is limited evidence regarding whether perceptual complexity (e.g., informational masking) and linguistic complexity (e.g., occurrence of contextually incongruous words) interact during processing of speech material that is longer and more complex than a single sentence. Our first research objective was to determine whether comprehension of spoken sentence pairs is impacted by the informational masking from a speech masker. Our second objective was to identify whether there is an interaction between perceptual and linguistic complexity during speech processing. Methods We used multiple measures including comprehension accuracy, reaction time, and processing effort (as indicated by task-evoked pupil response), making comparisons across three different levels of linguistic complexity in two different noise conditions. Context conditions varied by final word, with each sentence pair ending with an expected exemplar (EE), within-category violation (WV), or between-category violation (BV). Forty young adults with typical hearing performed a speech comprehension in noise task over three visits. Each participant heard sentence pairs presented in either multi-talker babble or spectrally shaped steady-state noise (SSN), with the same noise condition across all three visits. Results We observed an effect of context but not noise on accuracy. Further, we observed an interaction of noise and context in peak pupil dilation data. Specifically, the context effect was modulated by noise type: context facilitated processing only in the more perceptually complex babble noise condition. Discussion These findings suggest that when perceptual complexity arises, listeners make use of the linguistic context to facilitate comprehension of speech obscured by background noise. Our results extend existing accounts of speech processing in noise by demonstrating how perceptual and linguistic complexity affect our ability to engage in higher-level processes, such as construction of meaning from speech segments that are longer than a single sentence.
Collapse
Affiliation(s)
- Laura P. Fitzgerald
- Speech Perception and Cognition Laboratory, Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, United States
| | - Gayle DeDe
- Speech, Language, and Brain Laboratory, Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, United States
| | - Jing Shen
- Speech Perception and Cognition Laboratory, Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, United States
| |
Collapse
|
4
|
Johns MA, Calloway RC, Karunathilake IMD, Decruy LP, Anderson S, Simon JZ, Kuchinsky SE. Attention Mobilization as a Modulator of Listening Effort: Evidence From Pupillometry. Trends Hear 2024; 28:23312165241245240. [PMID: 38613337 PMCID: PMC11015766 DOI: 10.1177/23312165241245240] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 03/11/2024] [Accepted: 03/15/2024] [Indexed: 04/14/2024] Open
Abstract
Listening to speech in noise can require substantial mental effort, even among younger normal-hearing adults. The task-evoked pupil response (TEPR) has been shown to track the increased effort exerted to recognize words or sentences in increasing noise. However, few studies have examined the trajectory of listening effort across longer, more natural, stretches of speech, or the extent to which expectations about upcoming listening difficulty modulate the TEPR. Seventeen younger normal-hearing adults listened to 60-s-long audiobook passages, repeated three times in a row, at two different signal-to-noise ratios (SNRs) while pupil size was recorded. There was a significant interaction between SNR, repetition, and baseline pupil size on sustained listening effort. At lower baseline pupil sizes, potentially reflecting lower attention mobilization, TEPRs were more sustained in the harder SNR condition, particularly when attention mobilization remained low by the third presentation. At intermediate baseline pupil sizes, differences between conditions were largely absent, suggesting these listeners had optimally mobilized their attention for both SNRs. Lastly, at higher baseline pupil sizes, potentially reflecting overmobilization of attention, the effect of SNR was initially reversed for the second and third presentations: participants initially appeared to disengage in the harder SNR condition, resulting in reduced TEPRs that recovered in the second half of the story. Together, these findings suggest that the unfolding of listening effort over time depends critically on the extent to which individuals have successfully mobilized their attention in anticipation of difficult listening conditions.
Collapse
Affiliation(s)
- M. A. Johns
- Institute for Systems Research, University of Maryland, College Park, MD 20742, USA
| | - R. C. Calloway
- Institute for Systems Research, University of Maryland, College Park, MD 20742, USA
| | - I. M. D. Karunathilake
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA
| | - L. P. Decruy
- Institute for Systems Research, University of Maryland, College Park, MD 20742, USA
| | - S. Anderson
- Department of Hearing and Speech Sciences, University of Maryland, College Park, MD 20742, USA
| | - J. Z. Simon
- Institute for Systems Research, University of Maryland, College Park, MD 20742, USA
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA
- Department of Biology, University of Maryland, College Park, MD 20742, USA
| | - S. E. Kuchinsky
- Department of Hearing and Speech Sciences, University of Maryland, College Park, MD 20742, USA
- National Military Audiology and Speech Pathology Center, Walter Reed National Military Medical Center, Bethesda, MD 20889, USA
| |
Collapse
|
5
|
Le Cunff AL, Dommett E, Giampietro V. Neurophysiological measures and correlates of cognitive load in attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and dyslexia: A scoping review and research recommendations. Eur J Neurosci 2024; 59:256-282. [PMID: 38109476 DOI: 10.1111/ejn.16201] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Revised: 10/27/2023] [Accepted: 11/06/2023] [Indexed: 12/20/2023]
Abstract
Working memory is integral to a range of critical cognitive functions such as reasoning and decision-making. Although alterations in working memory have been observed in neurodivergent populations, there has been no review mapping how cognitive load is measured in common neurodevelopmental conditions such as attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and dyslexia. This scoping review explores the neurophysiological measures used to study cognitive load in these specific populations. Our findings highlight that electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are the most frequently used methods, with a limited number of studies employing functional near-infrared spectroscopy (fNIRs), magnetoencephalography (MEG) or eye-tracking. Notably, eye-related measures are less commonly used, despite their prominence in cognitive load research among neurotypical individuals. The review also highlights potential correlates of cognitive load, such as neural oscillations in the theta and alpha ranges for EEG studies, blood oxygenation level-dependent (BOLD) responses in lateral and medial frontal brain regions for fMRI and fNIRS studies and eye-related measures such as pupil dilation and blink rate. Finally, critical issues for future studies are discussed, including the technical challenges associated with multimodal approaches, the possible impact of atypical features on cognitive load measures and balancing data richness with participant well-being. These insights contribute to a more nuanced understanding of cognitive load measurement in neurodivergent populations and point to important methodological considerations for future neuroscientific research in this area.
Collapse
Affiliation(s)
- Anne-Laure Le Cunff
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Eleanor Dommett
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Vincent Giampietro
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| |
Collapse
|
6
|
Kolnes M, Uusberg A, Nieuwenhuis S. Broadening of attention dilates the pupil. Atten Percept Psychophys 2024; 86:146-158. [PMID: 37801189 PMCID: PMC10770199 DOI: 10.3758/s13414-023-02793-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/14/2023] [Indexed: 10/07/2023]
Abstract
Inconclusive evidence suggests that the pupil is more dilated when the breadth of attention is broad compared to narrow. To further investigate this relationship, we recorded pupil size from healthy volunteers while inducing trial-wise changes in breadth of attention using a shape-discrimination task where participants had to remember the location of a gap in a small or a large circle. A visual search task with targets presented at different distances from the centre of the screen was used to behaviourally assess the success of the manipulation of breadth of attention. Data were analysed using a generalised additive mixed model to test the experimental effects on pupil size after controlling for the effects of gaze location and eye vergence. The results showed that the pupil was more dilated in the broad-breadth-of-attention condition compared to the narrow-breadth-of-attention condition. However, the effect of attentional breadth on visual search performance was not mediated by pupil size, suggesting that more research is needed to understand the functional role of pupil dilation in relation to breadth of attention.
Collapse
Affiliation(s)
- Martin Kolnes
- Department of Psychology Institute of Psychology, University of Tartu, Näituse 2, 50409, Tartu, Estonia.
- Institute of Psychology and Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands.
| | - Andero Uusberg
- Department of Psychology Institute of Psychology, University of Tartu, Näituse 2, 50409, Tartu, Estonia
| | - Sander Nieuwenhuis
- Institute of Psychology and Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands
| |
Collapse
|
7
|
Mathôt S, Vilotijević A. Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis. Behav Res Methods 2023; 55:3055-3077. [PMID: 36028608 PMCID: PMC10556184 DOI: 10.3758/s13428-022-01957-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/12/2022] [Indexed: 11/08/2022]
Abstract
Cognitive pupillometry is the measurement of pupil size to investigate cognitive processes such as attention, mental effort, working memory, and many others. Currently, there is no commonly agreed-upon methodology for conducting cognitive-pupillometry experiments, and approaches vary widely between research groups and even between different experiments from the same group. This lack of consensus makes it difficult to know which factors to consider when conducting a cognitive-pupillometry experiment. Here we provide a comprehensive, hands-on guide to methods in cognitive pupillometry, with a focus on trial-based experiments in which the measure of interest is the task-evoked pupil response to a stimulus. We cover all methodological aspects of cognitive pupillometry: experimental design, preprocessing of pupil-size data, and statistical techniques to deal with multiple comparisons when testing pupil-size data. In addition, we provide code and toolboxes (in Python) for preprocessing and statistical analysis, and we illustrate all aspects of the proposed workflow through an example experiment and example scripts.
Collapse
Affiliation(s)
- Sebastiaan Mathôt
- Department of Psychology, University of Groningen, Grote Kruisstraat 2/1, 9712TS, Groningen, The Netherlands.
| | - Ana Vilotijević
- Department of Psychology, University of Groningen, Grote Kruisstraat 2/1, 9712TS, Groningen, The Netherlands
| |
Collapse
|
8
|
Walcher S, Korda Ž, Körner C, Benedek M. The effects of type and workload of internal tasks on voluntary saccades in a target-distractor saccade task. PLoS One 2023; 18:e0290322. [PMID: 37616320 PMCID: PMC10449167 DOI: 10.1371/journal.pone.0290322] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 08/07/2023] [Indexed: 08/26/2023] Open
Abstract
When we engage in internally directed cognition, like doing mental arithmetic or mind wandering, fewer cognitive resources are assigned for other activities like reacting to perceptual input-an effect termed perceptual decoupling. However, the exact conditions under which perceptual decoupling occurs and its underlying cognitive mechanisms are still unclear. Hence, the present study systematically manipulated the task type (arithmetic, visuospatial) and workload (control, low, high) of the internal task in a within-subject design and tested its effects on voluntary saccades in a target-distractor saccade task. As expected, engagement in internal tasks delayed saccades to the target. This effect was moderated by time, task, and workload: The delay was largest right after internal task onset and then decreased, potentially reflecting the intensity of internal task demands. Saccades were also more delayed for the high compared to the low workload condition in the arithmetic task, whereas workload conditions had similarly high effects in the visuospatial task. Findings suggests that perceptual decoupling of eye behavior gradually increases with internal demands on general resources and that perceptual decoupling is specifically sensitive to internal demands on visuospatial resources. The latter may be mediated by interference due to eye behavior elicited by the internal task itself. Internal tasks did not affect the saccade latency-deviation trade-off, indicating that while the internal tasks delayed the execution of the saccade, the perception of the saccade stimuli and spatial planning of the saccade continued unaffected in parallel to the internal tasks. Together, these findings shed further light on the specific mechanisms underlying perceptual decoupling by suggesting that perceptual decoupling of eye behavior increases as internal demands on cognitive resources overlap more strongly with demands of the external task.
Collapse
Affiliation(s)
- Sonja Walcher
- Creative Cognition Lab, Institute of Psychology, University of Graz, Graz, Austria
| | - Živa Korda
- Creative Cognition Lab, Institute of Psychology, University of Graz, Graz, Austria
| | - Christof Körner
- Cognitive Psychology & Neuroscience, Institute of Psychology, University of Graz, Graz, Austria
| | - Mathias Benedek
- Creative Cognition Lab, Institute of Psychology, University of Graz, Graz, Austria
| |
Collapse
|
9
|
Martohardjono G, Johns MA, Franciotti P, Castillo D, Porru I, Lowry C. Use of the first-acquired language modulates pupil size in the processing of island constraint violations. Front Psychol 2023; 14:1180989. [PMID: 37519378 PMCID: PMC10382202 DOI: 10.3389/fpsyg.2023.1180989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Accepted: 06/15/2023] [Indexed: 08/01/2023] Open
Abstract
Introduction Traditional studies of the population called "heritage speakers" (HS) have treated this group as distinct from other bilingual populations, e.g., simultaneous or late bilinguals (LB), focusing on group differences in the competencies of the first-acquired language or "heritage language". While several explanations have been proposed for such differences (e.g., incomplete acquisition, attrition, differential processing mechanisms), few have taken into consideration the individual variation that must occur, due to the fluctuation of factors such as exposure and use that characterize all bilinguals. In addition, few studies have used implicit measures, e.g., psychophysiological methods (ERPs; Eye-tracking), that can circumvent confounding variables such as resorting to conscious metalinguistic knowledge. Methodology This study uses pupillometry, a method that has only recently been used in psycholinguistic studies of bilingualism, to investigate pupillary responses to three syntactic island constructions in two groups of Spanish/English bilinguals: heritage speakers and late bilinguals. Data were analyzed using generalized additive mixed effects models (GAMMs) and two models were created and compared to one another: one with group (LB/HS) and the other with groups collapsed and current and historical use of Spanish as continuous variables. Results Results show that group-based models generally yield conflicting results while models collapsing groups and having usage as a predictor yield consistent ones. In particular, current use predicts sensitivity to L1 ungrammaticality across both HS and LB populations. We conclude that individual variation, as measured by use, is a critical factor tha must be taken into account in the description of the language competencies and processing of heritage and late bilinguals alike.
Collapse
Affiliation(s)
- Gita Martohardjono
- Department of Linguistics and Communication Disorders, Queens College, New York, NY, United States
- Second Language Acquisition Laboratory, Linguistics Program, The Graduate Center of the City University of New York, New York, NY, United States
| | - Michael A. Johns
- Institute for Systems Research, University of Maryland, College Park, MD, United States
| | - Pamela Franciotti
- Second Language Acquisition Laboratory, Linguistics Program, The Graduate Center of the City University of New York, New York, NY, United States
| | - Daniela Castillo
- Second Language Acquisition Laboratory, Linguistics Program, The Graduate Center of the City University of New York, New York, NY, United States
| | - Ilaria Porru
- Second Language Acquisition Laboratory, Linguistics Program, The Graduate Center of the City University of New York, New York, NY, United States
| | - Cass Lowry
- Second Language Acquisition Laboratory, Linguistics Program, The Graduate Center of the City University of New York, New York, NY, United States
| |
Collapse
|
10
|
Holmqvist K, Örbom SL, Hooge ITC, Niehorster DC, Alexander RG, Andersson R, Benjamins JS, Blignaut P, Brouwer AM, Chuang LL, Dalrymple KA, Drieghe D, Dunn MJ, Ettinger U, Fiedler S, Foulsham T, van der Geest JN, Hansen DW, Hutton SB, Kasneci E, Kingstone A, Knox PC, Kok EM, Lee H, Lee JY, Leppänen JM, Macknik S, Majaranta P, Martinez-Conde S, Nuthmann A, Nyström M, Orquin JL, Otero-Millan J, Park SY, Popelka S, Proudlock F, Renkewitz F, Roorda A, Schulte-Mecklenbeck M, Sharif B, Shic F, Shovman M, Thomas MG, Venrooij W, Zemblys R, Hessels RS. Eye tracking: empirical foundations for a minimal reporting guideline. Behav Res Methods 2023; 55:364-416. [PMID: 35384605 PMCID: PMC9535040 DOI: 10.3758/s13428-021-01762-8] [Citation(s) in RCA: 52] [Impact Index Per Article: 52.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/29/2021] [Indexed: 11/08/2022]
Abstract
In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "An empirically based minimal reporting guideline").
Collapse
Affiliation(s)
- Kenneth Holmqvist
- Department of Psychology, Nicolaus Copernicus University, Torun, Poland.
- Department of Computer Science and Informatics, University of the Free State, Bloemfontein, South Africa.
- Department of Psychology, Regensburg University, Regensburg, Germany.
| | - Saga Lee Örbom
- Department of Psychology, Regensburg University, Regensburg, Germany
| | - Ignace T C Hooge
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - Diederick C Niehorster
- Lund University Humanities Lab and Department of Psychology, Lund University, Lund, Sweden
| | - Robert G Alexander
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | | | - Jeroen S Benjamins
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
- Social, Health and Organizational Psychology, Utrecht University, Utrecht, The Netherlands
| | - Pieter Blignaut
- Department of Computer Science and Informatics, University of the Free State, Bloemfontein, South Africa
| | | | - Lewis L Chuang
- Department of Ergonomics, Leibniz Institute for Working Environments and Human Factors, Dortmund, Germany
- Institute of Informatics, LMU Munich, Munich, Germany
| | | | - Denis Drieghe
- School of Psychology, University of Southampton, Southampton, UK
| | - Matt J Dunn
- School of Optometry and Vision Sciences, Cardiff University, Cardiff, UK
| | | | - Susann Fiedler
- Vienna University of Economics and Business, Vienna, Austria
| | - Tom Foulsham
- Department of Psychology, University of Essex, Essex, UK
| | | | - Dan Witzner Hansen
- Machine Learning Group, Department of Computer Science, IT University of Copenhagen, Copenhagen, Denmark
| | | | - Enkelejda Kasneci
- Human-Computer Interaction, University of Tübingen, Tübingen, Germany
| | | | - Paul C Knox
- Department of Eye and Vision Science, Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, UK
| | - Ellen M Kok
- Department of Education and Pedagogy, Division Education, Faculty of Social and Behavioral Sciences, Utrecht University, Utrecht, The Netherlands
- Department of Online Learning and Instruction, Faculty of Educational Sciences, Open University of the Netherlands, Heerlen, The Netherlands
| | - Helena Lee
- University of Southampton, Southampton, UK
| | - Joy Yeonjoo Lee
- School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Maastricht University, Maastricht, The Netherlands
| | - Jukka M Leppänen
- Department of Psychology and Speed-Language Pathology, University of Turku, Turku, Finland
| | - Stephen Macknik
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Päivi Majaranta
- TAUCHI Research Center, Computing Sciences, Faculty of Information Technology and Communication Sciences, Tampere University, Tampere, Finland
| | - Susana Martinez-Conde
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Antje Nuthmann
- Institute of Psychology, University of Kiel, Kiel, Germany
| | - Marcus Nyström
- Lund University Humanities Lab, Lund University, Lund, Sweden
| | - Jacob L Orquin
- Department of Management, Aarhus University, Aarhus, Denmark
- Center for Research in Marketing and Consumer Psychology, Reykjavik University, Reykjavik, Iceland
| | - Jorge Otero-Millan
- Herbert Wertheim School of Optometry and Vision Science, University of California, Berkeley, CA, USA
| | - Soon Young Park
- Comparative Cognition, Messerli Research Institute, University of Veterinary Medicine Vienna, Medical University of Vienna, Vienna, Austria
| | - Stanislav Popelka
- Department of Geoinformatics, Palacký University Olomouc, Olomouc, Czech Republic
| | - Frank Proudlock
- The University of Leicester Ulverscroft Eye Unit, Department of Neuroscience, Psychology and Behaviour, University of Leicester, Leicester, UK
| | - Frank Renkewitz
- Department of Psychology, University of Erfurt, Erfurt, Germany
| | - Austin Roorda
- Herbert Wertheim School of Optometry and Vision Science, University of California, Berkeley, CA, USA
| | | | - Bonita Sharif
- School of Computing, University of Nebraska-Lincoln, Lincoln, Nebraska, USA
| | - Frederick Shic
- Center for Child Health, Behavior and Development, Seattle Children's Research Institute, Seattle, WA, USA
- Department of General Pediatrics, University of Washington School of Medicine, Seattle, WA, USA
| | - Mark Shovman
- Eyeviation Systems, Herzliya, Israel
- Department of Industrial Design, Bezalel Academy of Arts and Design, Jerusalem, Israel
| | - Mervyn G Thomas
- The University of Leicester Ulverscroft Eye Unit, Department of Neuroscience, Psychology and Behaviour, University of Leicester, Leicester, UK
| | - Ward Venrooij
- Electrical Engineering, Mathematics and Computer Science (EEMCS), University of Twente, Enschede, The Netherlands
| | | | - Roy S Hessels
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
11
|
Shen J, Heller Murray E, Kulick ER. The Effect of Breathy Vocal Quality on Speech Intelligibility and Listening Effort in Background Noise. Trends Hear 2023; 27:23312165231206925. [PMID: 37817666 PMCID: PMC10566269 DOI: 10.1177/23312165231206925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 09/06/2023] [Accepted: 09/25/2023] [Indexed: 10/12/2023] Open
Abstract
Speech perception is challenging under adverse conditions. However, there is limited evidence regarding how multiple adverse conditions affect speech perception. The present study investigated two conditions that are frequently encountered in real-life communication: background noise and breathy vocal quality. The study first examined the effects of background noise and breathiness on speech perception as measured by intelligibility. Secondly, the study tested the hypothesis that both noise and breathiness affect listening effort, as indicated by linear and nonlinear changes in pupil dilation. Low-context sentences were resynthesized to create three levels of breathiness (original, mild-moderate, and severe). The sentences were presented in a fluctuating nonspeech noise with two signal-to-noise ratios (SNRs) of -5 dB (favorable) and -9 dB (adverse) SNR. Speech intelligibility and pupil dilation data were collected from young listeners with normal hearing thresholds. The results demonstrated that a breathy vocal quality presented in noise negatively affected speech intelligibility, with the degree of breathiness playing a critical role. Listening effort, as measured by the magnitude of pupil dilation, showed significant effects with both severe and mild-moderate breathy voices that were independent of noise level. The findings contributed to the literature by demonstrating the impact of vocal quality on the perception of speech in noise. They also highlighted the complex dynamics between overall task demand and processing resources in understanding the combined impact of multiple adverse conditions.
Collapse
Affiliation(s)
- Jing Shen
- Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, USA
| | - Elizabeth Heller Murray
- Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, USA
| | - Erin R. Kulick
- Department of Epidemiology and Biostatistics, College of Public Health, Temple University, Philadelphia, PA, USA
| |
Collapse
|
12
|
Pupillometry as a Window into Young Children's Sustained Attention. J Intell 2022; 10:jintelligence10040107. [PMID: 36412787 PMCID: PMC9680391 DOI: 10.3390/jintelligence10040107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 10/26/2022] [Accepted: 11/09/2022] [Indexed: 11/18/2022] Open
Abstract
Sustained attention is critical to cognition, social competence, and academic success. Importantly, sustained attention undergoes significant development over the early childhood period. Yet, how sustained attention fluctuates over time on task has not been clearly outlined, particularly in young children. In this study, we provide a first test of whether the pupillary response can be used as an indicator of moment-to-moment sustained attention over time on task in young children. Children aged 5 to 7 years (N = 41) completed a psychomotor vigilance task, where they were asked to press a button as fast as possible at the onset of a target stimulus. We measured reaction times over the course of the task, pupil size prior to target onset (baseline pupil size), and pupil size in response to target onset (task-evoked pupil size). The results showed a stereotypical vigilance decrement in children's response times: as time on task increased, reaction times increased. Critically, children's task-evoked pupil size decreased over time on task, while no such change was present in baseline pupil size. These results suggest that young children's waning sustained attention may be linked to a decrease in alertness while overall arousal is maintained. We discuss the importance of leveraging pupillometry to understand the mechanisms of sustained attention over individuals and development.
Collapse
|
13
|
Shen J, Fitzgerald LP, Kulick ER. Interactions between acoustic challenges and processing depth in speech perception as measured by task-evoked pupil response. Front Psychol 2022; 13:959638. [PMID: 36389464 PMCID: PMC9641013 DOI: 10.3389/fpsyg.2022.959638] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 09/12/2022] [Indexed: 08/21/2023] Open
Abstract
Speech perception under adverse conditions is a multistage process involving a dynamic interplay among acoustic, cognitive, and linguistic factors. Nevertheless, prior research has primarily focused on factors within this complex system in isolation. The primary goal of the present study was to examine the interaction between processing depth and the acoustic challenge of noise and its effect on processing effort during speech perception in noise. Two tasks were used to represent different depths of processing. The speech recognition task involved repeating back a sentence after auditory presentation (higher-level processing), while the tiredness judgment task entailed a subjective judgment of whether the speaker sounded tired (lower-level processing). The secondary goal of the study was to investigate whether pupil response to alteration of dynamic pitch cues stems from difficult linguistic processing of speech content in noise or a perceptual novelty effect due to the unnatural pitch contours. Task-evoked peak pupil response from two groups of younger adult participants with typical hearing was measured in two experiments. Both tasks (speech recognition and tiredness judgment) were implemented in both experiments, and stimuli were presented with background noise in Experiment 1 and without noise in Experiment 2. Increased peak pupil dilation was associated with deeper processing (i.e., the speech recognition task), particularly in the presence of background noise. Importantly, there is a non-additive interaction between noise and task, as demonstrated by the heightened peak pupil dilation to noise in the speech recognition task as compared to in the tiredness judgment task. Additionally, peak pupil dilation data suggest dynamic pitch alteration induced an increased perceptual novelty effect rather than reflecting effortful linguistic processing of the speech content in noise. These findings extend current theories of speech perception under adverse conditions by demonstrating that the level of processing effort expended by a listener is influenced by the interaction between acoustic challenges and depth of linguistic processing. The study also provides a foundation for future work to investigate the effects of this complex interaction in clinical populations who experience both hearing and cognitive challenges.
Collapse
Affiliation(s)
- Jing Shen
- Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, United States
| | - Laura P. Fitzgerald
- Department of Communication Sciences and Disorders, College of Public Health, Temple University, Philadelphia, PA, United States
| | - Erin R. Kulick
- Department of Epidemiology and Biostatistics, College of Public Health, Temple University, Philadelphia, PA, United States
| |
Collapse
|
14
|
Hu X, Hisakata R, Kaneko H. The relationship between pupillary baseline manipulated by mental effort or luminance and subsequent pupillary responses. J Vis 2022; 22:7. [PMID: 35758900 PMCID: PMC9248750 DOI: 10.1167/jov.22.7.7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Measuring pupillary response is a prevalent technique to evaluate mental states. It is indispensable to conduct a correction procedure for the pupillary baseline to get a meaningful conclusion from the pupillary response. However, the relationship between pupillary baseline and subsequent stimulus-evoked pupillary response varies among studies. In this study, we used the subtractive and proportional baseline corrections to analyze the results. Furthermore, we manipulated the pupillary baseline through mental effort or luminance in the baseline period and investigated whether the subsequent stimulus-evoked pupillary responses were affected. We found that the mental effort–evoked pupillary response was attenuated with a larger pupillary baseline manipulated by a higher mental effort, whereas it was unaffected with the baseline manipulated by luminance. Also, the luminance-evoked pupillary response was attenuated with a smaller pupillary baseline manipulated by a brighter disk, whereas it was unaffected with the baseline manipulated by mental effort. The results could be obtained from subtractive and proportional baseline corrections. Our results suggest that mental effort manipulated pupillary baseline interacts with the subsequent mental effort elicited pupillary response, but not with the luminance elicited pupillary response; the luminance manipulated pupillary baseline interacts with the subsequent luminance elicited pupillary response, but not with the mental effort elicited pupillary response. It is important to consider the ways of controlling the pupillary baseline and subsequent pupillary response simultaneously.
Collapse
Affiliation(s)
- Xiaofei Hu
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, Midori-Ku, Yokohama, Japan.,
| | - Rumi Hisakata
- Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, Midori-Ku, Yokohama, Japan.,
| | - Hirohiko Kaneko
- Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, Midori-Ku, Yokohama, Japan.,
| |
Collapse
|
15
|
Comparing methods of analysis in pupillometry: application to the assessment of listening effort in hearing-impaired patients. Heliyon 2022; 8:e09631. [PMID: 35734572 PMCID: PMC9207619 DOI: 10.1016/j.heliyon.2022.e09631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 12/26/2021] [Accepted: 05/26/2022] [Indexed: 11/20/2022] Open
|
16
|
Steinhauer SR, Bradley MM, Siegle GJ, Roecklein KA, Dix A. Publication guidelines and recommendations for pupillary measurement in psychophysiological studies. Psychophysiology 2022; 59:e14035. [PMID: 35318693 PMCID: PMC9272460 DOI: 10.1111/psyp.14035] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Accepted: 02/11/2022] [Indexed: 11/27/2022]
Abstract
A variety of psychological and physical phenomena elicit variations in the diameter of pupil of the eye. Changes in pupil size are mediated by the relative activation of the sphincter pupillae muscle (decrease pupil diameter) and the dilator pupillae muscle (increase pupil diameter), innervated by the parasympathetic and sympathetic branches, respectively, of the autonomic nervous system. The current guidelines are intended to inform and guide psychophysiological research involving pupil measurement by (1) summarizing important aspects concerning the physiology of the pupil, (2) providing methodological and data-analytic guidelines and recommendations, and (3) briefly reviewing psychological phenomena that modulate pupillary reactivity. Because of the increased ease and tractability of pupil measurement, the goal of these guidelines is to promote accurate recording, analysis, and reporting of pupillary data in psychophysiological research.
Collapse
Affiliation(s)
- Stuart R. Steinhauer
- Veterans Affairs Pittsburgh Healthcare System, VISN 4 MIRECC, Pittsburgh, PA, USA
- Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | | | - Greg J. Siegle
- Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
- Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA
| | | | - Annika Dix
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany
- Centre for Tactile Internet with Human-in-the-Loop (CeTI), Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
17
|
Kay L, Keogh R, Andrillon T, Pearson J. The pupillary light response as a physiological index of aphantasia, sensory and phenomenological imagery strength. eLife 2022; 11:72484. [PMID: 35356890 PMCID: PMC9018072 DOI: 10.7554/elife.72484] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 03/30/2022] [Indexed: 11/24/2022] Open
Abstract
The pupillary light response is an important automatic physiological response which optimizes light reaching the retina. Recent work has shown that the pupil also adjusts in response to illusory brightness and a range of cognitive functions, however, it remains unclear what exactly drives these endogenous changes. Here, we show that the imagery pupillary light response correlates with objective measures of sensory imagery strength. Further, the trial-by-trial phenomenological vividness of visual imagery is tracked by the imagery pupillary light response. We also demonstrated that a group of individuals without visual imagery (aphantasia) do not show any significant evidence of an imagery pupillary light response, however they do show perceptual pupil light responses and pupil dilation with larger cognitive load. Our results provide evidence that the pupillary light response indexes the sensory strength of visual imagery. This work also provides the first physiological validation of aphantasia.
Collapse
Affiliation(s)
- Lachlan Kay
- School of Psychology, University of New South Wales, Sydney, Australia
| | - Rebecca Keogh
- School of Psychological Sciences, Macquarie University, Sydney, Australia
| | - Thomas Andrillon
- Institut du Cerveau - Paris Brain Institute, Sorbonne Université, Paris, France
| | - Joel Pearson
- School of Psychology, University of New South Wales, Sydney, Australia
| |
Collapse
|
18
|
Pandey P, Ray S. Influence of the Location of a Decision Cue on the Dynamics of Pupillary Light Response. Front Hum Neurosci 2022; 15:755383. [PMID: 35153699 PMCID: PMC8826249 DOI: 10.3389/fnhum.2021.755383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2021] [Accepted: 12/22/2021] [Indexed: 11/16/2022] Open
Abstract
The pupils of the eyes reflexively constrict in light and dilate in dark to optimize retinal illumination. Non-visual cognitive factors, like attention, arousal, decision-making, etc., also influence pupillary light response (PLR). During passive viewing, the eccentricity of a stimulus modulates the pupillary aperture size driven by spatially weighted corneal flux density (CFD), which is the product of luminance and the area of the stimulus. Whether the scope of attention also influences PLR remains unclear. In this study, we contrasted the pupil dynamics between diffused and focused attentional conditions during decision-making, while the global CFD remained the same in the two conditions. A population of 20 healthy humans participated in a pair of forced choice tasks. They distributed attention to the peripheral decision cue in one task, and concentrated at the center in the other to select the target from four alternatives for gaze orientation. The location of this cue did not influence participants' reaction time (RT). However, the magnitude of constriction was significantly less in the task that warranted attention to be deployed at the center than on the periphery. We observed similar pupil dynamics when participants either elicited or canceled a saccadic eye movement, which ruled out pre-saccadic obligatory attentional orientation contributing to PLR. We further addressed how the location of attentional deployment might have influenced PLR. We simulated a biomechanical model of PLR with visual stimulation of different strengths as inputs corresponding to the two attentional conditions. In this homeomorphic model, the computational characteristic of each element was derived from the physiological and/or mechanical properties of the corresponding biological element. The simulation of this model successfully mimicked the observed data. In contrast to common belief that the global ambient luminosity drives pupillary response, the results of our study suggest that the effective CFD (eCFD) determined via the luminance multiplied by the size of the stimulus at the location of deployed attention in the visual space is critical for the magnitude of pupillary constriction.
Collapse
Affiliation(s)
| | - Supriya Ray
- Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj, India
| |
Collapse
|
19
|
Abstract
We present PuPl (Pu pillometry P ipel iner), an Octave-compatible library of Matlab functions for processing pupillometry data with an easy-to-use graphical user interface (GUI). PuPl's preprocessing tools include blink correction, data smoothing, and gaze correction. PuPl can also define and sort trials, and segment data to isolate event-related pupil dilation responses. PuPl's flexible tabular export tools enable a wide variety of statistical analyses. Furthermore, PuPl can translate GUI interactions into a Matlab script, enabling easy creation and reconfiguration of reusable data processing pipelines. Finally, PuPl is designed to be extensible, and users can easily contribute functionality as best practices for pupillometry evolve. Here we demonstrate PuPl by replicating published results using publicly available data. PuPl can be downloaded from github.com/kinleyid/pupl .
Collapse
|
20
|
Eye movements during text reading align with the rate of speech production. Nat Hum Behav 2021; 6:429-442. [PMID: 34873275 DOI: 10.1038/s41562-021-01215-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2019] [Accepted: 09/10/2021] [Indexed: 11/08/2022]
Abstract
Across languages, the speech signal is characterized by a predominant modulation of the amplitude spectrum between about 4.3 and 5.5 Hz, reflecting the production and processing of linguistic information chunks (syllables and words) every ~200 ms. Interestingly, ~200 ms is also the typical duration of eye fixations during reading. Prompted by this observation, we demonstrate that German readers sample written text at ~5 Hz. A subsequent meta-analysis of 142 studies from 14 languages replicates this result and shows that sampling frequencies vary across languages between 3.9 Hz and 5.2 Hz. This variation systematically depends on the complexity of the writing systems (character-based versus alphabetic systems and orthographic transparency). Finally, we empirically demonstrate a positive correlation between speech spectrum and eye movement sampling in low-skilled non-native readers, with tentative evidence from post hoc analysis suggesting the same relationship in low-skilled native readers. On the basis of this convergent evidence, we propose that during reading, our brain's linguistic processing systems imprint a preferred processing rate-that is, the rate of spoken language production and perception-onto the oculomotor system.
Collapse
|
21
|
Wang CA, Munoz DP. Differentiating global luminance, arousal and cognitive signals on pupil size and microsaccades. Eur J Neurosci 2021; 54:7560-7574. [PMID: 34716728 DOI: 10.1111/ejn.15508] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 10/22/2021] [Accepted: 10/23/2021] [Indexed: 10/19/2022]
Abstract
Pupil size reflects a proxy for neural activity associated with global luminance, arousal and cognitive processing. Microsaccades are also modulated by arousal and cognitive processing. Are these effects of arousal and cognitive signals on pupil size and microsaccades coordinated? If so, via what neural mechanisms? We hypothesized that if pupil size and microsaccades are coordinately modulated by these processes, pupil size immediately before microsaccade onset, as an index for ongoing cognitive and arousal processing, should correlate with microsaccade responses during tasks alternating these signals. Here, we examined the relationship between pupil size and microsaccade responses in tasks that included variations in global luminance, arousal and inhibitory control. Higher microsaccade peak velocities correlated with larger pre-microsaccade pupil response related to arousal and inhibitory control signals. In contrast, pupil responses evoked by global luminance signals did not correlate with microsaccade responses. Given the central role of the superior colliculus in microsaccade generation, these results suggest the critical involvement of the superior colliculus to coordinate pupil and microsaccade responses for arousal and inhibitory control modulations, but not for the pupil luminance modulation.
Collapse
Affiliation(s)
- Chin-An Wang
- Institute of Cognitive Neuroscience, College of Health Science and Technology, National Central University, Taoyuan City, Taiwan.,Cognitive Intelligence and Precision Healthcare Research Center, National Central University, Taoyuan City, Taiwan
| | - Douglas P Munoz
- Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
22
|
Castellotti S, Scipioni L, Mastandrea S, Del Viva MM. Pupil responses to implied motion in figurative and abstract paintings. PLoS One 2021; 16:e0258490. [PMID: 34634092 PMCID: PMC8504727 DOI: 10.1371/journal.pone.0258490] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 09/28/2021] [Indexed: 11/18/2022] Open
Abstract
Motion can be perceived in static images, such as photos and figurative paintings, representing realistic subjects in motion, with or without directional information (e.g., motion blur or speed lines). Motion impression can be achieved even in non-realistic static images such as motion illusions and abstract paintings. It has been shown that visual motion processing affects the diameter of the pupil, responding differently to real, illusory, and implied motion in photographs (IM). It has been suggested that these different effects might be due to top-down modulations from different cortical areas underlying their processing. It is worthwhile to investigate pupillary response to figurative paintings, since they require an even higher level of interpretation than photos representing the same kind of subjects, given the complexity of cognitive processes involved in the aesthetic experience. Also, pupil responses to abstract paintings allows to study the effect of IM perception in representations devoid of real-life motion cues. We measured pupil responses to IM in figurative and abstract artworks depicting static and dynamic scenes, as rated by a large group of individuals not participating in the following experiment. Since the pupillary response is modulated by the subjective image interpretation, a motion rating test has been used to correct individual pupil data according to whether participants actually perceived the presence of motion in the paintings. Pupil responses to movies showing figurative and abstract subjects, and to motion illusions were also measured, to compare real and illusory motion with painted IM. Movies, both figurative and abstract, elicit the largest pupillary dilation of all static stimuli, whereas motion illusions cause the smallest pupil size, as previously shown. Interestingly, pupil responses to IM depend on the paintings' style. Figurative paintings depicting moving subjects cause more dilation than those representing static figures, and pupil size increases with the strength of IM, as already found with realistic photos. The opposite effect is obtained with abstract artworks. Abstract paintings depicting motion produce less dilation than those depicting stillness. In any case, these results reflect the individual subjective perception of dynamism, as the very same paintings can induce opposite responses in observer which interpreted it as static or dynamic. Overall, our data show that pupil size depends on high-level interpretation of motion in paintings, even when they do not represent real-world scenes. Our findings further suggest that the pupil is modulated by multiple top-down cortical mechanisms, involving the processing of motion, attention, memory, imagination, and other cognitive functions necessary for enjoying a complete aesthetic experience.
Collapse
Affiliation(s)
| | - Lisa Scipioni
- Department of Neurofarba, University of Florence, Florence, Italy
| | | | | |
Collapse
|
23
|
Pandey P, Ray S. Pupil dynamics: A potential proxy of neural preparation for goal-directed eye movement. Eur J Neurosci 2021; 54:6587-6607. [PMID: 34510602 DOI: 10.1111/ejn.15453] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 08/22/2021] [Accepted: 09/03/2021] [Indexed: 01/10/2023]
Abstract
The pupils reflexively constrict or dilate to regulate the influx of light on the retinae. Pupillary light reflex (PLR) is susceptible to many non-visual cognitive processes including covert orientation of attention and planning rapid saccadic eye movement. The frontal eye field (FEF) and superior colliculus (SC), which also send projections to the PLR pathway, are two important areas in primate's brain for planning saccade and orientation of attention. The saccadic reaction time (SRT) and the rate of increase in activity of movement neurons in these areas are inversely correlated. This study addressed how pupil dynamics, activity in the FEF and SC and SRT are related in a saccadic decision-making task. The rate of visually evoked pupil constriction was found inversely related to SRT. This was further verified by simulating a homeomorphic biomechanical model of pupillary muscle plants, wherein we projected signals similar to build-up activity in the FEF and SC to the parasympathetic (constriction) and sympathetic (dilation) division of the PLR pathway, respectively. A striking similarity between simulated and observed dynamics of pupil constriction suggests that PLR is a potential proxy of saccade planning by movement neurons in the FEF and SC. Indistinguishable pupil dynamics when planned saccades were elicited versus when they were cancelled eliminated the possibility that the obligatory pre-saccadic shift of attention alone influenced the rate of pupil constriction. Our study envisages a mechanism of how the oculomotor system influences the autonomic activity in an attempt to timely minimize saccadic visual transients by regulating the influx of light.
Collapse
Affiliation(s)
- Pragya Pandey
- Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj, India
| | - Supriya Ray
- Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj, India
| |
Collapse
|
24
|
Brascamp JW, de Hollander G, Wertheimer MD, DePew AN, Knapen T. Separable pupillary signatures of perception and action during perceptual multistability. eLife 2021; 10:66161. [PMID: 34378532 PMCID: PMC8378849 DOI: 10.7554/elife.66161] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2020] [Accepted: 08/11/2021] [Indexed: 02/04/2023] Open
Abstract
The pupil provides a rich, non-invasive measure of the neural bases of perception and cognition and has been of particular value in uncovering the role of arousal-linked neuromodulation, which alters both cortical processing and pupil size. But pupil size is subject to a multitude of influences, which complicates unique interpretation. We measured pupils of observers experiencing perceptual multistability-an ever-changing subjective percept in the face of unchanging but inconclusive sensory input. In separate conditions, the endogenously generated perceptual changes were either task-relevant or not, allowing a separation between perception-related and task-related pupil signals. Perceptual changes were marked by a complex pupil response that could be decomposed into two components: a dilation tied to task execution and plausibly indicative of an arousal-linked noradrenaline surge, and an overlapping constriction tied to the perceptual transient and plausibly a marker of altered visual cortical representation. Constriction, but not dilation, amplitude systematically depended on the time interval between perceptual changes, possibly providing an overt index of neural adaptation. These results show that the pupil provides a simultaneous reading on interacting but dissociable neural processes during perceptual multistability, and suggest that arousal-linked neuromodulator release shapes action but not perception in these circumstances.
Collapse
Affiliation(s)
- Jan W Brascamp
- Michigan State University, Department of Psychology, East Lansing, United States.,Michigan State University, Neuroscience Program, East Lansing, United States
| | - Gilles de Hollander
- Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland
| | - Michael D Wertheimer
- Michigan State University, Department of Psychology, East Lansing, United States
| | - Ashley N DePew
- Michigan State University, Department of Psychology, East Lansing, United States
| | - Tomas Knapen
- Vrije Universiteit Amsterdam, Amsterdam, Netherlands.,Spinoza Centre for Neuroimaging, Royal Academy of Sciences, Amsterdam, Netherlands
| |
Collapse
|
25
|
Abstract
Pupillometry - the study of temporal changes in pupil diameter as a function of external light stimuli or cognitive processing - requires the accurate and gaze-angle independent measurement of pupil dilation. Expected response amplitudes often are only a few percent relative to a pre-stimulus baseline, thus demanding for sub-millimeter accuracy. Video-based approaches to pupil-size measurement aim at inferring pupil dilation from eye images alone. Eyeball rotation in relation to the recording camera as well as optical effects due to refraction at corneal interfaces can, however, induce so-called pupil foreshortening errors (PFE), i.e. systematic gaze-angle dependent changes of apparent pupil size that are on a par with typical response amplitudes. While PFE and options for its correction have been discussed for remote eye trackers, for head-mounted eye trackers such an assessment is still lacking. In this work, we therefore gauge the extent of PFE in three measurement techniques, all based on eye images recorded with a single near-eye camera. We present both real world experimental data as well as results obtained on synthetically generated eye images. We discuss PFE effects at three different levels of data aggregation: the sample, subject, and population level. In particular, we show that a recently proposed refraction-aware approach employing a mathematical 3D eye model is successful in providing pupil-size measurements which are gaze-angle independent at the population level.
Collapse
|
26
|
DeRoy Milvae K, Kuchinsky SE, Stakhovskaya OA, Goupell MJ. Dichotic listening performance and effort as a function of spectral resolution and interaural symmetry. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2021; 150:920. [PMID: 34470337 PMCID: PMC8346288 DOI: 10.1121/10.0005653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Revised: 06/30/2021] [Accepted: 06/30/2021] [Indexed: 06/13/2023]
Abstract
One potential benefit of bilateral cochlear implants is reduced listening effort in speech-on-speech masking situations. However, the symmetry of the input across ears, possibly related to spectral resolution, could impact binaural benefits. Fifteen young adults with normal hearing performed digit recall with target and interfering digits presented to separate ears and attention directed to the target ear. Recall accuracy and pupil size over time (used as an index of listening effort) were measured for unprocessed, 16-channel vocoded, and 4-channel vocoded digits. Recall accuracy was significantly lower for dichotic (with interfering digits) than for monotic listening. Dichotic recall accuracy was highest when the target was less degraded and the interferer was more degraded. With matched target and interferer spectral resolution, pupil dilation was lower with more degradation. Pupil dilation grew more shallowly over time when the interferer had more degradation. Overall, interferer spectral resolution more strongly affected listening effort than target spectral resolution. These results suggest that interfering speech both lowers performance and increases listening effort, and that the relative spectral resolution of target and interferer affect the listening experience. Ignoring a clearer interferer is more effortful.
Collapse
Affiliation(s)
- Kristina DeRoy Milvae
- Department of Hearing and Speech Sciences, University of Maryland, College Park, Maryland 20742, USA
| | - Stefanie E Kuchinsky
- Audiology and Speech Pathology Center, Walter Reed National Military Medical Center, Bethesda, Maryland 20889, USA
| | - Olga A Stakhovskaya
- Department of Hearing and Speech Sciences, University of Maryland, College Park, Maryland 20742, USA
| | - Matthew J Goupell
- Department of Hearing and Speech Sciences, University of Maryland, College Park, Maryland 20742, USA
| |
Collapse
|
27
|
Castellotti S, Francisci C, Del Viva MM. Pupillary response to real, illusory, and implied motion. PLoS One 2021; 16:e0254105. [PMID: 34197536 PMCID: PMC8248605 DOI: 10.1371/journal.pone.0254105] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 06/21/2021] [Indexed: 11/19/2022] Open
Abstract
The perception of moving objects (real motion) is a critical function for interacting with a dynamic environment. Motion perception can be also induced by particular structural features of static images (illusory motion) or by photographic images of subjects in motion (implied motion, IM). Many cortical areas are involved in motion processing, particularly the medial temporal cortical area (MT), dedicated to the processing of real, illusory, and implied motion. Recently, there has been a growing interest in the influence of high-level visual processes on pupillary responses. However, just a few studies have measured the effect of motion processing on the pupil, and not always with consistent results. Here we systematically investigate the effects of real, illusory, and implied motion on the pupil diameter for the first time, by showing different types of stimuli (movies, illusions, and photos) with the same average luminance to the same observers. We find different pupillary responses depending on the nature of motion. Real motion elicits a larger pupillary dilation than IM, which in turn induces more dilation than control photos representing static subjects (No-IM). The pupil response is sensitive even to the strength of IM, as photos with enhanced IM (blur, motion streaks, speed lines) induce larger dilation than simple freezed IM (subjects captured in the instant they are moving). Also, the subject represented in the stimulus matters: human figures are interpreted as more dynamic and induce larger dilation than objects/animals. Interestingly, illusory motion induces much less dilation than all the other motion categories, despite being seen as moving. Overall, pupil responses depend on the individual perception of dynamicity, confirming that the pupil is modulated by the subjective interpretation of complex stimuli. We argue that the different pupillary responses to real, illusory, and implied motion reflect the top-down modulations of different cortical areas involved in their processing.
Collapse
Affiliation(s)
| | - Carlo Francisci
- Department of Neurofarba, University of Florence, Florence, Italy
| | | |
Collapse
|
28
|
Zandi B, Lode M, Herzog A, Sakas G, Khanh TQ. PupilEXT: Flexible Open-Source Platform for High-Resolution Pupillometry in Vision Research. Front Neurosci 2021; 15:676220. [PMID: 34220432 PMCID: PMC8249868 DOI: 10.3389/fnins.2021.676220] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 04/28/2021] [Indexed: 12/12/2022] Open
Abstract
The human pupil behavior has gained increased attention due to the discovery of the intrinsically photosensitive retinal ganglion cells and the afferent pupil control path's role as a biomarker for cognitive processes. Diameter changes in the range of 10-2 mm are of interest, requiring reliable and characterized measurement equipment to accurately detect neurocognitive effects on the pupil. Mostly commercial solutions are used as measurement devices in pupillometry which is associated with high investments. Moreover, commercial systems rely on closed software, restricting conclusions about the used pupil-tracking algorithms. Here, we developed an open-source pupillometry platform consisting of hardware and software competitive with high-end commercial stereo eye-tracking systems. Our goal was to make a professional remote pupil measurement pipeline for laboratory conditions accessible for everyone. This work's core outcome is an integrated cross-platform (macOS, Windows and Linux) pupillometry software called PupilEXT, featuring a user-friendly graphical interface covering the relevant requirements of professional pupil response research. We offer a selection of six state-of-the-art open-source pupil detection algorithms (Starburst, Swirski, ExCuSe, ElSe, PuRe and PuReST) to perform the pupil measurement. A developed 120-fps pupillometry demo system was able to achieve a calibration accuracy of 0.003 mm and an averaged temporal pupil measurement detection accuracy of 0.0059 mm in stereo mode. The PupilEXT software has extended features in pupil detection, measurement validation, image acquisition, data acquisition, offline pupil measurement, camera calibration, stereo vision, data visualization and system independence, all combined in a single open-source interface, available at https://github.com/openPupil/Open-PupilEXT.
Collapse
Affiliation(s)
- Babak Zandi
- Laboratory of Lighting Technology, Department of Electrical Engineering and Information Technology, Technical University of Darmstadt, Darmstadt, Germany
| | - Moritz Lode
- Laboratory of Lighting Technology, Department of Electrical Engineering and Information Technology, Technical University of Darmstadt, Darmstadt, Germany
| | - Alexander Herzog
- Laboratory of Lighting Technology, Department of Electrical Engineering and Information Technology, Technical University of Darmstadt, Darmstadt, Germany
| | - Georgios Sakas
- Interactive Graphic Systems, Department of Computer Science, Technical University of Darmstadt, Darmstadt, Germany
| | - Tran Quoc Khanh
- Laboratory of Lighting Technology, Department of Electrical Engineering and Information Technology, Technical University of Darmstadt, Darmstadt, Germany
| |
Collapse
|
29
|
Castellotti S, Conti M, Feitosa-Santana C, Del Viva MM. Pupillary response to representations of light in paintings. J Vis 2021; 20:14. [PMID: 33052409 PMCID: PMC7571318 DOI: 10.1167/jov.20.10.14] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
It is known that, although the level of light is the primary determinant of pupil size, cognitive factors can also affect pupil diameter. It has been demonstrated that photographs of the sun produce pupil constriction independently of their luminance and other low-level features, suggesting that high-level visual processing may also modulate pupil response. Here, we measure pupil response to artistic paintings of the sun, moon, or containing a uniform lighting, that, being mediated by the artist's interpretation of reality and his technical rendering, require an even higher level of interpretation compared with photographs. We also study how chromatic content and spatial layout affect the results by presenting grey-scale and inverted versions of each painting. Finally, we assess directly with a categorization test how subjective image interpretation affects pupil response. We find that paintings with the sun elicit a smaller pupil size than paintings with the moon, or paintings containing no visible light source. The effect produced by sun paintings is reduced by disrupting contextual information, such as by removing color or manipulating the relations between paintings features that make more difficult to identify the source of light. Finally, and more importantly, pupil diameter changes according to observers’ interpretation of the scene represented in the same stimulus. In conclusion, results show that the subcortical pupillary response to light is modulated by subjective interpretation of luminous objects, suggesting the involvement of cortical systems in charge of cognitive processes, such as attention, object recognition, familiarity, memory, and imagination.
Collapse
Affiliation(s)
| | - Martina Conti
- Department of Neurofarba, University of Florence, Florence, Italy.,
| | - Claudia Feitosa-Santana
- Federal University of ABC, Sao Bernardo do Campo, Brazil.,Neuroscience for Human Development, Sao Paulo, Brazil.,
| | | |
Collapse
|
30
|
de Winter JCF, Petermeijer SM, Kooijman L, Dodou D. Replicating five pupillometry studies of Eckhard Hess. Int J Psychophysiol 2021; 165:145-205. [PMID: 33766646 DOI: 10.1016/j.ijpsycho.2021.03.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2020] [Revised: 03/01/2021] [Accepted: 03/09/2021] [Indexed: 02/02/2023]
Abstract
Several papers by Eckhard Hess from the 1960s and 1970s report that the pupils dilate or constrict according to the interest value, arousing content, or mental demands of visual stimuli. However, Hess mostly used small sample sizes and undocumented luminance control. In a first experiment (N = 182) and a second preregistered experiment (N = 147), we replicated five studies of Hess using modern equipment. Our experiments (1) did not support the hypothesis of gender differences in pupil diameter change with respect to baseline (PC) when viewing stimuli of different interest value, (2) showed that solving more difficult multiplications yields a larger PC in the seconds before providing an answer and a larger maximum PC, but a smaller PC at a fixed time after the onset of the multiplication, (3) did not support the hypothesis that participants' PC mimics the pupil diameter in a pair of schematic eyes but not in single-eyed or three-eyed stimuli, (4) did not support the hypothesis of gender differences in PC when watching a video of a male trying to escape a mob, and (5) supported the hypothesis that arousing words yield a higher PC than non-arousing words. Although we did not observe consistent gender differences in PC, additional analyses showed gender differences in eye movements towards erogenous zones. Furthermore, PC strongly correlated with the luminance of the locations where participants looked. Overall, our replications confirm Hess's findings that pupils dilate in response to mental demands and stimuli of an arousing nature. Hess's hypotheses regarding pupil mimicry and gender differences in pupil dilation did not replicate.
Collapse
Affiliation(s)
- J C F de Winter
- Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, the Netherlands.
| | - S M Petermeijer
- Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, the Netherlands
| | - L Kooijman
- Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, the Netherlands
| | - D Dodou
- Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, the Netherlands
| |
Collapse
|
31
|
Is accommodation a confounder in pupillometry research? Biol Psychol 2021; 160:108046. [PMID: 33581231 DOI: 10.1016/j.biopsycho.2021.108046] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2020] [Revised: 02/07/2021] [Accepted: 02/07/2021] [Indexed: 11/24/2022]
Abstract
Much psychological research uses pupil diameter measurements to investigate the cognitive and emotional effects of visual stimuli. A potential problem is that accommodating at a nearby point causes the pupil to constrict. This study examined to what extent accommodation is a confounder in pupillometry research. Participants solved multiplication problems at different distances (Experiment 1) and looked at line drawings with different monocular depth cues (Experiment 2) while their pupil diameter, refraction, and vergence angle were recorded using a photorefractor. Experiment 1 showed that the pupils dilated while performing the multiplications, for all presentation distances. Pupillary constriction due to accommodation was not strong enough to override pupil dilation due to cognitive load. Experiment 2 showed that monocular depth cues caused a small shift in refraction in the expected direction. We conclude that, for the young student sample we used, pupil diameter measurements are not substantially affected by accommodation.
Collapse
|
32
|
Abstract
Eye-tracking is widely used throughout the scientific community, from vision science and psycholinguistics to marketing and human-computer interaction. Surprisingly, there is little consistency and transparency in preprocessing steps, making replicability and reproducibility difficult. To increase replicability, reproducibility, and transparency, a package in R (a free and widely used statistical programming environment) called gazeR was created to read and preprocess two types of data: gaze position and pupil size. For gaze position data, gazeR has functions for reading in raw eye-tracking data, formatting it for analysis, converting from gaze coordinates to areas of interest, and binning and aggregating data. For data from pupillometry studies, the gazeR package has functions for reading in and merging multiple raw pupil data files, removing observations with too much missing data, eliminating artifacts, blink identification and interpolation, subtractive baseline correction, and binning and aggregating data. The package is open-source and freely available for download and installation: https://github.com/dmirman/gazer . We provide step-by-step analyses of data from two tasks exemplifying the package's capabilities.
Collapse
|
33
|
Huijser S, Verkaik M, van Vugt MK, Taatgen NA. Captivated by thought: "Sticky" thinking leaves traces of perceptual decoupling in task-evoked pupil size. PLoS One 2020; 15:e0243532. [PMID: 33296415 PMCID: PMC7725397 DOI: 10.1371/journal.pone.0243532] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2019] [Accepted: 11/24/2020] [Indexed: 11/17/2022] Open
Abstract
Throughout the day, we may sometimes catch ourselves in patterns of thought that we experience as rigid and difficult to disengage from. Such "sticky" thinking can be highly disruptive to ongoing tasks, and when it turns into rumination constitutes a vulnerability for mental disorders such as depression and anxiety. The main goal of the present study was to explore the stickiness dimension of thought, by investigating how stickiness is reflected in task performance and pupil size. To measure spontaneous thought processes, we asked participants to perform a sustained attention to response task (SART), in which we embedded the participant's concerns to potentially increase the probability of observing sticky thinking. The results indicated that sticky thinking was most frequently experienced when participants were disengaged from the task. Such episodes of sticky thought could be discriminated from neutral and non-sticky thought by an increase in errors on infrequent no-go trials. Furthermore, we found that sticky thought was associated with smaller pupil responses during correct responding. These results demonstrate that participants can report on the stickiness of their thought, and that stickiness can be investigated using pupillometry. In addition, the results suggest that sticky thought may limit attention and exertion of cognitive control to the task.
Collapse
Affiliation(s)
- Stefan Huijser
- Bernoulli Institute for Mathematics, Computer Science, and Artificial Intelligence, University of Groningen, Groningen, Netherlands
| | - Mathanja Verkaik
- Bernoulli Institute for Mathematics, Computer Science, and Artificial Intelligence, University of Groningen, Groningen, Netherlands
| | - Marieke K van Vugt
- Bernoulli Institute for Mathematics, Computer Science, and Artificial Intelligence, University of Groningen, Groningen, Netherlands
| | - Niels A Taatgen
- Bernoulli Institute for Mathematics, Computer Science, and Artificial Intelligence, University of Groningen, Groningen, Netherlands
| |
Collapse
|
34
|
Martin JT, Whittaker AH, Johnston SJ. Component processes in free-viewing visual search: Insights from fixation-aligned pupillary response averaging. J Vis 2020; 20:5. [PMID: 32634226 PMCID: PMC7424908 DOI: 10.1167/jov.20.7.5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Accepted: 05/09/2020] [Indexed: 12/30/2022] Open
Abstract
Pupil size changes during a visual search may reflect cognitive processes, such as effort and memory accumulation, but methodological confounds and the general lack of literature in this area leave the reliability of findings open to question. We used a novel synthesis of experimental methods and averaging techniques to explore how cognitive processing unfolds during free-viewing visual search for multiple targets. Twenty-seven participants completed 152 searches across two separate 1-hour sessions. The number of targets present (Targets: 0, 1, 2, and 3) in each trial was the main manipulation and the task was to "find all of the targets" and report the total via mouse-click at the end of the trial. Search time lasted for 10 seconds or until the participant purported to have found all of the targets, in which case they could terminate the search via keypress. Whole-trial pupil analysis revealed a significant effect of button pressing as well as a significant main effect of targets for trials that were not self-terminated via button press. Fixation-aligned pupil responses revealed transient modulations in pupil size following initial fixations on targets but not distractors and refixations on both targets and distractors. Owing to rigorous control over experimental confounds and a detailed analysis and correction of eye-movement-related measurement error, we confidently discuss these findings in terms of task-related processing and underlying brain activity.
Collapse
Affiliation(s)
- Joel T. Martin
- School of Human and Health Sciences, Department of Psychology, University of Swansea, Swansea, Wales, UK
- Institute of Biomedical Engineering, University of Oxford, UK
| | | | - Stephen J. Johnston
- School of Human and Health Sciences, Department of Psychology, University of Swansea, Swansea, Wales, UK
| |
Collapse
|
35
|
Carter BT, Luke SG. Best practices in eye tracking research. Int J Psychophysiol 2020; 155:49-62. [PMID: 32504653 DOI: 10.1016/j.ijpsycho.2020.05.010] [Citation(s) in RCA: 77] [Impact Index Per Article: 19.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2019] [Revised: 05/26/2020] [Accepted: 05/27/2020] [Indexed: 12/14/2022]
Abstract
This guide describes best practices in using eye tracking technology for research in a variety of disciplines. A basic outline of the anatomy and physiology of the eyes and of eye movements is provided, along with a description of the sorts of research questions eye tracking can address. We then explain how eye tracking technology works and what sorts of data it generates, and provide guidance on how to select and use an eye tracker as well as selecting appropriate eye tracking measures. Challenges to the validity of eye tracking studies are described, along with recommendations for overcoming these challenges. We then outline correct reporting standards for eye tracking studies.
Collapse
|
36
|
Noradrenergic Responsiveness Supports Selective Attention across the Adult Lifespan. J Neurosci 2020; 40:4372-4390. [PMID: 32317388 DOI: 10.1523/jneurosci.0398-19.2020] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2019] [Revised: 03/05/2020] [Accepted: 03/26/2020] [Indexed: 12/17/2022] Open
Abstract
Selectively attending to relevant information while blocking out distractors is crucial for goal-directed behavior, yet with advancing age, deficits emerge in attentional selectivity. Decrements in attention have been associated with altered noradrenergic activity in animals. However, research linking noradrenergic functioning to attention in aging humans is scarce, likely reflecting long-standing methodological challenges in noninvasive assessments. We studied whether age-related differences in the noradrenergic system predict differences in attention. We measured pupil dilation, a noninvasive marker of arousal-related norepinephrine (NE) release, while concurrently recording the EEG of male younger (N = 39; 25.2 ± 3.2 years) and older adults (N = 38; 70.6 ± 2.7 years). Arousal was modulated on a trial-by-trial basis using fear-conditioned (CS+) stimuli. During conditioning, pupil and EEG markers related to heightened arousal were identified. Afterward, in a dichotic listening task, participants were cued to direct attention to either the left or right ear while highly similar syllable pairs were presented simultaneously to both ears. During the dichotic listening task, presentation of fear-conditioned stimuli reinstated the acquired arousal response, as reflected in pupil and EEG α-β band responses. Critically, pupil dilation to CS+ was correlated with stronger EEG α-β desynchronization, suggesting a common dependence on NE release. On a behavioral level, stronger arousal reactions were associated with better attention. In particular, structural equation modeling revealed that the responsiveness of the NE system is associated with attention on a latent construct level, measured by several indicator tasks. Overall, our results suggest that the responsiveness of the NE system supports attention across the lifespan.SIGNIFICANCE STATEMENT In old age, the ability to selectively process relevant aspects of the environment fades. Animal research suggests that the neuromodulator norepinephrine helps to maintain selective attention. We tested younger and older adults across a variety of attention tasks. In addition, we used arousing stimuli to experimentally activate participants' noradrenergic system while recording pupillometry and EEG to infer its functional capacity. Older adults showed compromised attention and reduced noradrenergic responsiveness as indicated by interrelated pupil and EEG markers. Crucially, in both age groups, a more responsive noradrenergic system was strongly associated with attention. Our findings link animal and human studies on the neural underpinning of attention in aging and underscore the importance of the noradrenergic system in late-life cognition.
Collapse
|
37
|
Hu X, Hisakata R, Kaneko H. Effects of stimulus size, eccentricity, luminance, and attention on pupillary light response examined by concentric stimulus. Vision Res 2020; 170:35-45. [PMID: 32244112 DOI: 10.1016/j.visres.2020.03.008] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2020] [Revised: 03/18/2020] [Accepted: 03/18/2020] [Indexed: 12/17/2022]
Abstract
Previous studies show that the amplitude of pupillary light response (PLR) depends on the corneal flux density (CFD), which is the product of stimulus area by luminance. However, the contribution of CFD has been investigated only when the stimulus was centered on the fovea, whereas perceived luminance to pupillary response would reduce with stimulus eccentricity. Additionally, it has been shown recently that attentional state modulates pupillary response. In this study, we aimed to clarify the complete mechanisms of PLR by manipulating the stimulus size, eccentricity, luminance, and the participants' attentional states. We focused on four indices to examine PLR, that is, pupillary latency (PL), maximum constriction velocity (MCV), maximum constriction (MC), and mean pupil change (MPC). Results showed that PL was a function of CFD, whereas MCV, MC, and MPC were functions of both CFD and stimulus eccentricity. Furthermore, the magnitude of effect due to stimulus eccentricity for MCV and MC was different from that for MPC. These results provided new evidence that the different processing systems in PLR existed.
Collapse
Affiliation(s)
- Xiaofei Hu
- Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, 4259-G2-3 Nagatsuta-Cho, Midori-Ku, Yokohama 226-8502, Japan.
| | - Rumi Hisakata
- Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, 4259-G2-3 Nagatsuta-Cho, Midori-Ku, Yokohama 226-8502, Japan
| | - Hirohiko Kaneko
- Department of Information and Communications Engineering, School of Engineering, Tokyo Institute of Technology, 4259-G2-3 Nagatsuta-Cho, Midori-Ku, Yokohama 226-8502, Japan
| |
Collapse
|
38
|
Abstract
Pupillometry has been one of the most widely used response systems in psychophysiology. Changes in pupil size can reflect diverse cognitive and emotional states, ranging from arousal, interest and effort to social decisions, but they are also widely used in clinical practice to assess patients’ brain functioning. As a result, research involving pupil size measurements has been reported in practically all psychology, psychiatry, and psychophysiological research journals, and now it has found its way into the primatology literature as well as into more practical applications, such as using pupil size as a measure of fatigue or a safety index during driving. The different systems used for recording pupil size are almost as variable as its applications, and all yield, as with many measurement techniques, a substantial amount of noise in addition to the real pupillometry data. Before analyzing pupil size, it is therefore of crucial importance first to detect this noise and deal with it appropriately, even prior to (if need be) resampling and baseline-correcting the data. In this article we first provide a short review of the literature on pupil size measurements, then we highlight the most important sources of noise and show how these can be detected. Finally, we provide step-by-step guidelines that will help those interested in pupil size to preprocess their data correctly. These guidelines are accompanied by an open source MATLAB script (available at https://github.com/ElioS-S/pupil-size). Given that pupil diameter is easily measured by standard eyetracking technologies and can provide fundamental insights into cognitive and emotional processes, it is hoped that this article will further motivate scholars from different disciplines to study pupil size.
Collapse
|
39
|
Ehinger BV, Dimigen O. Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis. PeerJ 2019; 7:e7838. [PMID: 31660265 PMCID: PMC6815663 DOI: 10.7717/peerj.7838] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2019] [Accepted: 09/05/2019] [Indexed: 12/21/2022] Open
Abstract
Electrophysiological research with event-related brain potentials (ERPs) is increasingly moving from simple, strictly orthogonal stimulation paradigms towards more complex, quasi-experimental designs and naturalistic situations that involve fast, multisensory stimulation and complex motor behavior. As a result, electrophysiological responses from subsequent events often overlap with each other. In addition, the recorded neural activity is typically modulated by numerous covariates, which influence the measured responses in a linear or non-linear fashion. Examples of paradigms where systematic temporal overlap variations and low-level confounds between conditions cannot be avoided include combined electroencephalogram (EEG)/eye-tracking experiments during natural vision, fast multisensory stimulation experiments, and mobile brain/body imaging studies. However, even "traditional," highly controlled ERP datasets often contain a hidden mix of overlapping activity (e.g., from stimulus onsets, involuntary microsaccades, or button presses) and it is helpful or even necessary to disentangle these components for a correct interpretation of the results. In this paper, we introduce unfold, a powerful, yet easy-to-use MATLAB toolbox for regression-based EEG analyses that combines existing concepts of massive univariate modeling ("regression-ERPs"), linear deconvolution modeling, and non-linear modeling with the generalized additive model into one coherent and flexible analysis framework. The toolbox is modular, compatible with EEGLAB and can handle even large datasets efficiently. It also includes advanced options for regularization and the use of temporal basis functions (e.g., Fourier sets). We illustrate the advantages of this approach for simulated data as well as data from a standard face recognition experiment. In addition to traditional and non-conventional EEG/ERP designs, unfold can also be applied to other overlapping physiological signals, such as pupillary or electrodermal responses. It is available as open-source software at http://www.unfoldtoolbox.org.
Collapse
Affiliation(s)
- Benedikt V. Ehinger
- Institute of Cognitive Science, Universität Osnabrück, Osnabrück, Germany
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| | - Olaf Dimigen
- Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany
| |
Collapse
|
40
|
Abstract
Pupil dilation is an effective indicator of cognitive and affective processes. Although several eyetracker systems on the market can provide effective solutions for pupil dilation measurement, there is a lack of tools for processing and analyzing the data provided by these systems. For this reason, we developed CHAP: open-source software written in MATLAB. This software provides a user-friendly graphical user interface for processing and analyzing pupillometry data. Our software creates uniform conventions for the preprocessing and analysis of pupillometry data and provides a quick and easy-to-use tool for researchers interested in pupillometry. To download CHAP or join our mailing list, please visit CHAP's website: http://in.bgu.ac.il/en/Labs/CNL/chap .
Collapse
|
41
|
Hu X, Hisakata R, Kaneko H. Effects of spatial frequency and attention on pupillary response. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2019; 36:1699-1708. [PMID: 31674435 DOI: 10.1364/josaa.36.001699] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Accepted: 08/22/2019] [Indexed: 06/10/2023]
Abstract
Research has shown that the pupil responds differently depending on the spatial frequency of the gazing stimulus. In this study, we examined the effects of spatial- and object-based attention on pupillary response as a function of spatial frequency using grating stimuli and filtered natural images by manipulating the participants' attentional state. Furthermore, we aimed to obtain the pupillary response to spatial frequency accurately by reducing the contamination of unintended spatial frequency components in the stimulus by using gratings with a Gabor envelope. We revealed that all stimuli could elicit large pupil constriction for an intermediate range (2-8 c/d) of spatial frequency and that both spatial- and object-based attention modulate the pupillary response function to spatial frequency. These facts may enhance Human Computer Interaction design to use people's attentional state.
Collapse
|
42
|
Frost-Karlsson M, Galazka MA, Gillberg C, Gillberg C, Miniscalco C, Billstedt E, Hadjikhani N, Åsberg Johnels J. Social scene perception in autism spectrum disorder: An eye-tracking and pupillometric study. J Clin Exp Neuropsychol 2019; 41:1024-1032. [DOI: 10.1080/13803395.2019.1646214] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Affiliation(s)
- Morgan Frost-Karlsson
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
- Center for Social and Affective Neuroscience, Linköping University, Linköping, Sweden
| | | | - Christopher Gillberg
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
| | - Carina Gillberg
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
| | - Carmela Miniscalco
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
| | - Eva Billstedt
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
| | - Nouchine Hadjikhani
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
- Harvard Medical School/MGH/MIT, Athinoula A. Martinos Center for Biomedical Imaging, Boston, MA, USA
| | - Jakob Åsberg Johnels
- Gillberg Neuropsychiatry Centre, Sahlgrenska academy, University of Gothenburg, Gothenburg, Sweden
- Section for Speech and Language Pathology, Sahlgrenska academy University of Gothenburg, Gothenburg, Sweden
| |
Collapse
|
43
|
Lohani M, Payne BR, Strayer DL. A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving. Front Hum Neurosci 2019; 13:57. [PMID: 30941023 PMCID: PMC6434408 DOI: 10.3389/fnhum.2019.00057] [Citation(s) in RCA: 90] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2018] [Accepted: 02/01/2019] [Indexed: 11/13/2022] Open
Abstract
As driving functions become increasingly automated, motorists run the risk of becoming cognitively removed from the driving process. Psychophysiological measures may provide added value not captured through behavioral or self-report measures alone. This paper provides a selective review of the psychophysiological measures that can be utilized to assess cognitive states in real-world driving environments. First, the importance of psychophysiological measures within the context of traffic safety is discussed. Next, the most commonly used physiology-based indices of cognitive states are considered as potential candidates relevant for driving research. These include: electroencephalography and event-related potentials, optical imaging, heart rate and heart rate variability, blood pressure, skin conductance, electromyography, thermal imaging, and pupillometry. For each of these measures, an overview is provided, followed by a discussion of the methods for measuring it in a driving context. Drawing from recent empirical driving and psychophysiology research, the relative strengths and limitations of each measure are discussed to highlight each measures' unique value. Challenges and recommendations for valid and reliable quantification from lab to (less predictable) real-world driving settings are considered. Finally, we discuss measures that may be better candidates for a near real-time assessment of motorists' cognitive states that can be utilized in applied settings outside the lab. This review synthesizes the literature on in-vehicle psychophysiological measures to advance the development of effective human-machine driving interfaces and driver support systems.
Collapse
Affiliation(s)
- Monika Lohani
- Department of Educational Psychology, University of Utah, Salt Lake City, UT, United States
| | - Brennan R. Payne
- Department of Psychology, University of Utah, Salt Lake City, UT, United States
| | - David L. Strayer
- Department of Psychology, University of Utah, Salt Lake City, UT, United States
| |
Collapse
|
44
|
Too good to be true? Psychological responses to uncommon options in risk–reward environments. JOURNAL OF BEHAVIORAL DECISION MAKING 2019. [DOI: 10.1002/bdm.2116] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
45
|
van Rij J, Hendriks P, van Rijn H, Baayen RH, Wood SN. Analyzing the Time Course of Pupillometric Data. Trends Hear 2019; 23:2331216519832483. [PMID: 31081486 PMCID: PMC6535748 DOI: 10.1177/2331216519832483] [Citation(s) in RCA: 69] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2018] [Revised: 11/30/2018] [Accepted: 12/12/2018] [Indexed: 11/21/2022] Open
Abstract
This article provides a tutorial for analyzing pupillometric data. Pupil dilation has become increasingly popular in psychological and psycholinguistic research as a measure to trace language processing. However, there is no general consensus about procedures to analyze the data, with most studies analyzing extracted features from the pupil dilation data instead of analyzing the pupil dilation trajectories directly. Recent studies have started to apply nonlinear regression and other methods to analyze the pupil dilation trajectories directly, utilizing all available information in the continuously measured signal. This article applies a nonlinear regression analysis, generalized additive mixed modeling, and illustrates how to analyze the full-time course of the pupil dilation signal. The regression analysis is particularly suited for analyzing pupil dilation in the fields of psychological and psycholinguistic research because generalized additive mixed models can include complex nonlinear interactions for investigating the effects of properties of stimuli (e.g., formant frequency) or participants (e.g., working memory score) on the pupil dilation signal. To account for the variation due to participants and items, nonlinear random effects can be included. However, one of the challenges for analyzing time series data is dealing with the autocorrelation in the residuals, which is rather extreme for the pupillary signal. On the basis of simulations, we explain potential causes of this extreme autocorrelation, and on the basis of the experimental data, we show how to reduce their adverse effects, allowing a much more coherent interpretation of pupillary data than possible with feature-based techniques.
Collapse
|
46
|
Liao HI, Yoneya M, Kashino M, Furukawa S. Pupillary dilation response reflects surprising moments in music. J Eye Mov Res 2018; 11. [PMID: 33828696 PMCID: PMC7899049 DOI: 10.16910/jemr.11.2.13] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
There are indications that the pupillary dilation response (PDR) reflects surprising moments in an auditory sequence such as the appearance of a deviant noise against repetitively presented pure tones (4), and salient and loud sounds that are evaluated by human paricipants subjectively (12). In the current study, we further examined whether the reflection of PDR in auditory surprise can be accumulated and revealed in complex and yet structured auditory stimuli, i.e., music, and when the surprise is defined subjectively. Participants listened to 15 excerpts of music while their pupillary responses were recorded. In the surprise-rating session, participants rated how surprising an instance in the excerpt was, i.e., rich in variation versus monotonous, while they listened to it. In the passive-listening session, they listened to the same 15 excerpts again but were not involved in any task. The pupil diameter data obtained from both sessions were time-aligned to the rating data obtained from the surprise-rating session. Results showed that in both sessions, mean pupil diameter was larger at moments rated more surprising than unsurprising. The result suggests that the PDR reflects surprise in music automatically.
Collapse
Affiliation(s)
- Hsin-I Liao
- NTT Communication Science Laboratories, NTT Cooperation,, Japan
| | - Makoto Yoneya
- NTT Communication Science Laboratories, NTT Cooperation,, Japan
| | - Makio Kashino
- NTT Communication Science Laboratories, NTT Cooperation,, Japan
| | | |
Collapse
|
47
|
Abstract
Measurement of pupil size (pupillometry) has recently gained renewed interest from psychologists, but there is little agreement on how pupil-size data is best analyzed. Here we focus on one aspect of pupillometric analyses: baseline correction, i.e., analyzing changes in pupil size relative to a baseline period. Baseline correction is useful in experiments that investigate the effect of some experimental manipulation on pupil size. In such experiments, baseline correction improves statistical power by taking into account random fluctuations in pupil size over time. However, we show that baseline correction can also distort data if unrealistically small pupil sizes are recorded during the baseline period, which can easily occur due to eye blinks, data loss, or other distortions. Divisive baseline correction (corrected pupil size = pupil size/baseline) is affected more strongly by such distortions than subtractive baseline correction (corrected pupil size = pupil size - baseline). We discuss the role of baseline correction as a part of preprocessing of pupillometric data, and make five recommendations: (1) before baseline correction, perform data preprocessing to mark missing and invalid data, but assume that some distortions will remain in the data; (2) use subtractive baseline correction; (3) visually compare your corrected and uncorrected data; (4) be wary of pupil-size effects that emerge faster than the latency of the pupillary response allows (within ±220 ms after the manipulation that induces the effect); and (5) remove trials on which baseline pupil size is unrealistically small (indicative of blinks and other distortions).
Collapse
|
48
|
Pupillary response to complex interdependent tasks: A cognitive-load theory perspective. Behav Res Methods 2018; 49:1905-1919. [PMID: 27928748 DOI: 10.3758/s13428-016-0833-y] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Pupil dilation is known to indicate cognitive load. In this study, we looked at the average pupillary responses of a cohort of 29 undergraduate students during graphical problem solving. Three questions were asked, based on the same graphical input. The questions were interdependent and comprised multiple steps. We propose a novel way of analyzing pupillometry data for such tasks on the basis of eye fixations, a commonly used eyetracking parameter. We found that pupil diameter increased during the solution process. However, pupil diameter did not always reflect the expected cognitive load. This result was studied within a cognitive-load theory model. Higher-performing students showed evidence of germane load and schema creation, indicating use of the interdependent nature of the tasks to inform their problem-solving process. However, lower-performing students did not recognize the interdependent nature of the tasks and solved each problem independently, which was expressed in a markedly different pupillary response pattern. We discuss the import of our findings for instructional design.
Collapse
|
49
|
Paladini RE, Müri RM, Meichtry J, Nef T, Mast FW, Mosimann UP, Nyffeler T, Cazzoli D. The Influence of Alertness on the Spatial Deployment of Visual Attention is Mediated by the Excitability of the Posterior Parietal Cortices. Cereb Cortex 2018; 27:233-243. [PMID: 28013233 PMCID: PMC5939216 DOI: 10.1093/cercor/bhw390] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2016] [Accepted: 12/01/2016] [Indexed: 12/25/2022] Open
Abstract
With a reduced level of alertness, healthy individuals typically show a rightward shift when deploying visual attention in space. The impact of alertness on the neural networks governing visuospatial attention is, however, poorly understood. By using a transcranial magnetic stimulation twin-coil approach, the present study aimed at investigating the effects of an alertness manipulation on the excitability of the left and the right posterior parietal cortices (PPCs), crucial nodes of the visuospatial attentional network. Participants’ visuospatial attentional deployment was assessed with a free visual exploration task and concurrent eye tracking. Their alertness level was manipulated through the time of the day, that is, by testing chronotypically defined evening types both during their circadian on- and off-peak times. The results revealed an increased excitability of the left compared with the right PPC during low alertness. On the horizontal dimension, these results were accompanied by a significant rightward shift in the center and a bilateral narrowing in the periphery of the visual exploration field, as well as a central upward shift on the vertical dimension. The findings show that the manipulation of non-spatial attentional aspects (i.e., alertness) can affect visuospatial attentional deployment and modulate the excitability of areas subtending spatial attentional control.
Collapse
Affiliation(s)
- Rebecca E Paladini
- Gerontechnology and Rehabilitation Group, University of Bern, Bern, Switzerland
| | - René M Müri
- Gerontechnology and Rehabilitation Group, University of Bern, Bern, Switzerland.,Perception and Eye Movement Laboratory, Departments of Neurology and Clinical Research, Inselspital, University Hospital Bern, and University of Bern, Bern, Switzerland.,Division of Cognitive and Restorative Neurology, Department of Neurology, Inselspital, University Hospital Bern, and University of Bern, Bern, Switzerland.,Center for Cognition, Learning and Memory, University of Bern, Bern, Switzerland
| | - Jurka Meichtry
- Perception and Eye Movement Laboratory, Departments of Neurology and Clinical Research, Inselspital, University Hospital Bern, and University of Bern, Bern, Switzerland.,Division of Cognitive and Restorative Neurology, Department of Neurology, Inselspital, University Hospital Bern, and University of Bern, Bern, Switzerland
| | - Tobias Nef
- Gerontechnology and Rehabilitation Group, University of Bern, Bern, Switzerland.,ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
| | - Fred W Mast
- Center for Cognition, Learning and Memory, University of Bern, Bern, Switzerland.,Department of Psychology, University of Bern, Bern, Switzerland
| | - Urs P Mosimann
- Gerontechnology and Rehabilitation Group, University of Bern, Bern, Switzerland.,ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland.,University Hospital of Old Age Psychiatry, University of Bern, Bern, Switzerland
| | - Thomas Nyffeler
- Gerontechnology and Rehabilitation Group, University of Bern, Bern, Switzerland.,Perception and Eye Movement Laboratory, Departments of Neurology and Clinical Research, Inselspital, University Hospital Bern, and University of Bern, Bern, Switzerland.,Center of Neurology and Neurorehabilitation, Luzerner Kantonsspital, Lucerne, Switzerland
| | - Dario Cazzoli
- Gerontechnology and Rehabilitation Group, University of Bern, Bern, Switzerland.,ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
| |
Collapse
|
50
|
Winn MB, Wendt D, Koelewijn T, Kuchinsky SE. Best Practices and Advice for Using Pupillometry to Measure Listening Effort: An Introduction for Those Who Want to Get Started. Trends Hear 2018; 22:2331216518800869. [PMID: 30261825 PMCID: PMC6166306 DOI: 10.1177/2331216518800869] [Citation(s) in RCA: 113] [Impact Index Per Article: 18.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2018] [Revised: 08/07/2018] [Accepted: 08/14/2018] [Indexed: 01/12/2023] Open
Abstract
Within the field of hearing science, pupillometry is a widely used method for quantifying listening effort. Its use in research is growing exponentially, and many labs are (considering) applying pupillometry for the first time. Hence, there is a growing need for a methods paper on pupillometry covering topics spanning from experiment logistics and timing to data cleaning and what parameters to analyze. This article contains the basic information and considerations needed to plan, set up, and interpret a pupillometry experiment, as well as commentary about how to interpret the response. Included are practicalities like minimal system requirements for recording a pupil response and specifications for peripheral, equipment, experiment logistics and constraints, and different kinds of data processing. Additional details include participant inclusion and exclusion criteria and some methodological considerations that might not be necessary in other auditory experiments. We discuss what data should be recorded and how to monitor the data quality during recording in order to minimize artifacts. Data processing and analysis are considered as well. Finally, we share insights from the collective experience of the authors and discuss some of the challenges that still lie ahead.
Collapse
Affiliation(s)
- Matthew B. Winn
- Speech-Language-Hearing Sciences,
University
of Minnesota, Minneapolis, MN, USA
| | - Dorothea Wendt
- Eriksholm Research Centre, Snekkersten,
Denmark
- Hearing Systems, Department of
Electrical Engineering, Technical University of Denmark, Kongens Lyngby,
Denmark
| | - Thomas Koelewijn
- Section Ear & Hearing, Department of
Otolaryngology–Head and Neck Surgery, Amsterdam Public Health Research Institute, VU
University Medical Center, the Netherlands
| | - Stefanie E. Kuchinsky
- National Military Audiology and Speech
Pathology Center, Walter Reed National Military Medical Center, Bethesda, MD,
USA
| |
Collapse
|