1
|
Cai Y, Strauch C, Van der Stigchel S, Naber M. Open-DPSM: An open-source toolkit for modeling pupil size changes to dynamic visual inputs. Behav Res Methods 2024; 56:5605-5621. [PMID: 38082113 PMCID: PMC11335788 DOI: 10.3758/s13428-023-02292-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/07/2023] [Indexed: 08/21/2024]
Abstract
Pupil size change is a widely adopted, sensitive indicator for sensory and cognitive processes. However, the interpretation of these changes is complicated by the influence of multiple low-level effects, such as brightness or contrast changes, posing challenges to applying pupillometry outside of extremely controlled settings. Building on and extending previous models, we here introduce Open Dynamic Pupil Size Modeling (Open-DPSM), an open-source toolkit to model pupil size changes to dynamically changing visual inputs using a convolution approach. Open-DPSM incorporates three key steps: (1) Modeling pupillary responses to both luminance and contrast changes; (2) Weighing of the distinct contributions of visual events across the visual field on pupil size change; and (3) Incorporating gaze-contingent visual event extraction and modeling. These steps improve the prediction of pupil size changes beyond the here-evaluated benchmarks. Open-DPSM provides Python functions, as well as a graphical user interface (GUI), enabling the extension of its applications to versatile scenarios and adaptations to individualized needs. By obtaining a predicted pupil trace using video and eye-tracking data, users can mitigate the effects of low-level features by subtracting the predicted trace or assess the efficacy of the low-level feature manipulations a priori by comparing estimated traces across conditions.
Collapse
Affiliation(s)
- Yuqing Cai
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands.
| | - Christoph Strauch
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands
| | - Stefan Van der Stigchel
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands
| | - Marnix Naber
- Experimental Psychology, Helmholtz Institute, Faculty of Social Sciences, Utrecht University, Heidelberglaan 1, 3584, CS, Utrecht, The Netherlands
| |
Collapse
|
2
|
Fink L, Simola J, Tavano A, Lange E, Wallot S, Laeng B. From pre-processing to advanced dynamic modeling of pupil data. Behav Res Methods 2024; 56:1376-1412. [PMID: 37351785 PMCID: PMC10991010 DOI: 10.3758/s13428-023-02098-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/20/2023] [Indexed: 06/24/2023]
Abstract
The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.
Collapse
Affiliation(s)
- Lauren Fink
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany.
- Department of Psychology, Neuroscience & Behavior, McMaster University, 1280 Main St. West, Hamilton, Ontario, L8S 4L8, Canada.
| | - Jaana Simola
- Helsinki Collegium for Advanced Studies, University of Helsinki, Helsinki, Finland
- Department of Education, University of Helsinki, Helsinki, Finland
| | - Alessandro Tavano
- Department of Cognitive Neuropsychology, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Elke Lange
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany
| | - Sebastian Wallot
- Department of Literature, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- Institute for Sustainability Education and Psychologyy, Leuphana University, Lüneburg, Germany
| | - Bruno Laeng
- Department of Psychology, University of Oslo, Oslo, Norway
- RITMO Centre for Interdisciplinary studies in Rhythm, Time, and Motion, University of Oslo, Oslo, Norway
| |
Collapse
|
3
|
Wilding M, Körner C, Ischebeck A, Zaretskaya N. Increased insula activity precedes the formation of subjective illusory Gestalt. Neuroimage 2022; 257:119289. [PMID: 35537599 DOI: 10.1016/j.neuroimage.2022.119289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 03/26/2022] [Accepted: 05/06/2022] [Indexed: 11/29/2022] Open
Abstract
The constructive nature of human perception sometimes leads us to perceiving rather complex impressions from simple sensory input: for example, recognizing animal contours in cloud formations or seeing living creatures in shadows of objects. A special type of bistable stimuli gives us a rare opportunity to study the neural mechanisms behind this process. Such stimuli can be visually interpreted either as simple or as more complex illusory content on the basis of the same sensory input. Previous studies demonstrated increased activity in the superior parietal cortex during the perception of an illusory Gestalt impression compared to a simpler interpretation. Here, we examined the role of slow fluctuations of resting-state fMRI activity in shaping the subsequent illusory interpretation by investigating activity related to the illusory Gestalt not only during, but also prior to its perception. We presented 31 participants with a bistable motion stimulus, which can be perceived either as four moving dot pairs (local) or two moving illusory squares (global). fMRI was used to measure brain activity in a slow event-related design. We observed stronger IPS and putamen responses to the stimulus when participants perceived the global interpretation compared to the local, confirming the findings of previous studies. Most importantly, we also observed that the global stimulus interpretation was preceded by an increased activity of the bilateral dorsal insula, which is known to process saliency and gate information for conscious access. Our data suggest an important role of the dorsal insula in shaping complex illusory interpretations of the sensory input.
Collapse
Affiliation(s)
- Marilena Wilding
- Institute of Psychology, University of Graz, Universitätsplatz 2, 8010 Graz, Austria; BioTechMed-Graz, Mozartgasse 12, 8010 Graz, Austria.
| | - Christof Körner
- Institute of Psychology, University of Graz, Universitätsplatz 2, 8010 Graz, Austria; BioTechMed-Graz, Mozartgasse 12, 8010 Graz, Austria
| | - Anja Ischebeck
- Institute of Psychology, University of Graz, Universitätsplatz 2, 8010 Graz, Austria; BioTechMed-Graz, Mozartgasse 12, 8010 Graz, Austria
| | - Natalia Zaretskaya
- Institute of Psychology, University of Graz, Universitätsplatz 2, 8010 Graz, Austria; BioTechMed-Graz, Mozartgasse 12, 8010 Graz, Austria.
| |
Collapse
|
4
|
Strauch C, Wang CA, Einhäuser W, Van der Stigchel S, Naber M. Pupillometry as an integrated readout of distinct attentional networks. Trends Neurosci 2022; 45:635-647. [PMID: 35662511 DOI: 10.1016/j.tins.2022.05.003] [Citation(s) in RCA: 60] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 04/15/2022] [Accepted: 05/09/2022] [Indexed: 10/18/2022]
Abstract
The course of pupillary constriction and dilation provides an easy-to-access, inexpensive, and noninvasive readout of brain activity. We propose a new taxonomy of factors affecting the pupil and link these to associated neural underpinnings in an ascending hierarchy. In addition to two well-established low-level factors (light level and focal distance), we suggest two further intermediate-level factors, alerting and orienting, and a higher-level factor, executive functioning. Alerting, orienting, and executive functioning - including their respective underlying neural circuitries - overlap with the three principal attentional networks, making pupil size an integrated readout of distinct states of attention. As a now widespread technique, pupillometry is ready to provide meaningful applications and constitutes a viable part of the psychophysiological toolbox.
Collapse
Affiliation(s)
- Christoph Strauch
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands.
| | - Chin-An Wang
- Institute of Cognitive Neuroscience, National Central University, Taoyuan City, Taiwan; Cognitive Intelligence and Precision Healthcare Center, National Central University, Taoyuan City, Taiwan
| | - Wolfgang Einhäuser
- Physics of Cognition Group, Chemnitz University of Technology, Chemnitz, Germany
| | | | - Marnix Naber
- Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
5
|
Steinhauer SR, Bradley MM, Siegle GJ, Roecklein KA, Dix A. Publication guidelines and recommendations for pupillary measurement in psychophysiological studies. Psychophysiology 2022; 59:e14035. [PMID: 35318693 PMCID: PMC9272460 DOI: 10.1111/psyp.14035] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Accepted: 02/11/2022] [Indexed: 11/27/2022]
Abstract
A variety of psychological and physical phenomena elicit variations in the diameter of pupil of the eye. Changes in pupil size are mediated by the relative activation of the sphincter pupillae muscle (decrease pupil diameter) and the dilator pupillae muscle (increase pupil diameter), innervated by the parasympathetic and sympathetic branches, respectively, of the autonomic nervous system. The current guidelines are intended to inform and guide psychophysiological research involving pupil measurement by (1) summarizing important aspects concerning the physiology of the pupil, (2) providing methodological and data-analytic guidelines and recommendations, and (3) briefly reviewing psychological phenomena that modulate pupillary reactivity. Because of the increased ease and tractability of pupil measurement, the goal of these guidelines is to promote accurate recording, analysis, and reporting of pupillary data in psychophysiological research.
Collapse
Affiliation(s)
- Stuart R. Steinhauer
- Veterans Affairs Pittsburgh Healthcare System, VISN 4 MIRECC, Pittsburgh, PA, USA
- Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | | | - Greg J. Siegle
- Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
- Department of Psychology, University of Pittsburgh, Pittsburgh, PA, USA
| | | | - Annika Dix
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany
- Centre for Tactile Internet with Human-in-the-Loop (CeTI), Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
6
|
Kuraoka K, Nakamura K. Facial temperature and pupil size as indicators of internal state in primates. Neurosci Res 2022; 175:25-37. [PMID: 35026345 DOI: 10.1016/j.neures.2022.01.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Accepted: 01/07/2022] [Indexed: 11/15/2022]
Abstract
Studies in human subjects have revealed that autonomic responses provide objective and biologically relevant information about cognitive and affective states. Measures of autonomic responses can also be applied to studies of non-human primates, which are neuro-anatomically and physically similar to humans. Facial temperature and pupil size are measured remotely and can be applied to physiological experiments in primates, preferably in a head-fixed condition. However, detailed guidelines for the use of these measures in non-human primates is lacking. Here, we review the neuronal circuits and methodological considerations necessary for measuring and analyzing facial temperature and pupil size in non-human primates. Previous studies have shown that the modulation of these measures primarily reflects sympathetic reactions to cognitive and emotional processes, including alertness, attention, and mental effort, over different time scales. Integrated analyses of autonomic, behavioral, and neurophysiological data in primates are promising methods that reflect multiple dimensions of emotion and could potentially provide tools for understanding the mechanisms underlying neuropsychiatric disorders and vulnerabilities characterized by cognitive and affective disturbances.
Collapse
Affiliation(s)
- Koji Kuraoka
- Department of Physiology, Kansai Medical University, Hirakata, Osaka 573-1010, Japan
| | - Kae Nakamura
- Department of Physiology, Kansai Medical University, Hirakata, Osaka 573-1010, Japan.
| |
Collapse
|
7
|
Marumo C, Nakano T. Early phase of pupil dilation is mediated by the peripheral parasympathetic pathway. J Neurophysiol 2021; 126:2130-2137. [PMID: 34851753 PMCID: PMC8715046 DOI: 10.1152/jn.00401.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022] Open
Abstract
Pupil diameter fluctuates in association with changes in brain states induced by the neuromodulator systems. However, it remains unclear how the neuromodulator systems control the activity of the iris sphincter (constrictor) and dilator muscles to change the pupil size. The present study compared temporal patterns of pupil dilation during movement when each muscle was pharmacologically manipulated in the human eye. When the iris sphincter muscle was blocked with tropicamide, the latency of pupil dilation was delayed and the magnitude of pupil dilation was reduced during movement. In contrast, when the iris dilator muscle was continuously stimulated with phenylephrine, the latency and magnitude of rapid pupil dilation did not differ from the untreated control eye, but sustained pupil dilation was reduced until the end of movement. These results suggest that the iris sphincter muscle, which is under the control of the parasympathetic pathway, is quickly modulated by the neuromodulator system and plays a major role in rapid pupil dilation. However, the iris dilator muscle receives signals from the neuromodulator system with a slow latency and is involved in maintaining sustained pupil dilation. NEW & NOTEWORTHY By pharmacologically manipulating the pupil dilator and constrictor muscles of human eye separately, we found that the pupil constrictor muscle is a primary controller of rapid pupil dilation upon brain arousal. However, the pupil dilator muscle, which is innervated by the sympathetic nervous system and is generally considered as a major regulator of pupil dilation, is not involved in rapid pupil dilation, but was involved in long-lasting pupil dilation.
Collapse
Affiliation(s)
| | - Tamami Nakano
- Faculty of Medicine, Osaka University, Osaka, Japan.,Graduate School of Frontiers Bioscience, Osaka University, Osaka, Japan
| |
Collapse
|