A circuit for pupil orienting responses: implications for cognitive modulation of pupil size

Eye tracking identifies biomarkers in α-synucleinopathies versus progressive supranuclear palsy

Objectives

This study (1) describes and compares saccade and pupil abnormalities in patients with manifest alpha-synucleinopathies (αSYN: Parkinson’s disease (PD), Multiple System Atrophy (MSA)) and a tauopathy (progressive supranuclear palsy (PSP)); (2) determines whether patients with rapid-eye-movement sleep behaviour disorder (RBD), a prodromal stage of αSYN, already have abnormal responses that may indicate a risk for developing PD or MSA.

Methods

Ninety (46 RBD, 27 PD, 17 MSA) patients with an αSYN, 10 PSP patients, and 132 healthy age-matched controls (CTRL) were examined with a 10-min video-based eye-tracking task (Free Viewing). Participants were free to look anywhere on the screen while saccade and pupil behaviours were measured.

Results

PD, MSA, and PSP spent more time fixating the centre of the screen than CTRL. All patient groups made fewer macro-saccades (> 2^◦ amplitude) with smaller amplitude than CTRL. Saccade frequency was greater in RBD than in other patients. Following clip change, saccades were temporarily suppressed, then rebounded at a slower pace than CTRL in all patient groups. RBD had distinct, although discrete saccade abnormalities that were more marked in PD, MSA, and even more in PSP. The vertical saccade rate was reduced in all patients and decreased most in PSP. Clip changes produced large increases or decreases in screen luminance requiring pupil constriction or dilation, respectively. PSP elicited smaller pupil constriction/dilation responses than CTRL, while MSA elicited the opposite.

Conclusion

RBD patients already have discrete but less pronounced saccade abnormalities than PD and MSA patients. Vertical gaze palsy and altered pupil control differentiate PSP from αSYN.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00415-022-11136-5.

Dopamine influences attentional rate modulation in Macaque posterior parietal cortex

Cognitive neuroscience has made great strides in understanding the neural substrates of attention, but our understanding of its neuropharmacology remains incomplete. Although dopamine has historically been studied in relation to frontal functioning, emerging evidence suggests important dopaminergic influences in parietal cortex. We recorded single- and multi-unit activity whilst iontophoretically administering dopaminergic agonists and antagonists while rhesus macaques performed a spatial attention task. Out of 88 units, 50 revealed activity modulation by drug administration. Dopamine inhibited firing rates according to an inverted-U shaped dose-response curve and increased gain variability. D1 receptor antagonists diminished firing rates according to a monotonic function and interacted with attention modulating gain variability. Finally, both drugs decreased the pupil light reflex. These data show that dopamine shapes neuronal responses and modulates aspects of attentional processing in parietal cortex.

A unified model of the task-evoked pupil response

The pupil dilates and reconstricts following task events. It is popular to model this task-evoked pupil response as a linear transformation of event-locked impulses, whose amplitudes are used as estimates of arousal. We show that this model is incorrect and propose an alternative model based on the physiological finding that a common neural input drives saccades and pupil size. The estimates of arousal from our model agreed with key predictions: Arousal scaled with task difficulty and behavioral performance but was invariant to small differences in trial duration. Moreover, the model offers a unified explanation for a wide range of phenomena: entrainment of pupil size and saccades to task timing, modulation of pupil response amplitude and noise with task difficulty, reaction time-dependent modulation of pupil response timing and amplitude, a constrictory pupil response time-locked to saccades, and task-dependent distortion of this saccade-locked pupil response.

Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain-body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.

The oculomotor signature of expected surprise

Expected surprise, defined as the anticipation of uncertainty associated with the occurrence of a future event, plays a major role in gaze shifting and spatial attention. In the present study, we analyzed its impact on oculomotor behavior. We hypothesized that the occurrence of anticipatory saccades could decrease with increasing expected surprise and that its influence on visually-guided responses could be different given the presence of sensory information and perhaps competitive attentional effects. This hypothesis was tested in humans using a saccadic reaction time task in which a cue indicated the future stimulus position. In the ‘no expected surprise’ condition, the visual target could appear only at one previously cued location. In other conditions, more likely future positions were cued with increasing expected surprise. Anticipation was more frequent and pupil size was larger in the ‘no expected surprise’ condition compared with all other conditions, probably due to increased arousal. The latency of visually-guided saccades increased linearly with the logarithm of surprise (following Hick’s law) but their maximum velocity repeated the arousal-related pattern. Therefore, expected surprise affects anticipatory and visually-guided responses differently. Moreover, these observations suggest a causal chain linking surprise, attention and saccades that could be disrupted in attentional or impulse control disorders.

Arousal-based pupil modulation is dictated by luminance

Pupillometry has become a standard measure for assessing arousal state. However, environmental factors such as luminance, a primary dictator of pupillary responses, often vary across studies. To what degree does luminance interact with arousal-driven pupillary changes? Here, we parametrically assessed luminance-driven pupillary responses across a wide-range of luminances, while concurrently manipulating cognitive arousal using auditory math problems of varying difficulty. At the group-level, our results revealed that the modulatory effect of cognitive arousal on pupil size interacts multiplicatively with luminance, with the largest effects occurring at low and mid-luminances. However, at the level of individuals, there were qualitatively distinct individual differences in the modulatory effect of cognitive arousal on luminance-driven pupillary responses. Our findings suggest that pupillometry as a measure for assessing arousal requires more careful consideration: there are ranges of luminance levels that are more ideal in observing pupillary differences between arousal conditions than others.

Influence of the Location of a Decision Cue on the Dynamics of Pupillary Light Response

The pupils of the eyes reflexively constrict in light and dilate in dark to optimize retinal illumination. Non-visual cognitive factors, like attention, arousal, decision-making, etc., also influence pupillary light response (PLR). During passive viewing, the eccentricity of a stimulus modulates the pupillary aperture size driven by spatially weighted corneal flux density (CFD), which is the product of luminance and the area of the stimulus. Whether the scope of attention also influences PLR remains unclear. In this study, we contrasted the pupil dynamics between diffused and focused attentional conditions during decision-making, while the global CFD remained the same in the two conditions. A population of 20 healthy humans participated in a pair of forced choice tasks. They distributed attention to the peripheral decision cue in one task, and concentrated at the center in the other to select the target from four alternatives for gaze orientation. The location of this cue did not influence participants' reaction time (RT). However, the magnitude of constriction was significantly less in the task that warranted attention to be deployed at the center than on the periphery. We observed similar pupil dynamics when participants either elicited or canceled a saccadic eye movement, which ruled out pre-saccadic obligatory attentional orientation contributing to PLR. We further addressed how the location of attentional deployment might have influenced PLR. We simulated a biomechanical model of PLR with visual stimulation of different strengths as inputs corresponding to the two attentional conditions. In this homeomorphic model, the computational characteristic of each element was derived from the physiological and/or mechanical properties of the corresponding biological element. The simulation of this model successfully mimicked the observed data. In contrast to common belief that the global ambient luminosity drives pupillary response, the results of our study suggest that the effective CFD (eCFD) determined via the luminance multiplied by the size of the stimulus at the location of deployed attention in the visual space is critical for the magnitude of pupillary constriction.

Facial temperature and pupil size as indicators of internal state in primates

Studies in human subjects have revealed that autonomic responses provide objective and biologically relevant information about cognitive and affective states. Measures of autonomic responses can also be applied to studies of non-human primates, which are neuro-anatomically and physically similar to humans. Facial temperature and pupil size are measured remotely and can be applied to physiological experiments in primates, preferably in a head-fixed condition. However, detailed guidelines for the use of these measures in non-human primates is lacking. Here, we review the neuronal circuits and methodological considerations necessary for measuring and analyzing facial temperature and pupil size in non-human primates. Previous studies have shown that the modulation of these measures primarily reflects sympathetic reactions to cognitive and emotional processes, including alertness, attention, and mental effort, over different time scales. Integrated analyses of autonomic, behavioral, and neurophysiological data in primates are promising methods that reflect multiple dimensions of emotion and could potentially provide tools for understanding the mechanisms underlying neuropsychiatric disorders and vulnerabilities characterized by cognitive and affective disturbances.

Associations between the pupil light reflex and the broader autism phenotype in children and adults

The pupil light reflex (PLR), a marker of neuronal response to light, is a well-studied index of autonomic functioning. Studies have found that autistic children and adults have slower and weaker PLR responses compared to non-autistic peers, suggesting lower autonomic control. Altered autonomic control has also been associated with increased sensory difficulties in autistic children. With autistic traits varying in the general population, recent studies have begun to examine similar questions in non-autistic individuals. The current study looked at the PLR in relation to individual differences in autistic traits in non-autistic children and adults, asking how differences in the PLR could lead to variation in autistic traits, and how this might change across development. Children and adults completed a PLR task as a measure of sensitivity to light and autonomic response. Results showed that, in adults, increased levels of restricted and repetitive behaviors (RRB) were associated with a weaker and slower PLR. However, in children, PLR responses were not associated with autistic traits. Differences in PLR were also found across age groups, with adults showing smaller baseline pupil diameter and stronger PLR constriction as compared with children. The current study expanded on past work to examine the PLR and autistic traits in non-autistic children and adults, and the relevance of these findings to sensory processing difficulties is discussed. Future studies should continue to examine the neural pathways that might underlie the links between sensory processing and challenging behaviors.

Eye pupil – a window into central autonomic regulation via emotional/cognitive processing

If the eyes are windows into the soul, then the pupils represent at least the gateway to the brain and can provide a unique insight into the human mind from several aspects. The changes in the pupil size primarily mediated by different lighting conditions are controlled by the autonomic nervous system regulated predominantly at the subcortical level. Specifically, parasympathetically-linked pupillary constriction is under the Edinger-Westphal nucleus control and sympathetically-mediated pupillary dilation is regulated from the posterior hypothalamic nuclei. However, the changes in the pupil size can be observed at resting state even under constant lighting, these pupillary changes are mediated by global arousal level as well as by various cognitive factors. In this context, autonomic pathways modulating changes in the pupil size in response to the different light levels can be influenced by multiple central descending inputs driving pupillary changes under steady lighting conditions. Moreover, as the pupillary response is involved in emotional (task-evoked pupillary dilation as an index of emotional arousal) and cognitive (task-evoked pupillary dilation as an index of cognitive workload) stimulation, it can be used to detect the impact of mutual subcortical and cortical structures (i.e. overlapping brain structures included in autonomic, emotional and cognitive regulation) on the pupillary innervation system. Thus, complex understanding of the baseline pupil size´ and pupillary dynamics´ mechanisms may provide an important insight into the central nervous system functioning pointing to the pupillometry as a promising tool in the clinical application.

Eye pupil - a window into central autonomic regulation via emotional/cognitive processing

If the eyes are windows into the soul, then the pupils represent at least the gateway to the brain and can provide a unique insight into the human mind from several aspects. The changes in the pupil size primarily mediated by different lighting conditions are controlled by the autonomic nervous system regulated predominantly at the subcortical level. Specifically, parasympathetically-linked pupillary constriction is under the Edinger-Westphal nucleus control and sympathetically-mediated pupillary dilation is regulated from the posterior hypothalamic nuclei. However, the changes in the pupil size can be observed at resting state even under constant lighting, these pupillary changes are mediated by global arousal level as well as by various cognitive factors. In this context, autonomic pathways modulating changes in the pupil size in response to the different light levels can be influenced by multiple central descending inputs driving pupillary changes under steady lighting conditions. Moreover, as the pupillary response is involved in emotional (task-evoked pupillary dilation as an index of emotional arousal) and cognitive (task-evoked pupillary dilation as an index of cognitive workload) stimulation, it can be used to detect the impact of mutual subcortical and cortical structures (i.e. overlapping brain structures included in autonomic, emotional and cognitive regulation) on the pupillary innervation system. Thus, complex understanding of the baseline pupil size´ and pupillary dynamics´ mechanisms may provide an important insight into the central nervous system functioning pointing to the pupillometry as a promising tool in the clinical application.

Is Pupil Activity Associated With the Strength of Memory Signal for Words in a Continuous Recognition Memory Paradigm?

Research on pupillometry provides an increasing evidence for associations between pupil activity and memory processing. The most consistent finding is related to an increase in pupil size for old items compared with novel items, suggesting that pupil activity is associated with the strength of memory signal. However, the time course of these changes is not completely known, specifically, when items are presented in a running recognition task maximizing interference by requiring the recognition of the most recent items from a sequence of old/new items. The sample comprised 42 healthy participants who performed a visual word recognition task under varying conditions of retention interval. Recognition responses were evaluated using behavioral variables for discrimination accuracy, reaction time, and confidence in recognition decisions. Pupil activity was recorded continuously during the entire experiment. The results suggest a decrease in recognition performance with increasing study-test retention interval. Pupil size decreased across retention intervals, while pupil old/new effects were found only for words recognized at the shortest retention interval. Pupillary responses consisted of a pronounced early pupil constriction at retrieval under longer study-test lags corresponding to weaker memory signals. However, the pupil size was also sensitive to the subjective feeling of familiarity as shown by pupil dilation to false alarms (new items judged as old). These results suggest that the pupil size is related not only to the strength of memory signal but also to subjective familiarity decisions in a continuous recognition memory paradigm.

PuPl: an open-source tool for processing pupillometry data

We present PuPl (Pu pillometry P ipel iner), an Octave-compatible library of Matlab functions for processing pupillometry data with an easy-to-use graphical user interface (GUI). PuPl's preprocessing tools include blink correction, data smoothing, and gaze correction. PuPl can also define and sort trials, and segment data to isolate event-related pupil dilation responses. PuPl's flexible tabular export tools enable a wide variety of statistical analyses. Furthermore, PuPl can translate GUI interactions into a Matlab script, enabling easy creation and reconfiguration of reusable data processing pipelines. Finally, PuPl is designed to be extensible, and users can easily contribute functionality as best practices for pupillometry evolve. Here we demonstrate PuPl by replicating published results using publicly available data. PuPl can be downloaded from github.com/kinleyid/pupl .

Early phase of pupil dilation is mediated by the peripheral parasympathetic pathway

Pupil diameter fluctuates in association with changes in brain states induced by the neuromodulator systems. However, it remains unclear how the neuromodulator systems control the activity of the iris sphincter (constrictor) and dilator muscles to change the pupil size. The present study compared temporal patterns of pupil dilation during movement when each muscle was pharmacologically manipulated in the human eye. When the iris sphincter muscle was blocked with tropicamide, the latency of pupil dilation was delayed and the magnitude of pupil dilation was reduced during movement. In contrast, when the iris dilator muscle was continuously stimulated with phenylephrine, the latency and magnitude of rapid pupil dilation did not differ from the untreated control eye, but sustained pupil dilation was reduced until the end of movement. These results suggest that the iris sphincter muscle, which is under the control of the parasympathetic pathway, is quickly modulated by the neuromodulator system and plays a major role in rapid pupil dilation. However, the iris dilator muscle receives signals from the neuromodulator system with a slow latency and is involved in maintaining sustained pupil dilation.

NEW & NOTEWORTHY By pharmacologically manipulating the pupil dilator and constrictor muscles of human eye separately, we found that the pupil constrictor muscle is a primary controller of rapid pupil dilation upon brain arousal. However, the pupil dilator muscle, which is innervated by the sympathetic nervous system and is generally considered as a major regulator of pupil dilation, is not involved in rapid pupil dilation, but was involved in long-lasting pupil dilation.

Visual Strategies Underpinning the Spatiotemporal Demands During Visuomotor Tasks in Predicting Ball Direction

We investigated gaze behavior of expert goalkeepers during the prediction of penalty kicks in different spatiotemporal constraints: penalties taken from 11 and 6 m. From 11 m, goalkeepers were more successful in predicting ball direction, with longer movement time initiation and a visual strategy with more fixations and greater saccade rates than penalties from 6 m, where they exhibited fewer fixations with higher microsaccade rates. As long as the opponent's distance is large and time pressure low, gaze can be frequently shifted between the kicker's body and the ball, due to the low cost of saccades. Conversely, when the objects are close, there is increased reliance on foveal and parafoveal information. In conclusion, when the spatiotemporal constraint is less severe, goalkeepers adopt a visual strategy with more fixations and small saccades. When the spatiotemporal constraint is more severe, they rely on peripheral vision to monitor kickers' movements through the use of microsaccades.

Linking Pupil Size Modulated by Global Luminance and Motor Preparation to Saccade Behavior

Saccades are rapid eye movements that are used to move the high acuity fovea in a serial manner in the exploration of the visual scene. Stimulus contrast is known to modulate saccade latency and metrics possibly via changing visual activity in the superior colliculus (SC), a midbrain structure causally involved in saccade generation. However, the quality of visual signals should also be modulated by the amount of lights projected onto the retina, which is gated by the size of the pupil. Although absolute pupil size should modulate visual signals and in turn affect saccade responses, research examining this relationship is very limited. Besides, pupil size is associated with motor preparation. However, the role of pupil dilation in saccade metrics remains unexplored. Through varying peripheral background luminance level and target visual contrast in the saccade task, we investigated the role of absolute pupil size and baseline-corrected pupil dilation in saccade latency and metrics. Higher target detection accuracy was obtained with lower background luminance level, and larger absolute pupil diameter correlated with smaller saccade amplitude and higher saccade peak velocities. More interestingly, the comparable modulation between pupil dilation and stimulus contrast was obtained, showing larger pupil dilation (or higher contrast stimuli) correlating with faster saccade latencies, larger amplitude, higher peak velocities, and smaller endpoint deviation. Together, our results demonstrated the influence of absolute pupil size induced by global luminance level and baseline-corrected pupil dilation associated with motor preparation on saccade latency and metrics, implicating the role of the SC in this behavior.

Differentiating global luminance, arousal and cognitive signals on pupil size and microsaccades

Pupil size reflects a proxy for neural activity associated with global luminance, arousal and cognitive processing. Microsaccades are also modulated by arousal and cognitive processing. Are these effects of arousal and cognitive signals on pupil size and microsaccades coordinated? If so, via what neural mechanisms? We hypothesized that if pupil size and microsaccades are coordinately modulated by these processes, pupil size immediately before microsaccade onset, as an index for ongoing cognitive and arousal processing, should correlate with microsaccade responses during tasks alternating these signals. Here, we examined the relationship between pupil size and microsaccade responses in tasks that included variations in global luminance, arousal and inhibitory control. Higher microsaccade peak velocities correlated with larger pre-microsaccade pupil response related to arousal and inhibitory control signals. In contrast, pupil responses evoked by global luminance signals did not correlate with microsaccade responses. Given the central role of the superior colliculus in microsaccade generation, these results suggest the critical involvement of the superior colliculus to coordinate pupil and microsaccade responses for arousal and inhibitory control modulations, but not for the pupil luminance modulation.

The acute exercise-cognition interaction: From the catecholamines hypothesis to an interoception model

An interoception model for the acute exercise-cognition interaction is presented. During exercise following the norepinephrine threshold, interoceptive feedback induces increased tonic release of extracellular catecholamines, facilitating phasic release hence better cognitive performance of executive functions. When exercise intensity increases to maximum, the nature of task-induced norepinephrine release from the locus coeruleus is dependent on interaction between motivation, perceived effort costs and perceived availability of resources. This is controlled by interaction between the rostral and dorsolateral prefrontal cortices, orbitofrontal cortex, anterior cingulate cortex and anterior insula cortex. If perceived available resources are sufficient to meet predicted effort costs and reward value is high, tonic release from the locus coeruleus is attenuated thus facilitating phasic release, therefore cognition is not inhibited. However, if perceived available resources are insufficient to meet predicted effort costs or reward value is low, tonic release from the locus coeruleus is induced, attenuating phasic release. As a result, cognition is inhibited, although long-term memory and tasks that require switching to new stimuli-response couplings are probably facilitated.

MEYE: Web App for Translational and Real-Time Pupillometry

Pupil dynamics alterations have been found in patients affected by a variety of neuropsychiatric conditions, including autism. Studies in mouse models have used pupillometry for phenotypic assessment and as a proxy for arousal. Both in mice and humans, pupillometry is noninvasive and allows for longitudinal experiments supporting temporal specificity; however, its measure requires dedicated setups. Here, we introduce a convolutional neural network that performs online pupillometry in both mice and humans in a web app format. This solution dramatically simplifies the usage of the tool for the nonspecialist and nontechnical operators. Because a modern web browser is the only software requirement, this choice is of great interest given its easy deployment and setup time reduction. The tested model performances indicate that the tool is sensitive enough to detect both locomotor-induced and stimulus-evoked pupillary changes, and its output is comparable to state-of-the-art commercial devices.

Pupil dynamics: A potential proxy of neural preparation for goal-directed eye movement

The pupils reflexively constrict or dilate to regulate the influx of light on the retinae. Pupillary light reflex (PLR) is susceptible to many non-visual cognitive processes including covert orientation of attention and planning rapid saccadic eye movement. The frontal eye field (FEF) and superior colliculus (SC), which also send projections to the PLR pathway, are two important areas in primate's brain for planning saccade and orientation of attention. The saccadic reaction time (SRT) and the rate of increase in activity of movement neurons in these areas are inversely correlated. This study addressed how pupil dynamics, activity in the FEF and SC and SRT are related in a saccadic decision-making task. The rate of visually evoked pupil constriction was found inversely related to SRT. This was further verified by simulating a homeomorphic biomechanical model of pupillary muscle plants, wherein we projected signals similar to build-up activity in the FEF and SC to the parasympathetic (constriction) and sympathetic (dilation) division of the PLR pathway, respectively. A striking similarity between simulated and observed dynamics of pupil constriction suggests that PLR is a potential proxy of saccade planning by movement neurons in the FEF and SC. Indistinguishable pupil dynamics when planned saccades were elicited versus when they were cancelled eliminated the possibility that the obligatory pre-saccadic shift of attention alone influenced the rate of pupil constriction. Our study envisages a mechanism of how the oculomotor system influences the autonomic activity in an attempt to timely minimize saccadic visual transients by regulating the influx of light.

Separable pupillary signatures of perception and action during perceptual multistability

The pupil provides a rich, non-invasive measure of the neural bases of perception and cognition and has been of particular value in uncovering the role of arousal-linked neuromodulation, which alters both cortical processing and pupil size. But pupil size is subject to a multitude of influences, which complicates unique interpretation. We measured pupils of observers experiencing perceptual multistability-an ever-changing subjective percept in the face of unchanging but inconclusive sensory input. In separate conditions, the endogenously generated perceptual changes were either task-relevant or not, allowing a separation between perception-related and task-related pupil signals. Perceptual changes were marked by a complex pupil response that could be decomposed into two components: a dilation tied to task execution and plausibly indicative of an arousal-linked noradrenaline surge, and an overlapping constriction tied to the perceptual transient and plausibly a marker of altered visual cortical representation. Constriction, but not dilation, amplitude systematically depended on the time interval between perceptual changes, possibly providing an overt index of neural adaptation. These results show that the pupil provides a simultaneous reading on interacting but dissociable neural processes during perceptual multistability, and suggest that arousal-linked neuromodulator release shapes action but not perception in these circumstances.

Decoding expectation and surprise in dementia: the paradigm of music

Making predictions about the world and responding appropriately to unexpected events are essential functions of the healthy brain. In neurodegenerative disorders, such as frontotemporal dementia and Alzheimer's disease, impaired processing of 'surprise' may underpin a diverse array of symptoms, particularly abnormalities of social and emotional behaviour, but is challenging to characterize. Here, we addressed this issue using a novel paradigm: music. We studied 62 patients (24 female; aged 53-88) representing major syndromes of frontotemporal dementia (behavioural variant, semantic variant primary progressive aphasia, non-fluent-agrammatic variant primary progressive aphasia) and typical amnestic Alzheimer's disease, in relation to 33 healthy controls (18 female; aged 54-78). Participants heard famous melodies containing no deviants or one of three types of deviant note-acoustic (white-noise burst), syntactic (key-violating pitch change) or semantic (key-preserving pitch change). Using a regression model that took elementary perceptual, executive and musical competence into account, we assessed accuracy detecting melodic deviants and simultaneously recorded pupillary responses and related these to deviant surprise value (information-content) and carrier melody predictability (entropy), calculated using an unsupervised machine learning model of music. Neuroanatomical associations of deviant detection accuracy and coupling of detection to deviant surprise value were assessed using voxel-based morphometry of patients' brain MRI. Whereas Alzheimer's disease was associated with normal deviant detection accuracy, behavioural and semantic variant frontotemporal dementia syndromes were associated with strikingly similar profiles of impaired syntactic and semantic deviant detection accuracy and impaired behavioural and autonomic sensitivity to deviant information-content (all P < 0.05). On the other hand, non-fluent-agrammatic primary progressive aphasia was associated with generalized impairment of deviant discriminability (P < 0.05) due to excessive false-alarms, despite retained behavioural and autonomic sensitivity to deviant information-content and melody predictability. Across the patient cohort, grey matter correlates of acoustic deviant detection accuracy were identified in precuneus, mid and mesial temporal regions; correlates of syntactic deviant detection accuracy and information-content processing, in inferior frontal and anterior temporal cortices, putamen and nucleus accumbens; and a common correlate of musical salience coding in supplementary motor area (all P < 0.05, corrected for multiple comparisons in pre-specified regions of interest). Our findings suggest that major dementias have distinct profiles of sensory 'surprise' processing, as instantiated in music. Music may be a useful and informative paradigm for probing the predictive decoding of complex sensory environments in neurodegenerative proteinopathies, with implications for understanding and measuring the core pathophysiology of these diseases.

Oculomotor Behaviors and Integrative Memory Functions in the Alzheimer's Clinical Syndrome

BACKGROUND

Biological information drawn from eye-tracking metrics is providing evidence regarding drivers of cognitive decline in Alzheimer's disease. In particular, pupil size has proved useful to investigate cognitive performance during online activities.

OBJECTIVE

To investigate the oculomotor correlates of impaired performance of patients with mild Alzheimer's Clinical Syndrome (ACS) on a recently developed memory paradigm, namely the Short-Term Memory Binding Test (STMBT).

METHODS

We assessed a sample of eighteen healthy controls (HC) and eighteen patients with a diagnosis of mild ACS with the STMBT while we recorded their oculomotor behaviors using pupillometry and eye-tracking.

RESULTS

As expected, a group (healthy controls versus ACS) by condition (Unbound Colours versus Bound Colours) interaction was found whereby behavioral group differences were paramount in the Bound Colours condition. Healthy controls' pupils dilated significantly more in the Bound Colours than in the Unbound Colours condition, a discrepancy not observed in ACS patients. Furthermore, ROC analysis revealed the abnormal pupil behaviors distinguished ACS patients from healthy controls with values of sensitivity and specify of 100%, thus outperforming both recognition scores and gaze duration.

CONCLUSION

The biological correlates of Short-Term Memory Binding impairments appear to involve a network much wider than we have thought to date, which expands across cortical and subcortical structures. We discuss these findings focusing on their implications for our understanding of neurocognitive phenotypes in the preclinical stages of Alzheimer's disease and potential development of cognitive biomarkers that can support ongoing initiatives to prevent dementia.

Salient objects dominate the central fixation bias when orienting toward images

Short-latency saccades are often biased toward salient objects or toward the center of images, for example, when inspecting photographs of natural scenes. Here, we measured the contribution of salient objects and central fixation bias to visual selection over time. Participants made saccades to images containing one salient object on a structured background and were instructed to either look at (i) the image center, (ii) the salient object, or (iii) at a cued position halfway in between the two. Results revealed, first, an early involuntary bias toward the image center irrespective of strategic behavior or the location of objects in the image. Second, the salient object bias was stronger than the center bias and prevailed over the latter when they directly competed for visual selection. In a second experiment, we tested whether the center bias depends on how well the image can be segregated from the monitor background. We asked participants to explore images that either did or did not contain a salient object while we manipulated the contrast between image background and monitor background to make the image borders more or less visible. The initial orienting toward the image was not affected by the image-monitor contrast, but only by the presence of objects-with a strong bias toward the center of images containing no object. Yet, a low image-monitor contrast reduced this center bias during the subsequent image exploration.

Visuospatial Attention Allocation as an Indicator of Cognitive Deficit in Traumatic Brain Injury: A Systematic Review and Meta-Analysis

Traumatic Brain Injury (TBI) is defined by changes in brain function resulting from external forces acting on the brain and is typically characterized by a host of physiological and functional changes such as cognitive deficits including attention problems. In the present study, we focused on the effect of TBI on the ability to allocate attention in vision (i.e., the use of endogenous and exogenous visual cues) by systematically reviewing previous literature on the topic. We conducted quantitative synthesis of 16 selected studies of visual attention following TBI, calculating 80 effect size estimates. The combined effect size was large (g = 0.79, p < 0.0001) with medium heterogeneity (I² = 68.39%). Subgroup analyses revealed an increase in deficit with moderate-to-severe and severe TBI as compared to mild TBI [F_{(2, 76)} = 24.14, p < 0.0001]. Task type was another key source of variability and subgroup analyses indicated that higher order attention processes were severely affected by TBI [F_{(2, 77)} = 5.66, p = 0.0051). Meta-regression analyses revealed significant improvement in visual attention deficit with time [p(mild) = 0.031, p(moderate-to-severe) = 0.002, p(severe) < 0.0001]. Taken together, these results demonstrate that visual attention is affected by TBI and that regular assessment of visual attention, using a systematic attention allocation task, may provide a useful clinical measure of cognitive impairment and change after TBI.

Sustained Pupil Responses Are Modulated by Predictability of Auditory Sequences

The brain is highly sensitive to auditory regularities and exploits the predictable order of sounds in many situations, from parsing complex auditory scenes, to the acquisition of language. To understand the impact of stimulus predictability on perception, it is important to determine how the detection of predictable structure influences processing and attention. Here, we use pupillometry to gain insight into the effect of sensory regularity on arousal. Pupillometry is a commonly used measure of salience and processing effort, with more perceptually salient or perceptually demanding stimuli consistently associated with larger pupil diameters. In two experiments we tracked human listeners' pupil dynamics while they listened to sequences of 50-ms tone pips of different frequencies. The order of the tone pips was either random, contained deterministic (fully predictable) regularities (experiment 1, n = 18, 11 female) or had a probabilistic regularity structure (experiment 2, n = 20, 17 female). The sequences were rapid, preventing conscious tracking of sequence structure thus allowing us to focus on the automatic extraction of different types of regularities. We hypothesized that if regularity facilitates processing by reducing processing demands, a smaller pupil diameter would be seen in response to regular relative to random patterns. Conversely, if regularity is associated with heightened arousal and attention (i.e., engages processing resources) the opposite pattern would be expected. In both experiments we observed a smaller sustained (tonic) pupil diameter for regular compared with random sequences, consistent with the former hypothesis and confirming that predictability facilitates sequence processing.SIGNIFICANCE STATEMENT The brain is highly sensitive to auditory regularities. To appreciate the impact that the presence of predictability has on perception, we need to better understand how a predictable structure influences processing and attention. We recorded listeners' pupil responses to sequences of tones that followed either a predictable or unpredictable pattern, as the pupil can be used to implicitly tap into these different cognitive processes. We found that the pupil showed a smaller sustained diameter to predictable sequences, indicating that predictability eased processing rather than boosted attention. The findings suggest that the pupil response can be used to study the automatic extraction of regularities, and that the effects are most consistent with predictability helping the listener to efficiently process upcoming sounds.

Non-Contact Measurement of Motion Sickness Using Pupillary Rhythms from an Infrared Camera

Both physiological and neurological mechanisms are reflected in pupillary rhythms via neural pathways between the brain and pupil nerves. This study aims to interpret the phenomenon of motion sickness such as fatigue, anxiety, nausea and disorientation using these mechanisms and to develop an advanced non-contact measurement method from an infrared webcam. Twenty-four volunteers (12 females) experienced virtual reality content through both two-dimensional and head-mounted device interpretations. An irregular pattern of the pupillary rhythms, demonstrated by an increasing mean and standard deviation of pupil diameter and decreasing pupillary rhythm coherence ratio, was revealed after the participants experienced motion sickness. The motion sickness was induced while watching the head-mounted device as compared to the two-dimensional virtual reality, with the motion sickness strongly related to the visual information processing load. In addition, the proposed method was verified using a new experimental dataset for 23 participants (11 females), with a classification performance of 89.6% (n = 48) and 80.4% (n = 46) for training and test sets using a support vector machine with a radial basis function kernel, respectively. The proposed method was proven to be capable of quantitatively measuring and monitoring motion sickness in real-time in a simple, economical and contactless manner using an infrared camera.

To Explore the Predictive Power of Visuomotor Network Dysfunctions in Mild Cognitive Impairment and Alzheimer's Disease

Background

Research into Alzheimer’s disease has shifted toward the identification of minimally invasive and less time-consuming modalities to define preclinical stages of Alzheimer’s disease.

Method

Here, we propose visuomotor network dysfunctions as a potential biomarker in AD and its prodromal stage, mild cognitive impairment with underlying the Alzheimer’s disease pathology. The functionality of this network was tested in terms of timing, accuracy, and speed with goal-directed eye-hand tasks. The predictive power was determined by comparing the classification performance of a zero-rule algorithm (baseline), a decision tree, a support vector machine, and a neural network using functional parameters to classify controls without cognitive disorders, mild cognitive impaired patients, and Alzheimer’s disease patients.

Results

Fair to good classification was achieved between controls and patients, controls and mild cognitive impaired patients, and between controls and Alzheimer’s disease patients with the support vector machine (77–82% accuracy, 57–93% sensitivity, 63–90% specificity, 0.74–0.78 area under the curve). Classification between mild cognitive impaired patients and Alzheimer’s disease patients was poor, as no algorithm outperformed the baseline (63% accuracy, 0% sensitivity, 100% specificity, 0.50 area under the curve).

Comparison with Existing Method(s)

The classification performance found in the present study is comparable to that of the existing CSF and MRI biomarkers.

Conclusion

The data suggest that visuomotor network dysfunctions have potential in biomarker research and the proposed eye-hand tasks could add to existing tests to form a clear definition of the preclinical phenotype of AD.

Ocular measures during associative learning predict recall accuracy

The ability to form associations between stimuli and commit those associations to memory is a cornerstone of human cognition. Dopamine and noradrenaline are critical neuromodulators implicated in a range of cognitive functions, including learning and memory. Eye blink rate (EBR) and pupil diameter have been shown to index dopaminergic and noradrenergic activity. Here, we examined how these ocular measures relate to accuracy in a paired-associate learning task where participants (N = 73) learned consistent object-location associations over eight trials consisting of pre-trial fixation, encoding, delay, and retrieval epochs. In order to examine how within-subject changes and between-subject changes in ocular metrics related to accuracy, we mean centered individual metric values on each trial based on within-person and across-subject means for each epoch. Within-participant variation in EBR was positively related to accuracy in both encoding and delay epochs: faster EBR within the individual predicted better retrieval. Differences in EBR across participants was negatively related to accuracy in the encoding epoch and in early trials of the pre-trial fixation: faster EBR, relative to other subjects, predicted poorer retrieval. Visual scanning behavior in pre-trial fixation and delay epochs was also positively related to accuracy in early trials: more scanning predicted better retrieval. We found no relationship between pupil diameter and accuracy. These results provide novel evidence supporting the utility of ocular metrics in illuminating cognitive and neurobiological mechanisms of paired-associate learning.

Effect of glare illusion-induced perceptual brightness on temporal perception

Temporal perception and the ability to precisely ascertain time duration are central to essentially all behaviors. Since stimulus magnitude is assumed to be positively related to the perceived duration from the early days of experimental psychology, most studies so far have assessed this effect by presenting stimuli with relatively different intensities in physical quantity. However, it remains unclear how perceptual magnitude itself directly affects temporal perception. In this study (n = 21, n = 20), we conducted a two‐interval duration‐discrimination task adapting a glare illusion (a visual illusion that enhances perceived brightness without changing physical luminance) to investigate whether the temporal perception is also influenced by perceptual magnitude. Based on the mean difference in the point of subjective equality derived from a psychometric function and pupil diameter, we found that temporal perception is influenced by the illusory brightness of glare stimuli. Interestingly, the perceived duration of the apparently brighter stimuli (glare stimuli; larger pupillary light reflex) was shorter than that of control stimuli (halo stimuli; smaller pupillary light reflex) despite the stimuli remaining physically equiluminant, in contrast with the well‐known "magnitude effect." Furthermore, this temporal modulation did not occur when the physical luminance of the stimuli was manipulated to match the illusory‐induced magnitude. These results indicate that temporal processing depends on the confluence of both external and perceived subjective magnitude and even illusory brightness is sufficient to affect the sense of duration; which may be explained by the internal magnitude decrease of the glare stimuli due to pupillary constriction decreasing the light entering the eye.

Our findings suggest a new viewpoint on the positive relationship between temporal perception and stimulus magnitude, in demonstrating that temporal processing depends on the confluence of both external and perceived internal magnitude. We provide evidence that illusory brightness induced by the glare‐illusion also influences the perceived duration which may be explained by the size of the pupil.

Multisensory signals inhibit pupillary light reflex: Evidence from pupil oscillation

Multisensory integration, which enhances stimulus saliency at the early stage of the processing hierarchy, has been recently shown to produce a larger pupil size than its unisensory constituents. Theoretically, any modulation on pupil size ought to be associated with the sympathetic and parasympathetic pathways that are sensitive to light. But it remains poorly understood how the pupillary light reflex is changed in a multisensory context. The present study evoked an oscillation of the pupillary light reflex by periodically changing the luminance of a visual stimulus at 1.25 Hz. It was found that such induced pupil size oscillation was substantially attenuated when the bright but not the dark phase of the visual flicker was periodically and synchronously presented with a burst of tones. This inhibition effect persisted when the visual flicker was task-irrelevant and out of attentional focus, but disappeared when the visual flicker was moved from the central field to the periphery. These findings not only offer a comprehensive characterization of the multisensory impact on pupil response to light, but also provide valuable clues about the individual contributions of the sympathetic and parasympathetic pathways to multisensory modulation of pupil size.

Pupillary response to representations of light in paintings

It is known that, although the level of light is the primary determinant of pupil size, cognitive factors can also affect pupil diameter. It has been demonstrated that photographs of the sun produce pupil constriction independently of their luminance and other low-level features, suggesting that high-level visual processing may also modulate pupil response.

Here, we measure pupil response to artistic paintings of the sun, moon, or containing a uniform lighting, that, being mediated by the artist's interpretation of reality and his technical rendering, require an even higher level of interpretation compared with photographs. We also study how chromatic content and spatial layout affect the results by presenting grey-scale and inverted versions of each painting. Finally, we assess directly with a categorization test how subjective image interpretation affects pupil response.

We find that paintings with the sun elicit a smaller pupil size than paintings with the moon, or paintings containing no visible light source. The effect produced by sun paintings is reduced by disrupting contextual information, such as by removing color or manipulating the relations between paintings features that make more difficult to identify the source of light. Finally, and more importantly, pupil diameter changes according to observers’ interpretation of the scene represented in the same stimulus.

In conclusion, results show that the subcortical pupillary response to light is modulated by subjective interpretation of luminous objects, suggesting the involvement of cortical systems in charge of cognitive processes, such as attention, object recognition, familiarity, memory, and imagination.

Role of the frontal eye field in human pupil and saccade orienting responses

The appearance of a salient stimulus evokes a series of orienting responses including saccades and pupil size to prepare the body for appropriate action. The midbrain superior colliculus (SC) that receives critical control signals from the frontal eye field (FEF) is hypothesized to coordinate all components of orienting. It has shown recently that the FEF, together with the SC, is also importantly involved in the control of pupil size, in addition to its well-documented role in eye movements. Although the role of the FEF in pupil size is demonstrated in monkeys, its role in human pupil responses and the coordination between pupil size and saccades remains to be established. Through applying continuous theta-burst stimulation over the right FEF and vertex, we investigated the role of the FEF in human pupil and saccade responses evoked by a salient stimulus, and the coordination between pupil size and saccades. Our results showed that neither saccade reaction times (SRT) nor pupil responses evoked by salient stimuli were modulated by FEF stimulation. In contrast, the correlation between pupil size and SRTs in the contralateral stimulus condition was diminished with FEF stimulation, but intact with vertex stimulation. Moreover, FEF stimulation effects between saccade and pupil responses associated with salient stimuli correlated across participants. This is the first transcranial magnetic stimulation (TMS) study on the pupil orienting response, and our findings suggest that human FEF was involved in coordinating pupil size and saccades, but not involved in the control of pupil orienting responses.

Saccade, Pupil, and Blink Responses in Rapid Eye Movement Sleep Behavior Disorder

Background

Parkinson's disease (PD) patients exhibit deficits in saccade performance, pupil function, and blink rate. Isolated REM (rapid eye movement) Sleep Behavior Disorder (RBD) is a harbinger to PD making them candidates to investigate for early oculomotor abnormalities as PD biomarkers.

Objectives

We tested whether saccade, pupillary, and blink responses in RBD were similar to PD.

Methods

RBD (n = 22), PD (n = 22) patients, and healthy controls (CTRL) (n = 74) were studied with video‐based eye‐tracking.

Results

RBD patients did not have significantly different saccadic behavior compared to CTRL, but PD patients differed from CTRL and RBD. Both patient groups had significantly lower blink rates, dampened pupil constriction, and dilation responses compared to CTRL.

Conclusion

RBD and PD patients had altered pupil and blink behavior compared to CTRL. Because RBD saccade parameters were comparable to CTRL, pupil and blink brain areas may be impacted before saccadic control areas, making them potential prodromal PD biomarkers. © 2021 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society

Fixation Duration and Pupil Size as Diagnostic Tools in Parkinson's Disease

BACKGROUND

Visual and oculomotor problems are very common in Parkinson's disease (PD) and by using eye-tracking such problems could be characterized in more detail. However, eye-tracking is not part of the routine clinical investigation of parkinsonism.

OBJECTIVE

To evaluate gaze stability and pupil size in stable light conditions, as well as eye movements during sustained fixation in a population of PD patients and healthy controls (HC).

METHODS

In total, 50 PD patients (66% males) with unilateral to mild-to-moderate disease (Hoehn & Yahr 1-3, Schwab and England 70-90%) and 43 HC (37% males) were included in the study. Eye movements were recorded with Tobii Pro Spectrum, a screen-based eye tracker with a sampling rate of 1200 Hz. Logistic regression analysis was applied to investigate the strength of association of eye-movement measures with diagnosis.

RESULTS

Median pupil size (OR 0.811; 95% CI 0.666-0.987; p = 0.037) and longest fixation period (OR 0.798; 95% CI 0.691-0.921; p = 0.002), were the eye-movement parameters that were independently associated with diagnosis, after adjustment for sex (OR 4.35; 95% CI 1.516-12.483; p = 0.006) and visuospatial/executive score in Montreal Cognitive Assessment (OR 0.422; 95% CI 0.233-0.764; p = 0.004). The area under the ROC curve was determined to 0.817; 95% (CI) 0.732-0.901.

CONCLUSION

Eye-tracking based measurements of gaze fixation and pupil reaction may be useful biomarkers of PD diagnosis. However, larger studies of eye-tracking parameters integrated into the screening of patients with suspected PD are necessary, to further investigate and confirm their diagnostic value.

Coordination of Pupil and Saccade Responses by the Superior Colliculus

The appearance of a salient stimulus evokes saccadic eye movements and pupil dilation as part of the orienting response. Although the role of the superior colliculus (SC) in saccade and pupil dilation has been established separately, whether and how these responses are coordinated remains unknown. The SC also receives global luminance signals from the retina, but whether global luminance modulates saccade and pupil responses coordinated by the SC remains unknown. Here, we used microstimulation to causally determine how the SC coordinates saccade and pupil responses and whether global luminance modulates these responses by varying stimulation frequency and global luminance in male monkeys. Stimulation frequency modulated saccade and pupil responses, with trial-by-trial correlations between the two responses. Global luminance only modulated pupil, but not, saccade responses. Our results demonstrate an integrated role of the SC on coordinating saccade and pupil responses, characterizing luminance independent modulation in the SC, together elucidating the differentiated pathways underlying this behavior.

Human and Human-Interfaced AI Interactions: Modulation of Human Male Autonomic Nervous System via Pupil Mimicry

Pupillary alterations in virtual humans induce neurophysiological responses within an observer. Technological advances have enabled rapid developments in artificial intelligence (AI), from verbal systems, to visual AI interfaces with the ability to express, and respond to emotional states of a user. Visual AI interfaces are able to change their physical parameters, such as pupil diameter. Pupillary changes can alter heart rate, however, effects on heart rate variability (HRV) are unknown. HRV, is an autonomic, non-conscious parameter which monitors sympathetic and parasympathetic nervous system (PNS) activity. N = 34 male participants aged between 19–33 were subjected to a number of conditions such as pupil dilation, constriction and blushing. The present research is the first to investigate the effects of virtual human interactions on human HRV. Outcomes of this study were obtained using eye tracking and HRV measurements. Pupil dilation relative to constriction presented in the female virtual partner induced a significant right pupillary diameter increase (p = 0.041) in human observers. Additionally, female virtual partner pupil constriction relative to dilation induced a significant increase in participants’ PNS HRV response (p = 0.036). These findings indicate the ability of a female virtual interaction partner to modulate parasympathetic autonomic functioning in young healthy male humans. This allows first insights into the effects of interacting with virtual AI interaction partners, on human autonomic functioning, and may aid development of future virtual humans, and their implementation into relevant clinical settings.

Pupil response is modulated with optokinetic nystagmus in transparent motion

When two visual patterns moving in opposite directions are superimposed on the same depth plane, they appear to have two transparent surfaces moving independently (transparent motion). Additionally, the direction of the slow phase of optokinetic nystagmus (OKN) corresponds to the direction of motion that dominates the perceptual appearance. This study examines whether pupil changes correspond to the luminance of the dominated objects related to the transition of the slow-phase direction in OKN following objects. Stimuli consisted of two random dot patterns of different luminance that moved in opposite directions. The results showed that pupil size changed in accordance with the luminance of the pattern in the slow phase of OKN immediately after OKN transition. This suggests that pupil size is modulated with OKN in transparent motion.

Saccade dysmetria indicates attenuated visual exploration in autism spectrum disorder

BACKGROUND

Visual exploration in autism spectrum disorder (ASD) is characterized by attenuated social attention. The underlying oculomotor function during visual exploration is understudied, whereas oculomotor function during restricted viewing suggested saccade dysmetria in ASD by altered pontocerebellar motor modulation.

METHODS

Oculomotor function was recorded using remote eye tracking in 142 ASD participants and 142 matched neurotypical controls during free viewing of naturalistic videos with and without human content. The sample was heterogenous concerning age (6-30 years), cognitive ability (60-140 IQ), and male/female ratio (3:1). Oculomotor function was defined as saccade, fixation, and pupil-dilation features that were compared between groups in linear mixed models. Oculomotor function was investigated as ASD classifier and features were correlated with clinical measures.

RESULTS

We observed decreased saccade duration (∆M = -0.50, CI [-0.21, -0.78]) and amplitude (∆M = -0.42, CI [-0.12, -0.72]), which was independent of human video content. We observed null findings concerning fixation and pupil-dilation features (POWER = .81). Oculomotor function is a valid ASD classifier comparable to social attention concerning discriminative power. Within ASD, saccade features correlated with measures of restricted and repetitive behavior.

CONCLUSIONS

We conclude saccade dysmetria as ASD oculomotor phenotype relevant to visual exploration. Decreased saccade amplitude and duration indicate spatially clustered fixations that attenuate visual exploration and emphasize endogenous over exogenous attention. We propose altered pontocerebellar motor modulation as underlying mechanism that contributes to atypical (oculo-)motor coordination and attention function in ASD.

The additive nature of the human multisensory evoked pupil response

Pupillometry has received increased interest for its usefulness in measuring various sensory processes as an alternative to behavioural assessments. This is also apparent for multisensory investigations. Studies of the multisensory pupil response, however, have produced conflicting results. Some studies observed super-additive multisensory pupil responses, indicative of multisensory integration (MSI). Others observed additive multisensory pupil responses even though reaction time (RT) measures were indicative of MSI. Therefore, in the present study, we investigated the nature of the multisensory pupil response by combining methodological approaches of previous studies while using supra-threshold stimuli only. In two experiments we presented auditory and visual stimuli to observers that evoked a(n) (onset) response (be it constriction or dilation) in a simple detection task and a change detection task. In both experiments, the RT data indicated MSI as shown by race model inequality violation. Still, the multisensory pupil response in both experiments could best be explained by linear summation of the unisensory pupil responses. We conclude that the multisensory pupil response for supra-threshold stimuli is additive in nature and cannot be used as a measure of MSI, as only a departure from additivity can unequivocally demonstrate an interaction between the senses.

Phasic Activation of Dorsal Raphe Serotonergic Neurons Increases Pupil Size

Transient variations in pupil size (PS) under constant luminance are coupled to rapid changes in arousal state,1-3 which have been interpreted as vigilance,4 salience,5 or a surprise signal.6-8 Neural control of such fluctuations presumably involves multiple brain regions5,9-11 and neuromodulatory systems,3,12,13 but it is often associated with phasic activity of the noradrenergic system.9,12,14,15 Serotonin (5-HT), a neuromodulator also implicated in aspects of arousal16 such as sleep-wake transitions,17 motivational state regulation,18 and signaling of unexpected events,19 seems to affect PS,20-24 but these effects have not been investigated in detail. Here we show that phasic 5-HT neuron stimulation causes transient PS changes. We used optogenetic activation of 5-HT neurons in the dorsal raphe nucleus (DRN) of head-fixed mice performing a foraging task. 5-HT-driven modulations of PS were maintained throughout the photostimulation period and sustained for a few seconds after the end of stimulation. We found no evidence that the increase in PS with activation of 5-HT neurons resulted from interactions of photostimulation with behavioral variables, such as locomotion or licking. Furthermore, we observed that the effect of 5-HT on PS depended on the level of environmental uncertainty, consistent with the idea that 5-HT could report a surprise signal.19 These results advance our understanding of the neuromodulatory control of PS, revealing a tight relationship between phasic activation of 5-HT neurons and changes in PS.

Modeling pupil responses to rapid sequential events

Pupil size is an easily accessible, noninvasive online indicator of various perceptual and cognitive processes. Pupil measurements have the potential to reveal continuous processing dynamics throughout an experimental trial, including anticipatory responses. However, the relatively sluggish (~2 s) response dynamics of pupil dilation make it challenging to connect changes in pupil size to events occurring close together in time. Researchers have used models to link changes in pupil size to specific trial events, but such methods have not been systematically evaluated. Here we developed and evaluated a general linear model (GLM) pipeline that estimates pupillary responses to multiple rapid events within an experimental trial. We evaluated the modeling approach using a sample dataset in which multiple sequential stimuli were presented within 2-s trials. We found: (1) Model fits improved when the pupil impulse response function (PuRF) was fit for each observer. PuRFs varied substantially across individuals but were consistent for each individual. (2) Model fits also improved when pupil responses were not assumed to occur simultaneously with their associated trial events, but could have non-zero latencies. For example, pupil responses could anticipate predictable trial events. (3) Parameter recovery confirmed the validity of the fitting procedures, and we quantified the reliability of the parameter estimates for our sample dataset. (4) A cognitive task manipulation modulated pupil response amplitude. We provide our pupil analysis pipeline as open-source software (Pupil Response Estimation Toolbox: PRET) to facilitate the estimation of pupil responses and the evaluation of the estimates in other datasets.

Pupil size variation as a response to stress in European catfish and its application for social stress detection in albino conspecifics

Hormonal changes such as increased cortisol level in blood plasma in response to stress and social environmental stimuli are common among vertebrates including humans and typically accompanied by other physiological processes, such as changes in body pigmentation and/or pupil dilatation. The role of pupil size variation (PSV) as a response to stress have yet to be investigated in fish. We exposed albino and pigmented European catfish to short-term stress and measured changes in pupil size and cortisol level. Albinos showed lower pupil dilatation and higher cortisol levels than did pigmented conspecifics. A clear positive relationship between pupil dilatation and cortisol concentrations was observed for both pigmented and albino specimens, suggesting that PSV can be used as a stress indicator in fish, irrespective of albino’s inability to express social communication by coloring. During the follow-up, we investigated whether a penultimate contest between albino individuals would impact contestants’ social stress during subsequent contact. We observed PSV during the contact of unfamiliar albino catfish with different penultimate experiences (winner (W) and/or loser (L)). Then, the following treatment combinations were tested: WW, WL and LL. Twenty-four-hour contact of two unfamiliar catfish resulted in higher pupil dilatation among individuals with previous winner experience. Among treatment combinations, a WL contest displayed the highest pupil dilatation for winners. PSV reflected socially induced stress in individuals that was accompanied by the “winner” experience and dominancy in albinos. To conclude, the present study validates pupil dilatation as a non-invasive method to evaluate stress level in pigmented as well as albino fish in various contexts.

The Wealth State Awareness Effect on Attention Allocation in People From Impoverished and Affluent Groups

Previous studies have shown that poverty influences cognitive abilities and that those who have a negative living environment exhibit worse cognitive performance. In addition, eye measures vary following the manipulation of cognitive processing. We examined the distinctive changes in impoverished and affluent persons during tasks that require a high level of concentration using eye-tracking measures. Based on the poverty effect in impoverished people, this study explored how wealth state awareness (WSA) influences them. It was found that the pupillary state indexes of the impoverished participants significantly changed when their WSA was regarding poverty. The results suggest that awareness of poverty may cause impoverished individuals to engage in tasks with more attention allocation and more concentration in the more difficult tasks but that a WSA regarding wealth does not have such effect on them. WSA has no significant effects on their more affluent peers. The findings of this study can contribute to research on WSA effects on impoverished individuals from the perspective of eye measures.

The intraocular pressure responses to oral academic examination: The influence of perceived levels of public speaking anxiety

We examined the intraocular pressure responsiveness to oral examination stress, as well as the mediating role of the perceived levels of public speaking anxiety on intraocular pressure changes. Thirty-two university students (intervention group) publicly defended their degree final project, and intraocular pressure and perceived levels of stress were measured before and after the oral presentation, and after 10 min of recovery. The control group (n = 32) was not exposed to any stressful situation, and the dependent measures were taken at the same time points. All participants completed the public speaking anxiety scale. The intervention group exhibited higher intraocular pressure values before the oral presentation (corrected p-values<0.001), with these changes being positively associated with the perceived levels of public speaking anxiety (p < 0.001, r = 0.71). Our results reveal a mediating role of public speaking anxiety on the intraocular pressure responsiveness to oral examination stress, and highlight the utility of intraocular pressure as an indicator of stress in applied situations.

Advances in Multimodal Emotion Recognition Based on Brain-Computer Interfaces

With the continuous development of portable noninvasive human sensor technologies such as brain-computer interfaces (BCI), multimodal emotion recognition has attracted increasing attention in the area of affective computing. This paper primarily discusses the progress of research into multimodal emotion recognition based on BCI and reviews three types of multimodal affective BCI (aBCI): aBCI based on a combination of behavior and brain signals, aBCI based on various hybrid neurophysiology modalities and aBCI based on heterogeneous sensory stimuli. For each type of aBCI, we further review several representative multimodal aBCI systems, including their design principles, paradigms, algorithms, experimental results and corresponding advantages. Finally, we identify several important issues and research directions for multimodal emotion recognition based on BCI.

Background luminance effects on pupil size associated with emotion and saccade preparation

Pupil dilation is consistently evoked by affective and cognitive processing, and this dilation can result from sympathetic activation or parasympathetic inhibition. The relative contributions of the sympathetic and parasympathetic systems on the pupillary response induced by emotion and cognition may be different. Sympathetic and parasympathetic activity is regulated by global luminance level. Higher luminance levels lead to greater activation of the parasympathetic system while lower luminance levels lead to greater activation of the sympathetic system. To understand the contributions of the sympathetic and parasympathetic nervous systems to pupillary responses associated with emotion and saccade preparation, emotional auditory stimuli were presented following the fixation cue whose color indicated instruction to perform a pro- or anti-saccade while varying the background luminance level. Pupil dilation was evoked by emotional auditory stimuli and modulated by arousal level. More importantly, greater pupil dilation was observed with a dark background, compared to a bright background. In contrast, pupil dilation responses associated with saccade preparation were larger with the bright background than the dark background. Together, these results suggest that arousal-induced pupil dilation was mainly mediated by sympathetic activation, but pupil dilation related to saccade preparation was primarily mediated by parasympathetic inhibition.

Understanding the underlying mechanisms of Quiet Eye: The role of microsaccades, small saccades and pupil-size before final movement initiation in a soccer penalty kick

Experts keep a steady final fixation at a specific location just before final movement initiation, the so-called "quiet eye" (QE). However, the eyes are rarely "quiet", and small eye movements occur during visual fixation. The current research investigated the subtle eye movements and underlying mechanisms immediately prior to and during QE. The gaze behaviour of 8 intermediate-level goalkeepers was recorded as they moved (either left or right) in an attempt to predict the future direction of the ball during a soccer penalty kick. Goalkeepers were more likely to predict the direction of the penalty, which was coupled with delaying movement initiation. The temporal sequence of microsaccade rates dropped ∼1000 ms before goalkeepers' final movement initiation. Saccade rates increased, reaching a peak ∼500 ms before final movement initiation, concomitant with microsaccades reduction. Microsaccades predicted the goalkeepers' direction, oriented to the right when goalkeepers moved to the right, and conversely to the left when they moved to the left. Microsaccades may be modulated by attention and appear functionally related to saccadic intrusions. Pupil-size increased proportionally with the lead up to the instance of the penalty being kicked, reaching a plateau at final movement initiation. In conclusion, microsaccades and small saccades could improve the perception of the soccer penalty kick, helping athletes during the period that precedes the critical movement initiation, shifting from covert to overt attention for identifying the useful cues necessary to guide the action.