Your eyes give you away: prestimulus changes in pupil diameter correlate with poststimulus task-related EEG dynamics

Are the P600 and P3 ERP components linked to the task-evoked pupillary response as a correlate of norepinephrine activity?

During language comprehension, anomalies and ambiguities in the input typically elicit the P600 event-related potential component. Although traditionally interpreted as a specific signal of combinatorial operations in sentence processing, the component has alternatively been proposed to be a variant of the oddball-sensitive, domain-general P3 component. In particular, both components might reflect phasic norepinephrine release from the locus coeruleus (LC/NE) to motivationally significant stimuli. In this preregistered study, we tested this hypothesis by relating both components to the task-evoked pupillary response, a putative biomarker of LC/NE activity. 36 participants completed a sentence comprehension task (containing 25% morphosyntactic violations) and a non-linguistic oddball task (containing 20% oddballs), while the EEG and pupil size were co-registered. Our results showed that the task-evoked pupillary response and the ERP amplitudes of both components were similarly affected by both experimental tasks. In the oddball task, there was also a temporally specific relationship between the P3 and the pupillary response beyond the shared oddball effect, thereby further linking the P3 to NE. Because this link was less reliable in the linguistic context, we did not find conclusive evidence for or against a relationship between the P600 and the pupillary response. Still, our findings further stimulate the debate on whether language-related ERPs are indeed specific to linguistic processes or shared across cognitive domains. However, further research is required to verify a potential link between the two ERP positivities and the LC/NE system as the common neural generator.

Dissociation of attentional state and behavioral outcome using local field potentials

Successful behavior depends on attentional state and other factors related to decision-making, which may modulate neuronal activity differently. Here, we investigated whether attentional state and behavioral outcome (i.e., whether a target is detected or missed) are distinguishable using the power and phase of local field potential (LFP) recorded bilaterally from area V4 of monkeys performing a cued visual attention task. To link each trial's outcome to pairwise measures of attention that are typically averaged across trials, we used several methods to obtain single-trial estimates of spike count correlation and phase consistency. Surprisingly, while attentional location was best discriminated using gamma and high-gamma power, behavioral outcome was best discriminated by alpha power and steady-state visually evoked potential. Power outperformed absolute phase in attentional/behavioral discriminability, although single-trial gamma phase consistency provided reasonably high attentional discriminability. Our results suggest a dissociation between the neuronal mechanisms that regulate attentional focus and behavioral outcome.

Correlated P300b and phasic pupil-dilation responses to motivationally significant stimuli

Motivationally significant events like oddball stimuli elicit both a characteristic event-related potential (ERPs) known as P300 and a set of autonomic responses including a phasic pupil dilation. Although co-occurring, P300 and pupil-dilation responses to oddball events have been repeatedly found to be uncorrelated, suggesting separate origins. We re-examined their relationship in the context of a three-stimulus version of the auditory oddball task, independently manipulating the frequency (rare vs. repeated) and motivational significance (relevance for the participant's task) of the stimuli. We used independent component analysis to derive a P300b component from EEG traces and linear modeling to separate a stimulus-related pupil-dilation response from a potentially confounding action-related response. These steps revealed that, once the complexity of ERP and pupil-dilation responses to oddball targets is accounted for, the amplitude of phasic pupil dilations and P300b are tightly and positively correlated (across participants: r = .69 p = .002), supporting their coordinated generation.

Double Dissociation of Spontaneous Alpha-Band Activity and Pupil-Linked Arousal on Additive and Multiplicative Perceptual Gain

Perception is a probabilistic process dependent on external stimulus properties and one's internal state. However, which internal states influence perception and via what mechanisms remain debated. We studied how spontaneous alpha-band activity (8-13 Hz) and pupil fluctuations impact visual detection and confidence across stimulus contrast levels (i.e., the contrast response function, CRF). In human subjects of both sexes, we found that low prestimulus alpha power induced an "additive" shift in the CRF, whereby stimuli were reported present more frequently at all contrast levels, including contrast of zero (i.e., false alarms). Conversely, prestimulus pupil size had a "multiplicative" effect on detection such that stimuli occurring during large pupil states (putatively corresponding to higher arousal) were perceived more frequently as contrast increased. Signal detection modeling reveals that alpha power changes detection criteria equally across the CRF but not detection sensitivity (d'), whereas pupil-linked arousal modulated sensitivity, particularly for higher contrasts. Interestingly, pupil size and alpha power were positively correlated, meaning that some of the effect of alpha on detection may be mediated by pupil fluctuations. However, pupil-independent alpha still induced an additive shift in the CRF corresponding to a criterion effect. Our data imply that low alpha boosts detection and confidence by an additive factor, rather than by a multiplicative scaling of contrast responses, a profile which captures the effect of pupil-linked arousal. We suggest that alpha power and arousal fluctuations have dissociable effects on behavior. Alpha reflects the baseline level of visual excitability, which can vary independent of arousal.

Linking tonic and phasic pupil responses to P300 amplitude in an emotional face-word Stroop task

The locus coeruleus-norepinephrine (LC-NE) system, which regulates arousal levels, is important for cognitive control, including emotional conflict resolution. Additionally, the LC-NE system is implicated in P300 generation. If the P300 is mediated by the LC-NE system, and considering the established correlations between LC activity and pupil dilation, P300 amplitude should correlate with task-evoked (phasic) pupil dilation on a trial-by-trial basis. However, prior studies, predominantly utilizing oddball-type paradigms, have not demonstrated correlations between concurrently recorded task-evoked pupil dilation and P300 responses. Using a recently developed emotional face-word Stroop task that links pupil dilation to the LC-NE system, here, we examined both intra- and inter-individual correlations between task-evoked pupil dilation and P300 amplitude. We found that lower accuracy, slower reaction times, and larger task-evoked pupil dilation were obtained in the incongruent compared to the congruent condition. Furthermore, we observed intra-individual correlations between task-evoked pupil dilation and P300 amplitude, with larger pupil dilation correlating with a greater P300 amplitude. In contrast, pupil dilation did not exhibit consistent correlations with N450 and N170 amplitudes. Baseline (tonic) pupil size also showed correlations with P300 and N170 amplitudes, with smaller pupil size corresponding to larger amplitude. Moreover, inter-individual differences in task-evoked pupil dilation between the congruent and incongruent conditions correlated with differences in reaction time and P300 amplitude, though these effects only approached significance. To summarize, our study provides evidence for a connection between task-evoked pupil dilation and P300 amplitude at the single-trial level, suggesting the involvement of the LC-NE system in P300 generation.

Self-regulating arousal via pupil-based biofeedback

The brain's arousal state is controlled by several neuromodulatory nuclei known to substantially influence cognition and mental well-being. Here we investigate whether human participants can gain volitional control of their arousal state using a pupil-based biofeedback approach. Our approach inverts a mechanism suggested by previous literature that links activity of the locus coeruleus, one of the key regulators of central arousal and pupil dynamics. We show that pupil-based biofeedback enables participants to acquire volitional control of pupil size. Applying pupil self-regulation systematically modulates activity of the locus coeruleus and other brainstem structures involved in arousal control. Furthermore, it modulates cardiovascular measures such as heart rate, and behavioural and psychophysiological responses during an oddball task. We provide evidence that pupil-based biofeedback makes the brain's arousal system accessible to volitional control, a finding that has tremendous potential for translation to behavioural and clinical applications across various domains, including stress-related and anxiety disorders.

Effects of pupil size as manipulated through ipRGC activation on visual processing

The size of the eyes' pupils determines how much light enters the eye and also how well this light is focused. Through this route, pupil size shapes the earliest stages of visual processing. Yet causal effects of pupil size on vision are poorly understood and rarely studied. Here we introduce a new way to manipulate pupil size, which relies on activation of intrinsically photosensitive retinal ganglion cells (ipRGCs) to induce sustained pupil constriction. We report the effects of both experimentally induced and spontaneous changes in pupil size on visual processing as measured through EEG. We compare these to the effects of stimulus intensity and covert visual attention, because previous studies have shown that these factors all have comparable effects on some common measures of early visual processing, such as detection performance and steady-state visual evoked potentials; yet it is still unclear whether these are superficial similarities, or rather whether they reflect similar underlying processes. Using a mix of neural-network decoding, ERP analyses, and time-frequency analyses, we find that induced pupil size, spontaneous pupil size, stimulus intensity, and covert visual attention all affect EEG responses, mainly over occipital and parietal electrodes, but-crucially-that they do so in qualitatively different ways. Induced and spontaneous pupil-size changes mainly modulate activity patterns (but not overall power or intertrial coherence) in the high-frequency beta range; this may reflect an effect of pupil size on oculomotor activity and/ or visual processing. In addition, spontaneous (but not induced) pupil size tends to correlate positively with intertrial coherence in the alpha band; this may reflect a non-causal relationship, mediated by arousal. Taken together, our findings suggest that pupil size has qualitatively different effects on visual processing from stimulus intensity and covert visual attention. This shows that pupil size as manipulated through ipRGC activation strongly affects visual processing, and provides concrete starting points for further study of this important yet understudied earliest stage of visual processing.

Pupil-linked arousal modulates network-level EEG signatures of attention reorienting during immersive multitasking

Objective. When multitasking, we must dynamically reorient our attention between different tasks. Attention reorienting is thought to arise through interactions of physiological arousal and brain-wide network dynamics. In this study, we investigated the relationship between pupil-linked arousal and electroencephalography (EEG) brain dynamics in a multitask driving paradigm conducted in virtual reality. We hypothesized that there would be an interaction between arousal and EEG dynamics and that this interaction would correlate with multitasking performance.Approach. We collected EEG and eye tracking data while subjects drove a motorcycle through a simulated city environment, with the instructions to count the number of target images they observed while avoiding crashing into a lead vehicle. The paradigm required the subjects to continuously reorient their attention between the two tasks. Subjects performed the paradigm under two conditions, one more difficult than the other.Main results. We found that task difficulty did not strongly correlate with pupil-linked arousal, and overall task performance increased as arousal level increased. A single-trial analysis revealed several interesting relationships between pupil-linked arousal and task-relevant EEG dynamics. Employing exact low-resolution electromagnetic tomography, we found that higher pupil-linked arousal led to greater EEG oscillatory activity, especially in regions associated with the dorsal attention network and ventral attention network (VAN). Consistent with our hypothesis, we found a relationship between EEG functional connectivity and pupil-linked arousal as a function of multitasking performance. Specifically, we found decreased functional connectivity between regions in the salience network (SN) and the VAN as pupil-linked arousal increased, suggesting that improved multitasking performance at high arousal levels may be due to a down-regulation in coupling between the VAN and the SN. Our results suggest that when multitasking, our brain rebalances arousal-based reorienting so that individual task demands can be met without prematurely reorienting to competing tasks.

Slow fluctuations in ongoing brain activity decrease in amplitude with ageing yet their impact on task-related evoked responses is dissociable from behavior

In humans, ageing is characterized by decreased brain signal variability and increased behavioral variability. To understand how reduced brain variability segregates with increased behavioral variability, we investigated the association between reaction time variability, evoked brain responses and ongoing brain signal dynamics, in young (N=36) and older adults (N=39). We studied the electroencephalogram (EEG) and pupil size fluctuations to characterize the cortical and arousal responses elicited by a cued go/no-go task. Evoked responses were strongly modulated by slow (<2 Hz) fluctuations of the ongoing signals, which presented reduced power in the older participants. Although variability of the evoked responses was lower in the older participants, once we adjusted for the effect of the ongoing signal fluctuations, evoked responses were equally variable in both groups. Moreover, the modulation of the evoked responses caused by the ongoing signal fluctuations had no impact on reaction time, thereby explaining why although ongoing brain signal variability is decreased in older individuals, behavioral variability is not. Finally, we showed that adjusting for the effect of the ongoing signal was critical to unmask the link between neural responses and behavior as well as the link between task-related evoked EEG and pupil responses.

EEG Signals Index a Global Signature of Arousal Embedded in Neuronal Population Recordings

Electroencephalography (EEG) has long been used to index brain states, from early studies describing activity in the presence and absence of visual stimulation to modern work employing complex perceptual tasks. These studies have shed light on brain-wide signals but often lack explanatory power at the single neuron level. Similarly, single neuron recordings can suffer from an inability to measure brain-wide signals accessible using EEG. Here, we combined these techniques while monkeys performed a change detection task and discovered a novel link between spontaneous EEG activity and a neural signal embedded in the spiking responses of neuronal populations. This "slow drift" was associated with fluctuations in the subjects' arousal levels over time: decreases in prestimulus α power were accompanied by increases in pupil size and decreases in microsaccade rate. These results show that brain-wide EEG signals can be used to index modes of activity present in single neuron recordings, that in turn reflect global changes in brain state that influence perception and behavior.

Coupling of pupil- and neuronal population dynamics reveals diverse influences of arousal on cortical processing

Fluctuations in arousal, controlled by subcortical neuromodulatory systems, continuously shape cortical state, with profound consequences for information processing. Yet, how arousal signals influence cortical population activity in detail has so far only been characterized for a few selected brain regions. Traditional accounts conceptualize arousal as a homogeneous modulator of neural population activity across the cerebral cortex. Recent insights, however, point to a higher specificity of arousal effects on different components of neural activity and across cortical regions. Here, we provide a comprehensive account of the relationships between fluctuations in arousal and neuronal population activity across the human brain. Exploiting the established link between pupil size and central arousal systems, we performed concurrent magnetoencephalographic (MEG) and pupillographic recordings in a large number of participants, pooled across three laboratories. We found a cascade of effects relative to the peak timing of spontaneous pupil dilations: Decreases in low-frequency (2-8 Hz) activity in temporal and lateral frontal cortex, followed by increased high-frequency (>64 Hz) activity in mid-frontal regions, followed by monotonic and inverted U relationships with intermediate frequency-range activity (8-32 Hz) in occipito-parietal regions. Pupil-linked arousal also coincided with widespread changes in the structure of the aperiodic component of cortical population activity, indicative of changes in the excitation-inhibition balance in underlying microcircuits. Our results provide a novel basis for studying the arousal modulation of cognitive computations in cortical circuits.

Superior frontal regions reflect the dynamics of task engagement and theta band-related control processes in time-on task effects

Impairment of cognitive performance is often observed in time-on tasks. Theoretical considerations suggest that especially prefrontal cortex cognitive control functions is affected by time-on-task effects, but the role of effort/task engagement is not understood. We examine time-on-task effects in cognitive control on a neurophysiological level using a working-memory modulated response inhibition task and inter-relate prefrontal neuroanatomical region-specific theta-band activity with pupil diameter data using EEG-beamforming approaches. We show that task performance declines with time-on tasks, which was paralleled by a concomitant decreases of task-evoked superior frontal gyrus theta-band activity and a reduction in phasic pupil diameter modulations. A strong relation between cognitive control-related superior frontal theta-band activity and effort/task engagement indexed by phasic pupil diameter modulations was observed in the beginning of the experiment, especially for tasks requiring inhibitory controls and demanding high working memory. This strong relation vanished at the end of the experiment, suggesting a decoupling of cognitive control resources useable for a task and effort invested that characterizes time-on-task effects in prefrontal cortical structures.

Time-on-task effects on working memory gating processes—A role of theta synchronization and the norepinephrine system

Performance impairment as an effect of prolonged engagement in a specific task is commonly observed. Although this is a well-known effect in everyday life, little is known about how this affects central cognitive functions such as working memory (WM) processes. In the current study, we ask how time-on-task affects WM gating processes and thus processes regulating WM maintenance and updating. To this end, we combined electroencephalography methods and recordings of the pupil diameter as an indirect of the norepinephrine (NE) system activity. Our results showed that only WM gate opening but not closing processes showed time-on-task effects. On the neurophysiological level, this was associated with modulation of dorsolateral prefrontal theta band synchronization processes, which vanished with time-on-task during WM gate opening. Interestingly, also the modulatory pattern of the NE system, as inferred using pupil diameter data, changed. At the beginning, a strong correlation of pupil diameter data and theta band synchronization processes during WM gate opening is observed. This modulatory effect vanished at the end of the experiment. The results show that time-on-task has very specific effects on WM gate opening and closing processes and suggests an important role of NE system in the time-on-task effect on WM gate opening process.

No evidence for a modulating effect of continuous transcutaneous auricular vagus nerve stimulation on markers of noradrenergic activity

Although transcutaneous auricular vagus nerve stimulation (taVNS) is thought to increase central noradrenergic activity, findings supporting such mechanism are scarce and inconsistent. This study aimed to investigate whether taVNS modulates indirect markers of phasic and tonic noradrenergic activity. Sixty-six healthy participants performed a novelty auditory oddball task twice on separate days: once while receiving taVNS (left cymba concha), once during sham (left earlobe) stimulation. To maximize potential effects, the stimulation was delivered continuously (frequency: 25 Hz; width: 250 μs) at an intensity individually calibrated to the maximal level below pain threshold. The stimulation was administered 10 min before the oddball task and maintained throughout the session. Event-related pupil dilation (ERPD) to target stimuli and pre-stimulus baseline pupil size were assessed during the oddball task as markers of phasic and tonic noradrenergic activity, respectively. Prior to and at the end of stimulation, tonic pupil size at rest, cortisol, and salivary alpha-amylase were assessed as markers of tonic noradrenergic activity. Finally, we explored the effect of taVNS on cardiac vagal activity, respiratory rate, and salivary flow rate. Results showed a greater ERPD to both target and novelty compared to standard stimuli in the oddball task. In contrast to our hypotheses, taVNS did not impact any of the tested markers. Our findings strongly suggest that continuous stimulation of the cymba concha with the tested stimulation parameters is ineffective to increase noradrenergic activity via a vagal pathway.

Integrating neural and ocular attention reorienting signals in virtual reality

Objective.Reorienting is central to how humans direct attention to different stimuli in their environment. Previous studies typically employ well-controlled paradigms with limited eye and head movements to study the neural and physiological processes underlying attention reorienting. Here, we aim to better understand the relationship between gaze and attention reorienting using a naturalistic virtual reality (VR)-based target detection paradigm.Approach.Subjects were navigated through a city and instructed to count the number of targets that appeared on the street. Subjects performed the task in a fixed condition with no head movement and in a free condition where head movements were allowed. Electroencephalography (EEG), gaze and pupil data were collected. To investigate how neural and physiological reorienting signals are distributed across different gaze events, we used hierarchical discriminant component analysis (HDCA) to identify EEG and pupil-based discriminating components. Mixed-effects general linear models (GLM) were used to determine the correlation between these discriminating components and the different gaze events time. HDCA was also used to combine EEG, pupil and dwell time signals to classify reorienting events.Main results.In both EEG and pupil, dwell time contributes most significantly to the reorienting signals. However, when dwell times were orthogonalized against other gaze events, the distributions of the reorienting signals were different across the two modalities, with EEG reorienting signals leading that of the pupil reorienting signals. We also found that the hybrid classifier that integrates EEG, pupil and dwell time features detects the reorienting signals in both the fixed (AUC = 0.79) and the free (AUC = 0.77) condition.Significance.We show that the neural and ocular reorienting signals are distributed differently across gaze events when a subject is immersed in VR, but nevertheless can be captured and integrated to classify target vs. distractor objects to which the human subject orients.

Responsiveness to left-prefrontal tDCS varies according to arousal levels

Over the past two decades, the postulated modulatory effects of transcranial direct current stimulation (tDCS) on the human brain have been extensively investigated. However, recent concerns on reliability of tDCS effects have been raised, principally due to reduced replicability and to interindividual variability in response to tDCS. These inconsistencies are likely due to the interplay between the level of induced cortical excitability and unaccounted structural and state‐dependent functional factors. On these grounds, we aimed at verifying whether the behavioural effects induced by a common tDCS montage (F3‐rSOA) were influenced by the participants' arousal levels, as part of a broader mechanism of state‐dependency. Pupillary dynamics were recorded during an auditory oddball task while applying either a sham or real tDCS. The tDCS effects were evaluated as a function of subjective and physiological arousal predictors (STAI‐Y State scores and pre‐stimulus pupil size, respectively). We showed that prefrontal tDCS hindered task learning effects on response speed such that performance improvement occurred during sham, but not real stimulation. Moreover, both subjective and physiological arousal predictors significantly explained performance during real tDCS, with interaction effects showing performance improvement only with moderate arousal levels; likewise, pupil response was affected by real tDCS according to the ongoing levels of arousal, with reduced dilation during higher arousal trials. These findings highlight the potential role of arousal in shaping the neuromodulatory outcome, thus emphasizing a more careful interpretation of null or negative results while also encouraging more individually tailored tDCS applications based on arousal levels, especially in clinical populations.

Increasing pupil size is associated with improved detection performance in the periphery

Visible light enters our body via the pupil. By changing its size, the pupil shapes visual input. Small apertures increase the resolution of high spatial frequencies, thus allowing discrimination of fine details. Large apertures, in contrast, provide a better signal-to-noise ratio, because more light can enter the eye. This should lead to better detection performance of peripheral stimuli. Experiment 1 shows that the effect can reliably be demonstrated even in a less controlled online setting. In Experiment 2, pupil size was measured in a laboratory using an eye tracker. The findings replicate findings showing that large pupils provide an advantage for peripheral detection of faint stimuli. Moreover, not only pupil size during information intake in the current trial n, but also its interaction with pupil size preceding information intake, i.e., in trial n-1, predicted performance. This suggests that in addition to absolute pupil size, the extent of pupillary change provides a mechanism to modulate perceptual functions. The results are discussed in terms of low-level sensory as well as higher-level arousal-driven changes in stimulus processing.

The confounding effects of eye blinking on pupillometry, and their remedy

Pupillometry, thanks to its strong relationship with cognitive factors and recent advancements in measuring techniques, has become popular among cognitive or neural scientists as a tool for studying the physiological processes involved in mental or neural processes. Despite this growing popularity of pupillometry, the methodological understanding of pupillometry is limited, especially regarding potential factors that may threaten pupillary measurements' validity. Eye blinking can be a factor because it frequently occurs in a manner dependent on many cognitive components and induces a pulse-like pupillary change consisting of constriction and dilation with substantive magnitude and length. We set out to characterize the basic properties of this "blink-locked pupillary response (BPR)," including the shape and magnitude of BPR and their variability across subjects and blinks, as the first step of studying the confounding nature of eye blinking. Then, we demonstrated how the dependency of eye blinking on cognitive factors could confound, via BPR, the pupillary responses that are supposed to reflect the cognitive states of interest. By building a statistical model of how the confounding effects of eye blinking occur, we proposed a probabilistic-inference algorithm of de-confounding raw pupillary measurements and showed that the proposed algorithm selectively removed BPR and enhanced the statistical power of pupillometry experiments. Our findings call for attention to the presence and confounding nature of BPR in pupillometry. The algorithm we developed here can be used as an effective remedy for the confounding effects of BPR on pupillometry.

Talker discontinuity disrupts attention to speech: Evidence from EEG and pupillometry

Speech is processed less efficiently from discontinuous, mixed talkers than one consistent talker, but little is known about the neural mechanisms for processing talker variability. Here, we measured psychophysiological responses to talker variability using electroencephalography (EEG) and pupillometry while listeners performed a delayed recall of digit span task. Listeners heard and recalled seven-digit sequences with both talker (single- vs. mixed-talker digits) and temporal (0- vs. 500-ms inter-digit intervals) discontinuities. Talker discontinuity reduced serial recall accuracy. Both talker and temporal discontinuities elicited P3a-like neural evoked response, while rapid processing of mixed-talkers' speech led to increased phasic pupil dilation. Furthermore, mixed-talkers' speech produced less alpha oscillatory power during working memory maintenance, but not during speech encoding. Overall, these results are consistent with an auditory attention and streaming framework in which talker discontinuity leads to involuntary, stimulus-driven attentional reorientation to novel speech sources, resulting in the processing interference classically associated with talker variability.

Spectral signature and behavioral consequence of spontaneous shifts of pupil-linked arousal in human

Arousal levels perpetually rise and fall spontaneously. How markers of arousal—pupil size and frequency content of brain activity—relate to each other and influence behavior in humans is poorly understood. We simultaneously monitored magnetoencephalography and pupil in healthy volunteers at rest and during a visual perceptual decision-making task. Spontaneously varying pupil size correlates with power of brain activity in most frequency bands across large-scale resting state cortical networks. Pupil size recorded at prestimulus baseline correlates with subsequent shifts in detection bias (c) and sensitivity (d’). When dissociated from pupil-linked state, prestimulus spectral power of resting state networks still predicts perceptual behavior. Fast spontaneous pupil constriction and dilation correlate with large-scale brain activity as well but not perceptual behavior. Our results illuminate the relation between central and peripheral arousal markers and their respective roles in human perceptual decision-making.

Neuronal correlates of the subjective experience of attention

The effect of top–down attention on stimulus‐evoked responses and alpha oscillations and the association between arousal and pupil diameter are well established. However, the relationship between these indices, and their contribution to the subjective experience of attention, remains largely unknown. Participants performed a sustained (10–30 s) attention task in which rare (10%) targets were detected within continuous tactile stimulation (16 Hz). Trials were followed by attention ratings on an 8‐point visual scale. Attention ratings correlated negatively with contralateral somatosensory alpha power and positively with pupil diameter. The effect of pupil diameter on attention ratings extended into the following trial, reflecting a sustained aspect of attention related to vigilance. The effect of alpha power did not carry over to the next trial and furthermore mediated the association between pupil diameter and attention ratings. Variations in steady‐state amplitude reflected stimulus processing under the influence of alpha oscillations but were only weakly related to subjective ratings of attention. Together, our results show that both alpha power and pupil diameter are reflected in the subjective experience of attention, albeit on different time spans, while continuous stimulus processing might not contribute to the experience of attention.

Non-Contact Measurement of Motion Sickness Using Pupillary Rhythms from an Infrared Camera

Both physiological and neurological mechanisms are reflected in pupillary rhythms via neural pathways between the brain and pupil nerves. This study aims to interpret the phenomenon of motion sickness such as fatigue, anxiety, nausea and disorientation using these mechanisms and to develop an advanced non-contact measurement method from an infrared webcam. Twenty-four volunteers (12 females) experienced virtual reality content through both two-dimensional and head-mounted device interpretations. An irregular pattern of the pupillary rhythms, demonstrated by an increasing mean and standard deviation of pupil diameter and decreasing pupillary rhythm coherence ratio, was revealed after the participants experienced motion sickness. The motion sickness was induced while watching the head-mounted device as compared to the two-dimensional virtual reality, with the motion sickness strongly related to the visual information processing load. In addition, the proposed method was verified using a new experimental dataset for 23 participants (11 females), with a classification performance of 89.6% (n = 48) and 80.4% (n = 46) for training and test sets using a support vector machine with a radial basis function kernel, respectively. The proposed method was proven to be capable of quantitatively measuring and monitoring motion sickness in real-time in a simple, economical and contactless manner using an infrared camera.

Pupil Dilation and the Slow Wave ERP Reflect Surprise about Choice Outcome Resulting from Intrinsic Variability in Decision Confidence

Central to human and animal cognition is the ability to learn from feedback in order to optimize future rewards. Such a learning signal might be encoded and broadcasted by the brain's arousal systems, including the noradrenergic locus coeruleus. Pupil responses and the positive slow wave component of event-related potentials reflect rapid changes in the arousal level of the brain. Here, we ask whether and how these variables may reflect surprise: the mismatch between one's expectation about being correct and the outcome of a decision, when expectations fluctuate due to internal factors (e.g., engagement). We show that during an elementary decision task in the face of uncertainty both physiological markers of phasic arousal reflect surprise. We further show that pupil responses and slow wave event-related potential are unrelated to each other and that prediction error computations depend on feedback awareness. These results further advance our understanding of the role of central arousal systems in decision-making under uncertainty.

Pupillometric Complexity and Symmetricity Follow Inverted-U Curves Against Baseline Diameter Due to Crossed Locus Coeruleus Projections to the Edinger-Westphal Nucleus

In addition to photic reflex function, the temporal behavior of the pupil diameter reflects levels of arousal and attention and thus internal cognitive neural activity. Recent studies have reported that these behaviors are characterized by baseline activity, temporal complexity, and symmetricity (i.e., degree of symmetry) between the right and left pupil diameters. We hypothesized that experimental analysis to reveal relationships among these characteristics and model-based analysis focusing on the newly discovered contralateral projection from the locus coeruleus (LC) to the Edinger-Westphal nucleus (EWN) within the neural system for controlling pupil diameter could contribute to another dimension of understanding of complex pupil dynamics. In this study, we aimed to validate our hypothesis by analyzing the pupillary hippus in the healthy resting state in terms of sample entropy (SampEn), to capture complexity, and transfer entropy (TranEn), to capture symmetricity. We also constructed a neural model embedded with the new findings on neural pathways. The following results were observed: first, according to surrogate data analysis, the complexity and symmetricity of pupil diameter changes reflect a non-linear deterministic process. Second, both the complexity and the symmetricity are unimodal, peaking at intermediate pupil diameters. Third, according to simulation results, the neural network that controls pupil diameter has an inverted U-shaped profile of complexity and symmetricity vs. baseline LC activity; this tendency is enhanced by the contralateral synaptic projections from the LCs to the EWNs. Thus, we characterized the typical relationships between the baseline activity and the complexity and symmetricity of the pupillometric data in terms of SampEn and TranEn. Our evaluation method and findings may facilitate the development of estimation and diagnostic tools for exploring states of the healthy brain and psychiatric disorders based on measurements of pupil diameter.

Effect of the Trigeminal Nerve Stimulation on Auditory Event-Related Potentials

Trigeminal sensorimotor activity stimulates arousal and cognitive performance, likely through activation of the locus coeruleus (LC). In this study we investigated, in normal subjects, the effects of bilateral trigeminal nerve stimulation (TNS) on the LC-dependent P300 wave, elicited by an acoustic oddball paradigm. Pupil size, a proxy of LC activity, and electroencephalographic power changes were also investigated. Before TNS/sham-TNS, pupil size did not correlate with P300 amplitude across subjects. After TNS but not sham-TNS, a positive correlation emerged between P300 amplitude and pupil size within frontal and median cortical regions. TNS also reduced P300 amplitude in several cortical areas. In both groups, before and after TNS/sham-TNS, subjects correctly indicated all the target stimuli. We propose that TNS activates LC, increasing the cortical norepinephrine release and the dependence of the P300 upon basal LC activity. Enhancing the signal-to-noise ratio of cortical neurons, norepinephrine may improve the sensory processing, allowing the subject to reach the best discriminative performance with a lower level of neural activation (i.e., a lower P300 amplitude). The study suggests that TNS could be used for improving cognitive performance in patients affected by cognitive disorders or arousal dysfunctions.

Pupillometry as a reliable metric of auditory detection and discrimination across diverse stimulus paradigms in animal models

Estimates of detection and discrimination thresholds are often used to explore broad perceptual similarities between human subjects and animal models. Pupillometry shows great promise as a non-invasive, easily-deployable method of comparing human and animal thresholds. Using pupillometry, previous studies in animal models have obtained threshold estimates to simple stimuli such as pure tones, but have not explored whether similar pupil responses can be evoked by complex stimuli, what other stimulus contingencies might affect stimulus-evoked pupil responses, and if pupil responses can be modulated by experience or short-term training. In this study, we used an auditory oddball paradigm to estimate detection and discrimination thresholds across a wide range of stimuli in guinea pigs. We demonstrate that pupillometry yields reliable detection and discrimination thresholds across a range of simple (tones) and complex (conspecific vocalizations) stimuli; that pupil responses can be robustly evoked using different stimulus contingencies (low-level acoustic changes, or higher level categorical changes); and that pupil responses are modulated by short-term training. These results lay the foundation for using pupillometry as a reliable method of estimating thresholds in large experimental cohorts, and unveil the full potential of using pupillometry to explore broad similarities between humans and animal models.

Perceptual Behavior Depends Differently on Pupil-Linked Arousal and Heartbeat Dynamics-Linked Arousal in Rats Performing Tactile Discrimination Tasks

Several physiology signals, including heart rate and pupil size, have been widely used as peripheral indices of arousal to evaluate the effects of arousal on brain functions. However, whether behavior depends differently on arousal indexed by these physiological signals remains unclear. We simultaneously recorded electrocardiogram (ECG) and pupil size in head-fixed rats performing tactile discrimination tasks. We found both heartbeat dynamics and pupil size co-varied with behavioral outcomes, indicating behavior was dependent upon arousal indexed by the two physiological signals. To estimate the potential difference between the effects of pupil-linked arousal and heart rate-linked arousal on behavior, we constructed a Bayesian decoder to predict animals' behavior from pupil size and heart rate prior to stimulus presentation. The performance of the decoder was significantly better when using both heart rate and pupil size as inputs than when using either of them alone, suggesting the effects of the two arousal systems on behavior are not completely redundant. Supporting this notion, we found that, on a substantial portion of trials correctly predicted by the heart rate-based decoder, the pupil size-based decoder failed to correctly predict animals' behavior. Taken together, these results suggest that pupil-linked and heart rate-linked arousal systems exert different influences on animals' behavior.

Ballistocardiogram Artifact Reduction in Simultaneous EEG-fMRI Using Deep Learning

OBJECTIVE

The concurrent recording of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) is a technique that has received much attention due to its potential for combined high temporal and spatial resolution. However, the ballistocardiogram (BCG), a large-amplitude artifact caused by cardiac induced movement contaminates the EEG during EEG-fMRI recordings. Removal of BCG in software has generally made use of linear decompositions of the corrupted EEG. This is not ideal as the BCG signal propagates in a manner which is non-linearly dependent on the electrocardiogram (ECG). In this paper, we present a novel method for BCG artifact suppression using recurrent neural networks (RNNs).

METHODS

EEG signals were recovered by training RNNs on the nonlinear mappings between ECG and the BCG corrupted EEG. We evaluated our model's performance against the commonly used Optimal Basis Set (OBS) method at the level of individual subjects, and investigated generalization across subjects.

RESULTS

We show that our algorithm can generate larger average power reduction of the BCG at critical frequencies, while simultaneously improving task relevant EEG based classification.

CONCLUSION

The presented deep learning architecture can be used to reduce BCG related artifacts in EEG-fMRI recordings.

SIGNIFICANCE

We present a deep learning approach that can be used to suppress the BCG artifact in EEG-fMRI without the use of additional hardware. This method may have scope to be combined with current hardware methods, operate in real-time and be used for direct modeling of the BCG.

Inter- and intra-individual coupling between pupillary, electrophysiological, and behavioral responses in a visual oddball task

Although the P3b component of the event-related brain potential is one of the most widely studied components, its underlying generators are not currently well understood. Recent theories have suggested that the P3b is triggered by phasic activation of the locus-coeruleus norepinephrine (LC-NE) system, an important control center implicated in facilitating optimal task-relevant behavior. Previous research has reported strong correlations between pupil dilation and LC activity, suggesting that pupil diameter is a useful indicator for ongoing LC-NE activity. Given the strong relationship between LC activity and pupil dilation, if the P3b is driven by phasic LC activity, there should be a robust trial-to-trial relationship with the phasic pupillary dilation response (PDR). However, previous work examining relationships between concurrently recorded pupillary and P3b responses has not supported this. One possibility is that the relationship between the measures might be carried primarily by either inter-individual (i.e., between-participant) or intra-individual (i.e., within-participant) contributions to coupling, and prior work has not systematically delineated these relationships. Doing so in the current study, we do not find evidence for either inter-individual or intra-individual relationships between the PDR and P3b responses. However, baseline pupil dilation did predict the P3b. Interestingly, both the PDR and P3b independently predicted inter-individual and intra-individual variability in decision response time. Implications for the LC-P3b hypothesis are discussed.

Memory failure predicted by attention lapsing and media multitasking

With the explosion of digital media and technologies, scholars, educators and the public have become increasingly vocal about the role that an 'attention economy' has in our lives1. The rise of the current digital culture coincides with longstanding scientific questions about why humans sometimes remember and sometimes forget, and why some individuals remember better than others2-6. Here we examine whether spontaneous attention lapses-in the moment7-12, across individuals13-15 and as a function of everyday media multitasking16-19-negatively correlate with remembering. Electroencephalography and pupillometry measures of attention20,21 were recorded as eighty young adults (mean age, 21.7 years) performed a goal-directed episodic encoding and retrieval task22. Trait-level sustained attention was further quantified using task-based23 and questionnaire measures24,25. Using trial-to-trial retrieval data, we show that tonic lapses in attention in the moment before remembering, assayed by posterior alpha power and pupil diameter, were correlated with reductions in neural signals of goal coding and memory, along with behavioural forgetting. Independent measures of trait-level attention lapsing mediated the relationship between neural assays of lapsing and memory performance, and between media multitasking and memory. Attention lapses partially account for why we remember or forget in the moment, and why some individuals remember better than others. Heavier media multitasking is associated with a propensity to have attention lapses and forget.

Simultaneous EEG and pupillary evidence for post-error arousal during a speeded performance task

Arousal evoked by detecting a performance error may provide a mechanism by which error detection leads to either adaptive or maladaptive changes in attention and performance. By pairing EEG data acquisition with simultaneous measurements of pupil diameter, which is thought to reflect norepinephrinergic arousal, this study tested whether transient changes in EEG oscillations in the alpha frequency range (8-12 Hz) following performance mistakes may reflect error-evoked arousal. In the inter-trial interval following performance mistakes (approximately 8% of trials), pupil diameter increased and EEG alpha power decreased, compared to the inter-trial interval following correct responses. Moreover when trials were binned based on pupil diameter on a within-subjects basis, trials with greater pupil diameter were associated with lower EEG alpha power during the inter-trial interval. This pattern of association suggests that error-related alpha suppression, like pupil dilation, reflects arousal in response to error commission. Errors were also followed by worse next-trial performance, implying that error-evoked arousal may not always be beneficial for adaptive control.

Brain dynamics for confidence-weighted learning

Learning in a changing, uncertain environment is a difficult problem. A popular solution is to predict future observations and then use surprising outcomes to update those predictions. However, humans also have a sense of confidence that characterizes the precision of their predictions. Bayesian models use a confidence-weighting principle to regulate learning: for a given surprise, the update is smaller when the confidence about the prediction was higher. Prior behavioral evidence indicates that human learning adheres to this confidence-weighting principle. Here, we explored the human brain dynamics sub-tending the confidence-weighting of learning using magneto-encephalography (MEG). During our volatile probability learning task, subjects’ confidence reports conformed with Bayesian inference. MEG revealed several stimulus-evoked brain responses whose amplitude reflected surprise, and some of them were further shaped by confidence: surprise amplified the stimulus-evoked response whereas confidence dampened it. Confidence about predictions also modulated several aspects of the brain state: pupil-linked arousal and beta-range (15–30 Hz) oscillations. The brain state in turn modulated specific stimulus-evoked surprise responses following the confidence-weighting principle. Our results thus indicate that there exist, in the human brain, signals reflecting surprise that are dampened by confidence in a way that is appropriate for learning according to Bayesian inference. They also suggest a mechanism for confidence-weighted learning: confidence about predictions would modulate intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations.

Learning in a changing and uncertain world is difficult. In this context, facing a discrepancy between my current belief and new observations may reflect random fluctuations (e.g. my commute train is unexpectedly late, but it happens sometimes), if so, I should ignore this discrepancy and not change erratically my belief. However, this discrepancy could also denote a profound change (e.g. the train company changed and is less reliable), in this case, I should promptly revise my current belief. Human learning is adaptive: we change how much we learn from new observations, in particular, we promote flexibility when facing profound changes. A mathematical analysis of the problem shows that we should increase flexibility when the confidence about our current belief is low, which occurs when a change is suspected. Here, I show that human learners entertain rational confidence levels during the learning of changing probabilities. This confidence modulates intrinsic properties of the brain state (oscillatory activity and neuromodulation) which in turn amplifies or reduces, depending on whether confidence is low or high, the neural responses to discrepant observations. This confidence-weighting mechanism could underpin adaptive learning.

Noradrenergic Responsiveness Supports Selective Attention across the Adult Lifespan

Selectively attending to relevant information while blocking out distractors is crucial for goal-directed behavior, yet with advancing age, deficits emerge in attentional selectivity. Decrements in attention have been associated with altered noradrenergic activity in animals. However, research linking noradrenergic functioning to attention in aging humans is scarce, likely reflecting long-standing methodological challenges in noninvasive assessments. We studied whether age-related differences in the noradrenergic system predict differences in attention. We measured pupil dilation, a noninvasive marker of arousal-related norepinephrine (NE) release, while concurrently recording the EEG of male younger (N = 39; 25.2 ± 3.2 years) and older adults (N = 38; 70.6 ± 2.7 years). Arousal was modulated on a trial-by-trial basis using fear-conditioned (CS⁺) stimuli. During conditioning, pupil and EEG markers related to heightened arousal were identified. Afterward, in a dichotic listening task, participants were cued to direct attention to either the left or right ear while highly similar syllable pairs were presented simultaneously to both ears. During the dichotic listening task, presentation of fear-conditioned stimuli reinstated the acquired arousal response, as reflected in pupil and EEG α-β band responses. Critically, pupil dilation to CS⁺ was correlated with stronger EEG α-β desynchronization, suggesting a common dependence on NE release. On a behavioral level, stronger arousal reactions were associated with better attention. In particular, structural equation modeling revealed that the responsiveness of the NE system is associated with attention on a latent construct level, measured by several indicator tasks. Overall, our results suggest that the responsiveness of the NE system supports attention across the lifespan.SIGNIFICANCE STATEMENT In old age, the ability to selectively process relevant aspects of the environment fades. Animal research suggests that the neuromodulator norepinephrine helps to maintain selective attention. We tested younger and older adults across a variety of attention tasks. In addition, we used arousing stimuli to experimentally activate participants' noradrenergic system while recording pupillometry and EEG to infer its functional capacity. Older adults showed compromised attention and reduced noradrenergic responsiveness as indicated by interrelated pupil and EEG markers. Crucially, in both age groups, a more responsive noradrenergic system was strongly associated with attention. Our findings link animal and human studies on the neural underpinning of attention in aging and underscore the importance of the noradrenergic system in late-life cognition.

Interplay between components of pupil-linked phasic arousal and its role in driving behavioral choice in Go/No-Go perceptual decision-making

In decision-making tasks, neural circuits involved in different aspects of information processing may activate the central arousal system, likely through their interconnection with brainstem arousal nuclei, collectively contributing to the observed pupil-linked phasic arousal. However, the individual components of the phasic arousal associated with different elements of information processing and their effects on behavior remain little known. In this study, we used machine learning techniques to decompose pupil-linked phasic arousal evoked by different components of information processing in rats performing a Go/No-Go perceptual decision-making task. We found that phasic arousal evoked by stimulus encoding was larger for the Go stimulus than the No-Go stimulus. For each session, the separation between distributions of phasic arousal evoked by the Go and by the No-Go stimulus was predictive of perceptual performance. The separation between distributions of decision-formation-evoked arousal on correct and incorrect trials was correlated with decision criterion but not perceptual performance. When a Go stimulus was presented, the action of go was primarily determined by the phasic arousal evoked by stimulus encoding. On the contrary, when a No-Go stimulus was presented, the action of go was determined by phasic arousal elicited by both stimulus encoding and decision formation. Drift diffusion modeling revealed that the four model parameters were better accounted for when phasic arousal elicited by both stimulus encoding and decision formation was considered. These results suggest that the interplay between phasic arousal evoked by both stimulus encoding and decision formation has important functional consequences on forming behavioral choice in perceptual decision-making.

Measures of Listening Effort Are Multidimensional

OBJECTIVES

Listening effort can be defined as the cognitive resources required to perform a listening task. The literature on listening effort is as confusing as it is voluminous: measures of listening effort rarely correlate with each other and sometimes result in contradictory findings. Here, we directly compared simultaneously recorded multimodal measures of listening effort. After establishing the reliability of the measures, we investigated validity by quantifying correlations between measures and then grouping-related measures through factor analysis.

DESIGN

One hundred and sixteen participants with audiometric thresholds ranging from normal to severe hearing loss took part in the study (age range: 55 to 85 years old, 50.3% male). We simultaneously measured pupil size, electroencephalographic alpha power, skin conductance, and self-report listening effort. One self-report measure of fatigue was also included. The signal to noise ratio (SNR) was adjusted at 71% criterion performance using sequences of 3 digits. The main listening task involved correct recall of a random digit from a sequence of six presented at a SNR where performance was around 82 to 93%. Test-retest reliability of the measures was established by retesting 30 participants 7 days after the initial session.

RESULTS

With the exception of skin conductance and the self-report measure of fatigue, interclass correlation coefficients (ICC) revealed good test-retest reliability (minimum ICC: 0.71). Weak or nonsignificant correlations were identified between measures. Factor analysis, using only the reliable measures, revealed four underlying dimensions: factor 1 included SNR, hearing level, baseline alpha power, and performance accuracy; factor 2 included pupillometry; factor 3 included alpha power (during speech presentation and during retention); factor 4 included self-reported listening effort and baseline alpha power.

CONCLUSIONS

The good ICC suggests that poor test reliability is not the reason for the lack of correlation between measures. We have demonstrated that measures traditionally used as indicators of listening effort tap into multiple underlying dimensions. We therefore propose that there is no "gold standard" measure of listening effort and that different measures of listening effort should not be used interchangeably. When choosing method(s) to measure listening effort, the nature of the task and aspects of increased listening demands that are of interest should be taken into account. The findings of this study provide a framework for understanding and interpreting listening effort measures.

Acute effects of aerobic exercise on response variability and neuroelectric indices during a serial n-back task

To determine the neuroelectric underpinnings of exercise-induced changes in working memory, this study investigated the acute effects ofaerobic exercise (AE) on the P3 component of an event-related potential and brain oscillations during a serial n-back task. Task-related electroencephalography was collected in 23 young adults following 20 min of rest and AE on separate, counterbalanced days. The results revealed reductions in standard deviation of response time and coefficient of variation of response time following AE compared to rest. Neuroelectric analyses showed increased P3 amplitude following AE compared to rest. Task-related frontal alpha desynchronization was stronger in the 2-back compared with the 1-back task following AE, while no such modulation was observed following rest. These findings suggest AE may temporarily enhance working memory, as reflected by decreases in response variability, which are accompanied by neuroelectric indices reflecting greater upregulation of attentional processes.

Brain Networks Underlying Eye's Pupil Dynamics

Phasic changes in eye’s pupil diameter have been repeatedly observed during cognitive, emotional and behavioral activity in mammals. Although pupil diameter is known to be associated with noradrenergic firing in the pontine Locus Coeruleus (LC), thus far the causal chain coupling spontaneous pupil dynamics to specific cortical brain networks remains unknown. In the present study, we acquired steady-state blood oxygenation level-dependent (BOLD) functional magnetic resonance imaging (fMRI) data combined with eye-tracking pupillometry from fifteen healthy subjects that were trained to maintain a constant attentional load. Regression analysis revealed widespread visual and sensorimotor BOLD-fMRI deactivations correlated with pupil diameter. Furthermore, we found BOLD-fMRI activations correlated with pupil diameter change rate within a set of brain regions known to be implicated in selective attention, salience, error-detection and decision-making. These regions included LC, thalamus, posterior cingulate cortex (PCC), dorsal anterior cingulate and paracingulate cortex (dACC/PaCC), orbitofrontal cortex (OFC), and right anterior insular cortex (rAIC). Granger-causality analysis performed on these regions yielded a complex pattern of interdependence, wherein LC and pupil dynamics were far apart in the network and separated by several cortical stages. Functional connectivity (FC) analysis revealed the ubiquitous presence of the superior frontal gyrus (SFG) in the networks identified by the brain regions correlated to the pupil diameter change rate. No significant correlations were observed between pupil dynamics, regional activation and behavioral performance. Based on the involved brain regions, we speculate that pupil dynamics reflects brain processing implicated in changes between self- and environment-directed awareness.

No modulation of pupil size and event-related pupil response by transcutaneous auricular vagus nerve stimulation (taVNS)

Transcutaneous auricular vagus nerve stimulation (taVNS) bears therapeutic potential for a wide range of medical conditions. However, previous studies have found substantial interindividual variability in responsiveness to taVNS, and no reliable predictive biomarker for stimulation success has been developed so far. In this study, we investigate pupil size and event-related pupil response as candidate biomarkers. Both measures have a direct physiological link to the activity of the locus coeruleus (LC), a brainstem structure and the main source of norepinephrine in the brain. LC activation is considered one of the key mechanisms of action of taVNS, therefore, we expected a clear increase of the pupillary measures under taVNS compared to sham (placebo) stimulation, such that it could serve as a prospective predictor for individual clinical and physiological taVNS effects in future studies. We studied resting pupil size and pupillary responses to target stimuli in an auditory oddball task in 33 healthy young volunteers. We observed stronger pupil responses to target than to standard stimuli. However, and contrary to our hypothesis, neither pupil size nor the event-related pupil response nor behavioral performance were modulated by taVNS. We discuss potential explanations for this negative finding and its implications for future clinical investigation and development of taVNS.

Strength of socio-political attitudes moderates electrophysiological responses to perceptual anomalies

People with strong (vs. moderate) political attitudes have been shown to exhibit less phasic reactivity to perceptual anomalies, presumably to prevent their committed meaning systems from being challenged by novel experiences. Several researchers have proposed that (but not tested whether) firmly committed individuals also engage in more attentional suppression of anomalies, likely mediated by prestimulus alpha power. We expected participants with strong (vs. moderate) political attitudes to display increased pre-stimulus alpha power when processing perceptual anomalies. We recorded electrophysiological activity during the presentation of normal cards (control group) or both normal and anomalous playing cards (experimental group; total N = 191). In line with our predictions, the presence of anomalous playing cards in the stimulus set increased prestimulus alpha power only among individuals with strong but not moderate political attitudes. As potential markers of phasic reactivity, we also analyzed the late positive potential (LPP) and earlier components of the event-related potential, namely P1, N1, and P300. The moderating effect of extreme attitudes on ERP amplitudes remained inconclusive. Altogether, our findings support the idea that ideological conviction is related to increased tonic responses to perceptual anomalies.

Ready, set, explore! Event-related potentials reveal the time-course of exploratory decisions

The decision trade-off between exploiting the known and exploring the unknown has been studied using a variety of approaches and techniques. Surprisingly, electroencephalography (EEG) has been underused in this area of study, even though its high temporal resolution has the potential to reveal the time-course of exploratory decisions. We addressed this issue by recording EEG data while participants tried to win as many points as possible in a two-choice gambling task called a two-armed bandit. After using a computational model to classify responses as either exploitations or explorations, we examined event-related potentials locked to two events preceding decisions to exploit/explore: the arrival of feedback, and the subsequent appearance of the next trial's choice stimuli. In particular, we examined the feedback-locked P300 component, thought to index a phasic release of norepinephrine (a neural interrupt signal), and the reward positivity, thought to index a phasic release of dopamine (a neural prediction error signal). We observed an exploration-dependent enhancement of the P300 only, suggesting a critical role of norepinephrine (but not dopamine) in triggering decisions to explore. Similarly, we examined the N200/P300 components evoked by the appearance of the choice stimuli. In this case, exploration was characterized by an enhancement of the N200, but not P300, a result we attribute to increased response conflict. These results demonstrate the usefulness of combining computational and EEG methodologies, and suggest that exploratory decisions are preceded by two characterizing events: a feedback-locked neural interrupt signal (enhanced P300), and a choice-locked increase in response conflict (enhanced N200).

IR-camera-based measurements of 2D/3D cognitive fatigue in 2D/3D display system using task-evoked pupillary response

This study was carried out to evaluate a method used to measure three-dimensional (3D) cognitive fatigue based on the pupillary response. This technique was designed to overcome measurement burdens by using non-contact methods. The pupillary response is related to cognitive function by a neural pathway and may be an indicator of 3D cognitive fatigue. Twenty-six undergraduate students (including 14 women) watched both 2D and 3D versions of a video for 70 min. The participants experienced visual fatigue after viewing the 3D content. Measures such as subjective rating, response time, event-related potential latency, heartbeat-evoked potential (HEP) alpha power, and task-evoked pupillary response (TEPR) latency were significantly different. Multitrait-multimethod matrix analysis indicated that HEP and TEPR latency measures had stronger reliability and higher correlations with 3D cognitive fatigue than other measures. TEPR latency may be useful for quantitatively determining 3D visual fatigue, as it can be easily used to evaluate 3D visual fatigue using a non-contact method without measuring burden.

Behavioural and neural signatures of perceptual decision-making are modulated by pupil-linked arousal

The timing and accuracy of perceptual decision-making is exquisitely sensitive to fluctuations in arousal. Although extensive research has highlighted the role of various neural processing stages in forming decisions, our understanding of how arousal impacts these processes remains limited. Here we isolated electrophysiological signatures of decision-making alongside signals reflecting target selection, attentional engagement and motor output and examined their modulation as a function of tonic and phasic arousal, indexed by baseline and task-evoked pupil diameter, respectively. Reaction times were shorter on trials with lower tonic, and higher phasic arousal. Additionally, these two pupil measures were predictive of a unique set of EEG signatures that together represent multiple information processing steps of decision-making. Finally, behavioural variability associated with fluctuations in tonic and phasic arousal, indicative of neuromodulators acting on multiple timescales, was mediated by its effects on the EEG markers of attentional engagement, sensory processing and the variability in decision processing.

Driving along a busy street requires you to constantly monitor the behavior of other road users. You need to be able to spot and avoid the car that suddenly changes lane, or the pedestrian who steps out in front of you. How fast you can react to such events depends in part on your brain's level of alertness, or 'arousal'. This in turn depends on chemicals within the brain called neuromodulators.

Neuromodulators are a type of neurotransmitter. But whereas other neurotransmitters enable brain cells to signal to each other, neuromodulators turn the volume of these signals up or down. The activity of brain regions that produce neuromodulators varies over time, leading to changes in brain arousal. These changes take place over different time scales. Sudden unexpected events, such as those on the busy street above, trigger sub-second changes in arousal. But arousal levels also show spontaneous fluctuations over minutes to hours. We can follow these changes in real-time by looking into a participant’s eyes. This is because the brain regions that produce neuromodulators also control pupil size.

Van Kempen et al. have now combined measurements of pupil size with recordings of electrical brain activity. Healthy volunteers learned to press a button as soon as a target appeared on a screen. The larger a volunteer’s pupils were before the target appeared, the more slowly the volunteer responded on that trial. Large baseline pupil size is thought to indicate a high baseline level of brain arousal. By contrast, the larger the increase in pupil size in response to the target, the faster the volunteer responded on that trial. This increase in pupil size is thought to reflect an increase in brain arousal.

The recordings of brain activity provided clues to the underlying mechanisms. In trials with large baseline pupil size – and therefore high baseline arousal – the volunteers’ brains showed more variable responses to the target. But in trials with a large increase in pupil size – and a large increase in arousal – the volunteers’ brains showed less variable responses, as well as stronger signals related to attention.

Neuromodulators thus act on different timescales to influence different aspects of cognitive performance, including attention and target detection. Fluctuating levels of neuromodulator activity may help explain the variability in our behavior. Monitoring pupil size is one way to gain insights into the mechanisms that bring about these changes in neuromodulator activity.

The Pupil Dilation Response to Auditory Stimuli: Current State of Knowledge

The measurement of cognitive resource allocation during listening, or listening effort, provides valuable insight in the factors influencing auditory processing. In recent years, many studies inside and outside the field of hearing science have measured the pupil response evoked by auditory stimuli. The aim of the current review was to provide an exhaustive overview of these studies. The 146 studies included in this review originated from multiple domains, including hearing science and linguistics, but the review also covers research into motivation, memory, and emotion. The present review provides a unique overview of these studies and is organized according to the components of the Framework for Understanding Effortful Listening. A summary table presents the sample characteristics, an outline of the study design, stimuli, the pupil parameters analyzed, and the main findings of each study. The results indicate that the pupil response is sensitive to various task manipulations as well as interindividual differences. Many of the findings have been replicated. Frequent interactions between the independent factors affecting the pupil response have been reported, which indicates complex processes underlying cognitive resource allocation. This complexity should be taken into account in future studies that should focus more on interindividual differences, also including older participants. This review facilitates the careful design of new studies by indicating the factors that should be controlled for. In conclusion, measuring the pupil dilation response to auditory stimuli has been demonstrated to be sensitive method applicable to numerous research questions. The sensitivity of the measure calls for carefully designed stimuli.

A linear oscillator model predicts dynamic temporal attention and pupillary entrainment to rhythmic patterns

Rhythm is a ubiquitous feature of music that induces specific neural modes of processing. In this paper, we assess the potential of a stimulus-driven linear oscillator model (57) to predict dynamic attention to complex musical rhythms on an instant-by-instant basis. We use perceptual thresholds and pupillometry as attentional indices against which to test our model predictions. During a deviance detection task, participants listened to continuously looping, multiinstrument, rhythmic patterns, while being eye-tracked. Their task was to respond anytime they heard an increase in intensity (dB SPL). An adaptive thresholding algorithm adjusted deviant intensity at multiple probed temporal locations throughout each rhythmic stimulus. The oscillator model predicted participants’ perceptual thresholds for detecting deviants at probed locations, with a low temporal salience prediction corresponding to a high perceptual threshold and vice versa. A pupil dilation response was observed for all deviants. Notably, the pupil dilated even when participants did not report hearing a deviant. Maximum pupil size and resonator model output were significant predictors of whether a deviant was detected or missed on any given trial. Besides the evoked pupillary response to deviants, we also assessed the continuous pupillary signal to the rhythmic patterns. The pupil exhibited entrainment at prominent periodicities present in the stimuli and followed each of the different rhythmic patterns in a unique way. Overall, these results replicate previous studies using the linear oscillator model to predict dynamic attention to complex auditory scenes and extend the utility of the model to the prediction of neurophysiological signals, in this case the pupillary time course; however, we note that the amplitude envelope of the acoustic patterns may serve as a similarly useful predictor. To our knowledge, this is the first paper to show entrainment of pupil dynamics by demonstrating a phase relationship between musical stimuli and the pupillary signal.

Infrared Camera-Based Non-contact Measurement of Brain Activity From Pupillary Rhythms

Pupillary responses are associated with affective processing, cognitive function, perception, memory, attention, and other brain activities involving neural pathways. The present study aimed to develop a noncontact system to measure brain activity based on pupillary rhythms using an infra-red web camera. Electroencephalogram (EEG) signals and pupil imaging of 70 undergraduate volunteers (35 female, 35 male) were measured in response to sound stimuli designed to evoke arousal, relaxation, happiness, sadness, or neutral responses. This study successfully developed a real-time system that could detect an EEG spectral index (relative power: low beta in FP1; mid beta in FP1; SMR in FP1; beta in F3; high beta in F8; gamma P4; mu in C4) from pupillary rhythms using the synchronization phenomenon in harmonic frequency (1/100 f) between the pupil and brain oscillations. This method was effective in measuring and evaluating brain activity using a simple, low-cost, noncontact system, and may be an alternative to previous methods used to evaluate brain activity.