Modulation of the pupillary response by the content of visual working memory

Mechanisms of visual working memory processing task-irrelevant information retrieved from visual long-term memory

Visual working memory (VWM) can selectively filter task-irrelevant information from incoming visual stimuli. However, whether a similar filtering process applies to task-irrelevant information retrieved from visual long-term memory (VLTM) remains elusive. We assume a "resource-limited retrieval mechanism" in VWM in charge of the retrieval of irrelevant VLTM information. To make a comprehensive understanding of this mechanism, we conducted three experiments using both a VLTM learning task and a VWM task combined with pupillometry. The presence of a significant pupil light response (PLR) served as empirical evidence that VLTM information can indeed make its way into VWM. Notably, task-relevant VLTM information induced a sustained PLR, contrasting with the transient PLR observed for task-irrelevant VLTM information. Importantly, the transience of the PLR occurred under conditions of low VWM load, but this effect was absent under conditions of high load. Collectively, these results show that task-irrelevant VLTM information can enter VWM and then fade away only under conditions of low VWM load. This dynamic underscores the resource-limited retrieval mechanism within VWM, exerting control over the entry of VLTM information.

Internal coupling: Eye behavior coupled to visual imagery

Our eyes do not only respond to visual perception but also to internal cognition involving visual imagery, which can be referred to as internal coupling. This review synthesizes evidence on internal coupling across diverse domains including episodic memory and simulation, visuospatial memory, numerical cognition, object movement, body movement, and brightness imagery. In each domain, eye movements consistently reflect distinct aspects of mental imagery typically akin to those seen in corresponding visual experiences. Several findings further suggest that internal coupling may not only coincide with but also supports internal cognition as evidenced by improved cognitive performance. Available theoretical accounts suggest that internal coupling may serve at least two functional roles in visual imagery: facilitating memory reconstruction and indicating shifts in internal attention. Moreover, recent insights into the neurobiology of internal coupling highlight substantially shared neural pathways in externally and internally directed cognition. The review concludes by identifying open questions and promising avenues for future research such as exploring moderating roles of context and individual differences in internal coupling.

Enhanced Working Memory Representations for Rare Events

Rare events (oddballs) produce a variety of enhanced physiological responses relative to frequent events (standards), including the P3b component of the event-related potential (ERP) waveform. Previous research has suggested that the P3b is related to working memory, which implies that working memory representations will be enhanced for rare stimuli. To test this hypothesis, we devised a modified oddball paradigm in which the target was a disk presented at one of 16 different locations, which were divided into a rare set and a frequent set. Participants made a binary response on each trial to report whether the target appeared in the rare set or the frequent set. As expected, the P3b was much larger for stimuli appearing at a location within the rare set. We also included occasional probe trials in which the subject reported the exact location of the target. We found that these reports were more accurate for locations within the rare set than for locations within the frequent set. Moreover, the mean accuracy of these reports was correlated with the mean amplitude of the P3b. We also applied multivariate pattern analysis to the ERP data to "decode" the remembered location of the target. Decoding accuracy was greater for locations within the rare set than for locations within the frequent set. We then replicated and extended our behavioral findings in a follow-up experiment. These behavioral and electrophysiological results demonstrate that although both frequent and rare events are stored in working memory, the representations are enhanced for rare oddball events.

Functional benefits of cognitively driven pupil-size changes

Pupil-size changes are typically associated with the pupil light response (PLR), where they are driven by the physical entry of light into the eye. However, pupil-size changes are also influenced by various cognitive processes, where they are driven by higher-level cognition. For example, the strength of the PLR is not solely affected by physical properties of the light but also by cognitive factors, such as whether the source of light is attended or not, which results in an increase or decrease in the strength of the PLR. Surprisingly, although cognitively driven pupil-size changes have been the focus of extensive research, their possible functions are rarely discussed. Here we consider the relative (dis)advantages of small versus large pupils in different situations from a theoretical point of view, and compare these to empirical results showing how pupil size actually changes in these situations. Based on this, we suggest that cognitively driven pupil-size changes optimize vision either through preparation, embodied representations, or a differential emphasis on central or peripheral vision. More generally, we argue that cognitively driven pupil-size changes are a form of sensory tuning: a subtle adjustment of the eyes to optimize vision for the current situation and the immediate future. This article is categorized under: Neuroscience > Cognition Neuroscience > Physiology Neuroscience > Behavior.

Revealing visual working memory operations with pupillometry: Encoding, maintenance, and prioritization

Pupillary dynamics reflect effects of distinct and important operations of visual working memory: encoding, maintenance, and prioritization. Here, we review how pupil size predicts memory performance and how it provides novel insights into the mechanisms of each operation. Visual information must first be encoded into working memory with sufficient precision. The depth of this encoding process couples to arousal-linked baseline pupil size as well as a pupil constriction response before and after stimulus onset, respectively. Subsequently, the encoded information is maintained over time to ensure it is not lost. Pupil dilation reflects the effortful maintenance of information, wherein storing more items is accompanied by larger dilations. Lastly, the most task-relevant information is prioritized to guide upcoming behavior, which is reflected in yet another dilatory component. Moreover, activated content in memory can be pupillometrically probed directly by tagging visual information with distinct luminance levels. Through this luminance-tagging mechanism, pupil light responses reveal whether dark or bright items receive more attention during encoding and prioritization. Together, conceptualizing pupil responses as a sum of distinct components over time reveals insights into operations of visual working memory. From this viewpoint, pupillometry is a promising avenue to study the most vital operations through which visual working memory works. This article is categorized under: Psychology > Attention Psychology > Memory Psychology > Theory and Methods.

Decoupling of the pupillary light response during internal attention: The modulating effect of luminance intensity

In a world full of sensory stimuli, attention guides us between the external environment and our internal thoughts. While external attention involves processing sensory stimuli, internal attention is devoted to self-generated representations such as planning or spontaneous mind wandering. They both draw from common cognitive resources, thus simultaneous engagement in both often leads to interference between processes. In order to maintain internal focus, an attentional mechanism known as perceptual decoupling takes effect. This mechanism supports internal cognition by decoupling attention from the perception of sensory information. Two previous studies of our lab investigated to what extent perceptual decoupling is evident in voluntary eye movements. Findings showed that the effect is mediated by the internal task modality and workload (visuospatial > arithmetic and high > low, respectively). However, it remains unclear whether it extends to involuntary eye behavior, which may not share cognitive resources with internal activities. Therefore, the present experiment aimed to further elucidate attentional dynamics by examining whether internal attention affects the pupillary light response (PLR). Specifically, we consistently observed that workload and task modality of the internal task reduced the PLR to luminance changes of medium intensity. However, the PLR to strong luminance changes was less or not at all affected by the internal task. These results suggest that perceptual decoupling effects may be less consistent in involuntary eye behavior, particularly in the context of a salient visual stimulus.

The Intensity of Internal and External Attention Assessed with Pupillometry

Not only is visual attention shifted to objects in the external world, attention can also be directed to objects in memory. We have recently shown that pupil size indexes how strongly items are attended externally, which was reflected in more precise encoding into visual working memory. Using a retro-cuing paradigm, we here replicated this finding by showing that stronger pupil constrictions during encoding were reflective of the depth of encoding. Importantly, we extend this previous work by showing that pupil size also revealed the intensity of internal attention toward content stored in visual working memory. Specifically, pupil dilation during the prioritization of one among multiple internally stored representations predicted the precision of the prioritized item. Furthermore, the dynamics of the pupillary responses revealed that the intensity of internal and external attention independently determined the precision of internalized visual representations. Our results show that both internal and external attention are not all-or-none processes, but should rather be thought of as continuous resources that can be deployed at varying intensities. The employed pupillometric approach allows to unravel the intricate interplay between internal and external attention and their effects on visual working memory.

Introspective inference counteracts perceptual distortion

Introspective agents can recognize the extent to which their internal perceptual experiences deviate from the actual states of the external world. This ability, also known as insight, is critically required for reality testing and is impaired in psychosis, yet little is known about its cognitive underpinnings. We develop a Bayesian modeling framework and a psychophysics paradigm to quantitatively characterize this type of insight while people experience a motion after-effect illusion. People can incorporate knowledge about the illusion into their decisions when judging the actual direction of a motion stimulus, compensating for the illusion (and often overcompensating). Furthermore, confidence, reaction-time, and pupil-dilation data all show signatures consistent with inferential adjustments in the Bayesian insight model. Our results suggest that people can question the veracity of what they see by making insightful inferences that incorporate introspective knowledge about internal distortions.

Time for Memories

The ability to store information about the past to dynamically predict and prepare for the future is among the most fundamental tasks the brain performs. To date, the problems of understanding how the brain stores and organizes information about the past (memory) and how the brain represents and processes temporal information for adaptive behavior have generally been studied as distinct cognitive functions. This Symposium explores the inherent link between memory and temporal cognition, as well as the potential shared neural mechanisms between them. We suggest that working memory and implicit timing are interconnected and may share overlapping neural mechanisms. Additionally, we explore how temporal structure is encoded in associative and episodic memory and, conversely, the influences of episodic memory on subsequent temporal anticipation and the perception of time. We suggest that neural sequences provide a general computational motif that contributes to timing and working memory, as well as the spatiotemporal coding and recall of episodes.

Autogenous cerebral processes: an invitation to look at the brain from inside out

While external stimulation can reliably trigger neuronal activity, cerebral processes can operate independently from the environment. In this study, we conceptualize autogenous cerebral processes (ACPs) as intrinsic operations of the brain that exist on multiple scales and can influence or shape stimulus responses, behavior, homeostasis, and the physiological state of an organism. We further propose that the field should consider exploring to what extent perception, arousal, behavior, or movement, as well as other cognitive functions previously investigated mainly regarding their stimulus-response dynamics, are ACP-driven.

Altered pupil responses to social and non-social stimuli in Shank3 mutant dogs

Pupillary response, an important process in visual perception and social and emotional cognition, has been widely studied for understanding the neural mechanisms of neuropsychiatric disorders. However, there have been few studies on pupil response to social and non-social stimuli in animal models of neurodevelopmental disorders including autism spectrum disorder (ASD) and attention deficit hyperactivity disorder. Here, we developed a pupilometer using a robust eye feature-detection algorithm for real-time pupillometry in dogs. In a pilot study, we found that a brief light flash induced a less-pronounced and slower pupil dilation response in gene-edited dogs carrying mutations in Shank3; mutations of its ortholog in humans were repeatedly identified in ASD patients. We further found that obnoxious, loud firecracker sound of 120 dB induced a stronger and longer pupil dilation response in Shank3 mutant dogs, whereas a high reward food induced a weaker pupillary response in Shank3 mutants than in wild-type control dogs. In addition, we found that Shank3 mutants showed compromised pupillary synchrony during dog-human interaction. These findings of altered pupil response in Shank3 mutant dogs recapitulate the altered sensory responses in ASD patients. Thus, this study demonstrates the validity and value of the pupilometer for dogs, and provides an effective paradigm for studying the underlying neural mechanisms of ASD and potentially other psychiatric disorders.

Methods in cognitive pupillometry: Design, preprocessing, and statistical analysis

Cognitive pupillometry is the measurement of pupil size to investigate cognitive processes such as attention, mental effort, working memory, and many others. Currently, there is no commonly agreed-upon methodology for conducting cognitive-pupillometry experiments, and approaches vary widely between research groups and even between different experiments from the same group. This lack of consensus makes it difficult to know which factors to consider when conducting a cognitive-pupillometry experiment. Here we provide a comprehensive, hands-on guide to methods in cognitive pupillometry, with a focus on trial-based experiments in which the measure of interest is the task-evoked pupil response to a stimulus. We cover all methodological aspects of cognitive pupillometry: experimental design, preprocessing of pupil-size data, and statistical techniques to deal with multiple comparisons when testing pupil-size data. In addition, we provide code and toolboxes (in Python) for preprocessing and statistical analysis, and we illustrate all aspects of the proposed workflow through an example experiment and example scripts.

Differential aspects of attention predict the depth of visual working memory encoding: Evidence from pupillometry

What determines how much one encodes into visual working memory? Traditionally, encoding depth is considered to be indexed by spatiotemporal properties of gaze, such as gaze position and dwell time. Although these properties inform about where and how long one looks, they do not necessarily inform about the current arousal state or how strongly attention is deployed to facilitate encoding. Here, we found that two types of pupillary dynamics predict how much information is encoded during a copy task. The task involved encoding a spatial pattern of multiple items for later reproduction. Results showed that smaller baseline pupil sizes preceding and stronger pupil orienting responses during encoding predicted that more information was encoded into visual working memory. Additionally, we show that pupil size reflects not only how much but also how precisely material is encoded. We argue that a smaller pupil size preceding encoding is related to increased exploitation, whereas larger pupil constrictions signal stronger attentional (re)orienting to the to-be-encoded pattern. Our findings support the notion that the depth of visual working memory encoding is the integrative outcome of differential aspects of attention: how alert one is, how much attention one deploys, and how long it is deployed. Together, these factors determine how much information is encoded into visual working memory.

Attention in flux

Selective attention comprises essential infrastructural functions supporting cognition-anticipating, prioritizing, selecting, routing, integrating, and preparing signals to guide adaptive behavior. Most studies have examined its consequences, systems, and mechanisms in a static way, but attention is at the confluence of multiple sources of flux. The world advances, we operate within it, our minds change, and all resulting signals progress through multiple pathways within the dynamic networks of our brains. Our aim in this review is to raise awareness of and interest in three important facets of how timing impacts our understanding of attention. These include the challenges posed to attention by the timing of neural processing and psychological functions, the opportunities conferred to attention by various temporal structures in the environment, and how tracking the time courses of neural and behavioral modulations with continuous measures yields surprising insights into the workings and principles of attention.

Examining the role of attentional allocation in working memory precision with pupillometry in children and adults

Working memory (WM) precision, or the fidelity with which items can be remembered, is an important aspect of WM capacity that increases over childhood. Why individuals are more or less precise from moment to moment and why WM becomes more stable with age are not yet fully understood. Here, we examined the role of attentional allocation in visual WM precision in children aged 8 to 13 years and young adults aged 18 to 27 years, as measured by fluctuations in pupil dilation during stimulus encoding and maintenance. Using mixed models, we examined intraindividual links between change in pupil diameter and WM precision across trials and the role of developmental differences in these associations. Through probabilistic modeling of error distributions and the inclusion of a visuomotor control task, we isolated mnemonic precision from other cognitive processes. We found an age-related increase in mnemonic precision that was independent of guessing behavior, serial position effects, fatigue or loss of motivation across the experiment, and visuomotor processes. Trial-by-trial analyses showed that trials with smaller changes in pupil diameter during encoding and maintenance predicted more precise responses than trials with larger changes in pupil diameter within individuals. At encoding, this relationship was stronger for older participants. Furthermore, the pupil-performance coupling grew across the delay period-particularly or exclusively for adults. These results suggest a functional link between pupil fluctuations and WM precision that grows over development; visual details may be stored more faithfully when attention is allocated efficiently to a sequence of objects at encoding and throughout a delay period.

Turning Attention Inside Out: How Working Memory Serves Behavior

Flexible behavior requires guidance not only by sensations that are available immediately but also by relevant mental contents carried forward through working memory. Therefore, selective-attention functions that modulate the contents of working memory to guide behavior (inside-out) are just as important as those operating on sensory signals to generate internal contents (outside-in). We review the burgeoning literature on selective attention in the inside-out direction and underscore its functional, flexible, and future-focused nature. We discuss in turn the purpose (why), targets (what), sources (when), and mechanisms (how) of selective attention inside working memory, using visual working memory as a model. We show how the study of internal selective attention brings new insights concerning the core cognitive processes of attention and working memory and how considering selective attention and working memory together paves the way for a rich and integrated understanding of how mind serves behavior.

Opening Questions in Visual Working Memory

In this reflective piece on visual working memory, I depart from the laboriously honed skills of writing a review. Instead of integrating approaches, synthesizing evidence, and building a cohesive perspective, I scratch my head and share niggles and puzzlements. I expose where my scholarship and understanding are stumped by findings and standard views in the literature.

Quantitative pupillometry and radiographic markers of intracranial midline shift: A pilot study

Background

Asymmetric pupil reactivity or size can be early clinical indicators of midbrain compression due to supratentorial ischemic stroke or primary intraparenchymal hemorrhage (IPH). Radiographic midline shift is associated with worse functional outcomes and life-saving interventions. Better understanding of quantitative pupil characteristics would be a non-invasive, safe, and cost-effective way to improve identification of life-threatening mass effect and resource utilization of emergent radiographic imaging. We aimed to better characterize the association between midline shift at various anatomic levels and quantitative pupil characteristics.

Methods

We conducted a multicenter retrospective study of brain CT images within 75 min of a quantitative pupil observation from patients admitted to Neuro-ICUs between 2016 and 2020 with large (>1/3 of the middle cerebral artery territory) acute supratentorial ischemic stroke or primary IPH > 30 mm³. For each image, we measured midline shift at the septum pellucidum (MLS-SP), pineal gland shift (PGS), the ratio of the ipsilateral to contralateral midbrain width (IMW/CMW), and other exploratory markers of radiographic shift/compression. Pupil reactivity was measured using an automated infrared pupillometer (NeurOptics^®, Inc.), specifically the proprietary algorithm for Neurological Pupil Index^® (NPi). We used rank-normalization and linear mixed-effects models, stratified by diagnosis and hemorrhagic conversion, to test associations of radiographic markers of shift and asymmetric pupil reactivity (Diff NPi), adjusting for age, lesion volume, Glasgow Coma Scale, and osmotic medications.

Results

Of 53 patients with 74 CT images, 26 (49.1%) were female, and median age was 67 years. MLS-SP and PGS were greater in patients with IPH, compared to patients with ischemic stroke (6.2 v. 4.0 mm, 5.6 v. 3.4 mm, respectively). We found no significant associations between pupil reactivity and the radiographic markers of shift when adjusting for confounders. However, we found potentially relevant relationships between MLS-SP and Diff NPi in our IPH cohort (β = 0.11, SE 0.04, P = 0.01), and PGS and Diff NPi in the ischemic stroke cohort (β = 0.16, SE 0.09, P = 0.07).

Conclusion

We found the relationship between midline shift and asymmetric pupil reactivity may differ between IPH and ischemic stroke. Our study may serve as necessary preliminary data to guide further prospective investigation into how clinical manifestations of radiographic midline shift differ by diagnosis and proximity to the midbrain.

Pupillometry signatures of sustained attention and working memory

There exists an intricate relationship between attention and working memory. Recent work has further established that attention and working memory fluctuate synchronously, by tightly interleaving sustained attention and working memory tasks. This work has raised many open questions about physiological signatures underlying these behavioral fluctuations. Across two experiments, we explore pupil dynamics using real-time triggering in conjunction with an interleaved sustained attention and working memory task. In Experiment 1, we use behavioral real-time triggering and replicate recent findings from our lab (deBettencourt et al., 2019) that sustained attention fluctuates concurrently with the number of items maintained in working memory. Furthermore, highly attentive moments, detected via behavior, also exhibited larger pupil sizes. In Experiment 2, we develop a novel real-time pupil-triggering technique to track pupil size fluctuations in real time and trigger working memory probes. We show that this pupil triggering procedure reveals differences in sustained attention, as indexed by response time. These experiments reflect methodological advances in real-time triggering and further disentangle the relationship among general arousal, sustained attention, and working memory.

The eyes reflect an internal cognitive state hidden in the population activity of cortical neurons

Decades of research have shown that global brain states such as arousal can be indexed by measuring the properties of the eyes. The spiking responses of neurons throughout the brain have been associated with the pupil, small fixational saccades, and vigor in eye movements, but it has been difficult to isolate how internal states affect the eyes, and vice versa. While recording from populations of neurons in the visual and prefrontal cortex (PFC), we recently identified a latent dimension of neural activity called "slow drift," which appears to reflect a shift in a global brain state. Here, we asked if slow drift is correlated with the action of the eyes in distinct behavioral tasks. We recorded from visual cortex (V4) while monkeys performed a change detection task, and PFC, while they performed a memory-guided saccade task. In both tasks, slow drift was associated with the size of the pupil and the microsaccade rate, two external indicators of the internal state of the animal. These results show that metrics related to the action of the eyes are associated with a dominant and task-independent mode of neural activity that can be accessed in the population activity of neurons across the cortex.

Pupil size variations reveal covert shifts of attention induced by numbers

The pupil light response is more than a pure reflexive mechanism that reacts to the amount of light entering the eye. The pupil size may also react to the luminance of objects lying in the visual periphery, revealing the locus of covert attention. In the present study, we took advantage of this response to study the spatial coding of abstract concepts with no physical counterpart: numbers. The participants' gaze was maintained fixed in the middle of a screen whose left and right parts were dark or bright, and variations in pupil size were recorded during an auditory number comparison task. The results showed that small numbers accentuated pupil dilation when the darker part of the screen was on the left, while large numbers accentuated pupil dilation when the darker part of the screen was on the right. This finding provides direct evidence for covert attention shifts on a left-to-right oriented mental spatial representation of numbers. From a more general perspective, it shows that the pupillary response to light is subject to modulation from spatial attention mechanisms operating on mental contents.

The pupillary light response as a physiological index of aphantasia, sensory and phenomenological imagery strength

The pupillary light response is an important automatic physiological response which optimizes light reaching the retina. Recent work has shown that the pupil also adjusts in response to illusory brightness and a range of cognitive functions, however, it remains unclear what exactly drives these endogenous changes. Here, we show that the imagery pupillary light response correlates with objective measures of sensory imagery strength. Further, the trial-by-trial phenomenological vividness of visual imagery is tracked by the imagery pupillary light response. We also demonstrated that a group of individuals without visual imagery (aphantasia) do not show any significant evidence of an imagery pupillary light response, however they do show perceptual pupil light responses and pupil dilation with larger cognitive load. Our results provide evidence that the pupillary light response indexes the sensory strength of visual imagery. This work also provides the first physiological validation of aphantasia.

Alpha EEG Activity and Pupil Diameter Coupling during Inactive Wakefulness in Humans

Variations in human behavior correspond to the adaptation of the nervous system to different internal and environmental demands. Attention, a cognitive process for weighing environmental demands, changes over time. Pupillary activity, which is affected by fluctuating levels of cognitive processing, appears to identify neural dynamics that relate to different states of attention. In mice, for example, pupil dynamics directly correlate with brain state fluctuations. Although, in humans, alpha-band activity is associated with inhibitory processes in cortical networks during visual processing, and its amplitude is modulated by attention, conclusive evidence linking this narrowband activity to pupil changes in time remains sparse. We hypothesize that, as alpha activity and pupil diameter indicate attentional variations over time, these two measures should be comodulated. In this work, we recorded the electroencephalographic (EEG) and pupillary activity of 16 human subjects who had their eyes fixed on a gray screen for 1 min. Our study revealed that the alpha-band amplitude and the high-frequency component of the pupil diameter covariate spontaneously. Specifically, the maximum alpha-band amplitude was observed to occur ∼300 ms before the peak of the pupil diameter. In contrast, the minimum alpha-band amplitude was noted to occur ∼350 ms before the trough of the pupil diameter. The consistent temporal coincidence of these two measurements strongly suggests that the subject's state of attention, as indicated by the EEG alpha amplitude, is changing moment to moment and can be monitored by measuring EEG together with the diameter pupil.

Eyes wide open: Regulation of arousal by temporal expectations

Maintaining adequate levels of arousal is essential for sustaining performance on extended tasks. To investigate arousal in prolonged tasks such as driving studies have traditionally used monotonous task designs. Both ecological and experimental settings often contain embedded temporal regularities, but it is unknown whether these enable adaptive modulation of arousal. We explored whether temporal predictability can modulate arousal according to the timing of anticipated relevant events. In two experiments, we manipulated the temporal predictability of events to test for behavioural benefits and arousal modulation, using pupillometry as a proxy measure. High temporal predictability significantly lowered the tonic level of arousal briefly increased arousal in anticipation of upcoming stimuli, whereas low temporal predictability resulted in tonically elevated arousal. These novel findings suggest that arousal levels flexibly adapt to the temporal structures of events and bring about energy efficiencies in the context of high levels of behavioural performance.

Interactions Between Visual Working Memory, Attention, and Color Categories: A Pupillometry Study

Recent studies have found that visual working memory (VWM) for color shows a categorical bias: observers typically remember colors as more prototypical to the category they belong to than they actually are. Here, we further examine color-category effects on VWM using pupillometry. Participants remembered a color for later reproduction on a color wheel. During the retention interval, a colored probe was presented, and we measured the pupil constriction in response to this probe, assuming that the strength of constriction reflects the visual saliency of the probe. We found that the pupil initially constricted most strongly for non-matching colors that were maximally different from the memorized color; this likely reflects a lack of visual adaptation for these colors, which renders them more salient than memory-matching colors (which were shown before). Strikingly, this effect reversed later in time, such that pupil constriction was more prolonged for memory-matching colors as compared to non-matching colors; this likely reflects that memory-matching colors capture attention more strongly, and perhaps for a longer time, than non-matching colors do. We found no effects of color categories on pupil constriction: after controlling for color distance, (non-matching) colors from the same category as the memory color did not result in a different pupil response as compared to colors from a different category; however, we did find that behavioral responses were biased by color categories. In summary, we found that pupil constriction to colored probes reflects both visual adaptation and VWM content, but, unlike behavioral measures, is not notably affected by color categories.

Shielding working-memory representations from temporally predictable external interference

Protecting working-memory content from distracting external sensory inputs and intervening tasks is an ubiquitous demand in daily life. Here, we ask whether and how temporal expectations about external events can help mitigate effects of such interference during working-memory retention. We manipulated the temporal predictability of interfering items that occurred during the retention period of a visual working-memory task and report that temporal expectations reduce the detrimental influence of external interference on subsequent memory performance. Moreover, to determine if the protective effects of temporal expectations rely on distractor suppression or involve shielding of internal representations, we compared effects after irrelevant distractors that could be ignored vs. interrupters that required a response. Whereas distractor suppression may be sufficient to confer protection from predictable distractors, any benefits after interruption are likely to involve memory shielding. We found similar benefits of temporal expectations after both types of interference. We conclude that temporal expectations may play an important role in safeguarding behaviour based on working memory - acting through mechanisms that include the shielding of internal content from external interference.

Pupillometry tracks cognitive load and salience network activity in a working memory functional magnetic resonance imaging task

The diameter of the human pupil tracks working memory processing and is associated with activity in the frontoparietal network. At the same time, recent neuroimaging research has linked human pupil fluctuations to activity in the salience network. In this combined functional magnetic resonance imaging (fMRI)/pupillometry study, we recorded the pupil size of healthy human participants while they performed a blockwise organized working memory task (N‐back) inside an MRI scanner in order to monitor the pupil fluctuations associated neural activity during working memory processing. We first confirmed that mean pupil size closely followed working memory load. Combining this with fMRI data, we focused on blood oxygen level dependent (BOLD) correlates of mean pupil size modeled onto the task blocks as a parametric modulation. Interrogating this modulated task regressor, we were able to retrieve the frontoparietal network. Next, to fully exploit the within‐block dynamics, we divided the blocks into 1 s time bins and filled these with corresponding pupil change values (first‐order derivative of pupil size). We found that pupil change within N‐back blocks was positively correlated with BOLD amplitudes in the areas of the salience network (namely bilateral insula, and anterior cingulate cortex). Taken together, fMRI with simultaneous measurement of pupil parameters constitutes a valuable tool to dissect working memory subprocesses related to both working memory load and salience of the presented stimuli.

Anticipatory Energization Revealed by Pupil and Brain Activity Guides Human Effort-Based Decision Making

An organism's fitness is determined by how it chooses to adapt to effort in response to challenges. Exertion of effort correlates with activity in dorsomedial prefrontal cortex (dmPFC) and noradrenergic pupil dilation, but little is known about the role of these neurophysiological processes for decisions about future efforts, they may provide anticipatory energization to help us accept the challenge or a cost representation that is weighted against the expected rewards. Here, we provide evidence for the former, by measuring pupil and functional magnetic resonance imaging (fMRI) brain responses while 52 human participants (29 females) chose whether to exert efforts to obtain rewards. Both pupil-dilation rate and dmPFC fMRI activity increased with anticipated effort level, and these increases differ depending on the choice outcome: they were stronger when participants chose to accept the challenge compared with when the challenge was declined. Crucially, the choice-dependent modulation of pupil and brain-activity effort representations were stronger in participants whose behavioral choices were more sensitive to effort. Our results identify a process involving the peripheral and central human nervous system that simulates the required energization before overt response, suggesting a role in guiding effort-based decisions.SIGNIFICANCE STATEMENT The brain's arousal system tracks the effort we engage in during strenuous activity. But much less is known about what role this effort signaling may play when we decide whether to exert effort in the future. Here, we characterize pupil-linked arousal and brain signals that guide decisions whether to engage in effort to gain money. During such choices, increases in brain activity and pupil dilation correlated with the effort involved in the chosen option, and these increases were stronger when people decided to accept the effort compared with when they rejected it. These results suggest that the brain arousal system guides decisions by energizing the organism for the prospective challenge.

Temporal attention selectively enhances target features

Temporal attention, the allocation of attention to a moment in time, improves perception. Here, we examined the computational mechanism by which temporal attention improves perception, under a divisive normalization framework. Under this framework, attention can improve perception of a target signal in three ways: stimulus enhancement (increasing gain across all sensory channels), signal enhancement (selectively increasing gain in channels that encode the target stimulus), or external noise exclusion (reducing the gain in channels that encode irrelevant features). These mechanisms make diverging predictions when a target is embedded in varying levels of noise: stimulus enhancement improves performance only when noise is low, signal enhancement improves performance at all noise intensities, and external noise exclusion improves performance only when noise is high. To date, temporal attention studies have used noise-free displays. Therefore, it is unclear whether temporal attention acts via stimulus enhancement (amplifying both target features and noise) or signal enhancement (selectively amplifying target features) because both mechanisms predict improved performance in the absence of noise. To tease these mechanisms apart, we manipulated temporal attention using an auditory cue while parametrically varying external noise in a fine-orientation discrimination task. Temporal attention improved perceptual thresholds across all noise levels. Formal model comparisons revealed that this cuing effect was best accounted for by a combination of signal enhancement and stimulus enhancement, suggesting that temporal attention improves perceptual performance, in part, by selectively increasing gain for target features.

Attractiveness in the Eyes: A Possibility of Positive Loop between Transient Pupil Constriction and Facial Attraction

Contrary to the long-held belief of a close linkage between pupil dilation and attractiveness, we found an early and transient pupil constriction response when participants viewed an attractive face (and the effect of luminance/contrast was controlled). While human participants were making an attractiveness judgment on faces, their pupil constricted more for the more attractive (as-to-be-rated) faces. Further experiments showed that the effect of pupil constriction to attractiveness judgment extended to intrinsically esthetic visual objects such as natural scene images (as well as faces) but not to line-drawing geometric figures. When participants were asked to judge the roundness of faces, pupil constriction still correlated with their attractiveness but not the roundness rating score, indicating the automaticity of the pupil constriction to attractiveness. When pupillary responses were manipulated implicitly by relative background luminance changes (from the prestimulus screen), the facial attractiveness ratings were in accordance with the amount of pupil constriction, which could not be explained solely by simultaneous or sequential luminance contrast. The overall results suggest that pupil constriction not only reflects but, as a part of self-monitoring and attribution mechanisms, also possibly contributes to facial attractiveness implicitly.

Prioritized verbal working memory content biases ongoing action

Working memory (WM) holds information temporarily in mind, imparting the ability to guide behavior based on internal goals rather than external stimuli. However, humans often maintain WM content for a future task while performing more immediate actions. Consequently, transient WM representations may inadvertently influence ongoing (but unrelated) motor behavior. Here, we tested the impact of WM on adult human action execution and examined how the attentional or "activation" state of WM content modulates that impact. In 3 dual-task experiments, verbal WM for directional words influenced the trajectory and speed of hand movements performed during WM maintenance. This movement bias was also modulated by the attentional state of the WM content. Prioritized WM content strongly influenced actions during WM maintenance, while de-prioritized WM content was less influential. In summary, WM can unintentionally shape ongoing motor behavior, but the behavioral relevance of WM content determines the degree of influence on motor output. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Temporal Expectations Prepare Visual Working Memory for Behavior

Working memory enables us to retain past sensations in service of anticipated task demands. How we prepare for anticipated task demands during working memory retention remains poorly understood. Here, we focused on the role of time—asking how temporal expectations help prepare for ensuing memory-guided behavior. We manipulated the expected probe time in a delayed change-detection task and report that temporal expectation can have a profound influence on memory-guided behavioral performance. EEG measurements corroborated the utilization of temporal expectations: demonstrating the involvement of a classic EEG signature of temporal expectation—the contingent negative variation—in the context of working memory. We also report the influence of temporal expectations on 2 EEG signatures associated with visual working memory—the lateralization of 8- to 12-Hz alpha activity, and the contralateral delay activity. We observed a dissociation between these signatures, whereby alpha lateralization (but not the contralateral delay activity) adapted to the time of expected memory utilization. These data show how temporal expectations prepare visual working memory for behavior and shed new light on the electrophysiological markers of both temporal expectation and working memory.

Pupil Size as a Window on Neural Substrates of Cognition

Cognitively driven pupil modulations reflect certain underlying brain functions. What do these reflections tell us? Here, we review findings that have identified key roles for three neural systems: cortical modulation of the pretectal olivary nucleus (PON), which controls the pupillary light reflex; the superior colliculus (SC), which mediates orienting responses, including pupil changes to salient stimuli; and the locus coeruleus (LC)-norepinephrine (NE) neuromodulatory system, which mediates relationships between pupil-linked arousal and cognition. We discuss how these findings can inform the interpretation of pupil measurements in terms of activation of these neural systems. We also highlight caveats, open questions, and key directions for future experiments for improving these interpretations in terms of the underlying neural dynamics throughout the brain.

Fluctuation in Pupil Size and Spontaneous Blinks Reflect Story Transportation

Thirty-nine participants listened to 28 neutral and horror excerpts of Stephen King short stories while constantly tracking their emotional arousal. Pupil size was measured with an Eyelink 1000+, and participants rated valence and transportation after each story. In addition to computing mean pupil size across 1-sec intervals, we extracted blink count and used detrended fluctuation analysis (DFA) to obtain the scaling exponents of long-range temporal correlations (LRTCs) in pupil size time-series. Pupil size was expected to be sensitive also to emotional arousal, whereas blink count and LRTC's were expected to reflect cognitive engagement. The results showed that self-reported arousal increased, pupil size was overall greater, and the decreasing slope of pupil size was flatter for horror than for neutral stories. Horror stories induced higher transportation than neutral stories. High transportation was associated with a steeper increase in self-reported arousal across time, stronger LRTCs in pupil size fluctuations, and lower blink count. These results indicate that pupil size reflects emotional arousal induced by the text content, while LRTCs and blink count are sensitive to cognitive engagement associated with transportation, irrespective of the text type. The study demonstrates the utility of pupillometric measures and blink count to study literature reception.

Tuning the Senses: How the Pupil Shapes Vision at the Earliest Stage

The pupil responds reflexively to changes in brightness and focal distance to maintain the smallest pupil (and thus the highest visual acuity) that still allows sufficient light to reach the retina. The pupil also responds to a wide variety of cognitive processes, but the functions of these cognitive responses are still poorly understood. In this review, I propose that cognitive pupil responses, like their reflexive counterparts, serve to optimize vision. Specifically, an emphasis on central vision over peripheral vision results in pupil constriction, and this likely reflects the fact that central vision benefits most from the increased visual acuity provided by small pupils. Furthermore, an intention to act with a bright stimulus results in preparatory pupil constriction, which allows the pupil to respond quickly when that bright stimulus is subsequently brought into view. More generally, cognitively driven pupil responses are likely a form of sensory tuning: a subtle adjustment of the eyes to optimize their properties for the current situation and the immediate future. Expected final online publication date for the Annual Review of Vision Science, Volume 6 is September 15, 2020. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Visual working memory and action: Functional links and bi-directional influences

Working memory bridges perception to action over extended delays, enabling flexible goal-directed behaviour. To date, studies of visual working memory – concerned with detailed visual representations such as shape and colour – have considered visual memory predominantly in the context of visual task demands, such as visual identification and search. Another key purpose of visual working memory is to directly inform and guide upcoming actions. Taking this as a starting point, I review emerging evidence for the pervasive bi-directional links between visual working memory and (planned) action, and discuss these links from the perspective of their common goal of enabling flexible and precise behaviour.

Under the Mind's Hood: What We Have Learned by Watching the Brain at Work

Imagine you were asked to investigate the workings of an engine, but to do so without ever opening the hood. Now imagine the engine fueled the human mind. This is the challenge faced by cognitive neuroscientists worldwide aiming to understand the neural bases of our psychological functions. Luckily, human ingenuity comes to the rescue. Around the same time as the Society for Neuroscience was being established in the 1960s, the first tools for measuring the human brain at work were becoming available. Noninvasive human brain imaging and neurophysiology have continued developing at a relentless pace ever since. In this 50 year anniversary, we reflect on how these methods have been changing our understanding of how brain supports mind.