Costs of storing colour and complex shape in visual working memory: Insights from pupil size and slow waves

Pain-induced effects on the pupillary light response under high and low illumination conditions

Objective

The present study investigated the impact of two different light intensities on the pain-modulated pupillary light response (PLR). Additionally, it aimed to demonstrate parasympathetic and sympathetic influences on PLR parameters in response to pain, as predicted by functional models.

Method

A total of 24 participants were included in a randomized, repeated-measures design. The PLR was measured in response to both dark and bright light stimuli within two test cycles. Pain was induced using the cold pressor test (CPT), which involved immersing participants' feet in ice water. PLR measurements were taken during baseline and ice-water immersion within each test cycle. The assessed PLR parameters included initial diameter (INIT), latency (LAT), amplitude (AMP), and re-dilation time (ReDIL25). Along with these parameters, heart rate (HR) and pain ratings were also computed and analyzed.

Main results

The CPT caused moderate pain in participants, and the resulting PLR parameters were found to be congruent with the expected parasympathetic and sympathetic nervous system activities. Although the luminance of the stimulus did influence PLR parameters, no interaction with pain exposure was found.

Significance

The results showed that different aspects of pain experienced by an individual, as modulated through the sympathetic and parasympathetic nervous systems, are visible in their pupillary reactions to light. Notably, within the range used in the current study, light intensity did not significantly affect the pain-related PLR effects.

Revealing visual working memory operations with pupillometry: Encoding, maintenance, and prioritization

Pupillary dynamics reflect effects of distinct and important operations of visual working memory: encoding, maintenance, and prioritization. Here, we review how pupil size predicts memory performance and how it provides novel insights into the mechanisms of each operation. Visual information must first be encoded into working memory with sufficient precision. The depth of this encoding process couples to arousal-linked baseline pupil size as well as a pupil constriction response before and after stimulus onset, respectively. Subsequently, the encoded information is maintained over time to ensure it is not lost. Pupil dilation reflects the effortful maintenance of information, wherein storing more items is accompanied by larger dilations. Lastly, the most task-relevant information is prioritized to guide upcoming behavior, which is reflected in yet another dilatory component. Moreover, activated content in memory can be pupillometrically probed directly by tagging visual information with distinct luminance levels. Through this luminance-tagging mechanism, pupil light responses reveal whether dark or bright items receive more attention during encoding and prioritization. Together, conceptualizing pupil responses as a sum of distinct components over time reveals insights into operations of visual working memory. From this viewpoint, pupillometry is a promising avenue to study the most vital operations through which visual working memory works. This article is categorized under: Psychology > Attention Psychology > Memory Psychology > Theory and Methods.

Processing symmetry between visual and auditory spatial representations in updating working memory

Updating spatial representations in visual and auditory working memory relies on common processes, and the modalities should compete for attentional resources. If competition occurs, one type of spatial information is presumably weighted over the other, irrespective of sensory modality. This study used incompatible spatial information conveyed from two different cue modalities to examine relative dominance in memory updating. Participants mentally manoeuvred a designated target in a matrix according to visual or auditory stimuli that were presented simultaneously, to identify a terminal location. Prior to the navigation task, the relative perceptual saliences of the visual cues were manipulated to be equal, superior, or inferior to the auditory cues. The results demonstrate that visual and auditory information competed for attentional resources, such that visual/auditory guidance was impaired by incongruent cues delivered from the other modality. Although visual bias was generally observed in working-memory navigation, stimuli of relatively high salience interfered with or facilitated other stimuli regardless of modality, demonstrating the processing symmetry of spatial updating in visual and auditory spatial working memory. Furthermore, this processing symmetry can be identified during the encoding of sensory inputs into working-memory representations. The results imply that auditory spatial updating is comparable to visual spatial updating in that salient stimuli receive a high priority when selecting inputs and are used when tracking spatial representations.

EEG and pupillometric signatures of working memory overload

Understanding the physiological correlates of cognitive overload has implications for gauging the limits of human cognition, developing novel methods to define cognitive overload, and mitigating the negative outcomes associated with overload. Most previous psychophysiological studies manipulated verbal working memory load in a narrow range (an average load of 5 items). It is unclear, however, how the nervous system responds to a working memory load exceeding typical capacity limits. The objective of the current study was to characterize the central and autonomic nervous system changes associated with memory overload, by means of combined recording of electroencephalogram (EEG) and pupillometry. Eighty-six participants were presented with a digit span task involving the serial auditory presentation of items. Each trial consisted of sequences of either 5, 9, or 13 digits, each separated by 2 s. Both theta activity and pupil size, after the initial rise, expressed a pattern of a short plateau and a decrease with reaching the state of memory overload, indicating that pupil size and theta possibly have similar neural mechanisms. Based on the described above triphasic pattern of pupil size temporal dynamics, we concluded that cognitive overload causes physiological systems to reset, and release effort. Although memory capacity limits were exceeded and effort was released (as indicated by pupil dilation), alpha continued to decrease with increasing memory load. These results suggest that associating alpha with the focus of attention and distractor suppression is not warranted.

Pupillary correlates of individual differences in long-term memory

The present study is the first to examine individual differences in long-term memory, arousal dysregulation, and intensity of attention within the same experiment. Participants (N = 106) completed 28 lists of an immediate free-recall task while their pupil diameter was recorded via an eye-tracker during the encoding period. Two main pupillary measures were extracted: intraindividual variability in pre-list pupil diameter and evoked pupillary responses during item encoding. Variability in pre-list pupil diameter served as a measure of arousal dysregulation, and evoked pupillary responses served as a measure of intensity of attention. Based on prior work, we hypothesized that there would be a positive association between intensity of attention and recall ability, and that there would be a negative association between arousal dysregulation and recall ability. Collectively these two measures accounted for 19% of interindividual variance in recall, with 5% attributable uniquely to intensity of attention and 12% attributable uniquely to arousal regulation. The findings demonstrate that there are sources of individual differences in long-term memory that can be revealed via pupillometry, notably the amount of effort deployed during item encoding and the degree to which people exhibit dysregulated arousal. Both findings are consistent with recent theorizing regarding the role of the locus coeruleus (LC)-norepinephrine (NE) system's role in goal-directed cognition. Specifically, the LC governs both moment-to-moment arousal and NE release to cortical regions subserving cognitive processing. Among people for whom this system operates most optimally, long-term memory retention is superior.

The Neural Responses of Visual Complexity in the Oddball Paradigm: An ERP Study

This research measured human neural responses to images of different visual complexity levels using the oddball paradigm to explore the neurocognitive responses of complexity perception in visual processing. In the task, 24 participants (12 females) were required to react to images with high complexity for all stimuli. We hypothesized that high-complexity stimuli would induce early visual and attentional processing effects and may elicit the visual mismatch negativity responses and the emergence of error-related negativity. Our results showed that the amplitude of P1 and N1 were unaffected by complexity in the early visual processing. Under the target stimuli, both N2 and P3b components were reported, suggesting that the N2 component was sensitive to the complexity deviation, and the attentional processing related to complexity may be derived from the occipital zone according to the feature of the P3b component. In addition, compared with the low-complexity stimulus, the high-complexity stimulus aroused a larger amplitude of the visual mismatch negativity. The detected error negativity (Ne) component reflected the error detection of the participants’ mismatch between visual complexity and psychological expectations.

The Sternberg Paradigm: Correcting Encoding Latencies in Visual and Auditory Test Designs

The Sternberg task is a widely used tool for assessing the working memory performance in vision and cognitive science. It is possible to apply a visual or auditory variant of the Sternberg task to query the memory load. However, previous studies have shown that the subjects' corresponding reaction times differ dependent on the used variant. In this work, we present an experimental approach that is intended to correct the reaction time differences observed between auditory and visual item presentation. We found that the subjects' reaction time offset is related to the encoding speed of a single probe item. After correcting for these individual encoding latencies, differences in the results of both the auditory and visual Sternberg task become non-significant, p=0.252. Thus, an equal task difficulty can be concluded for both variants of item presentation.

Bottom-up perceptual salience and top-down retro-cues concurrently determine state in visual working memory

Recent studies have demonstrated that in visual working memory (VWM), only items in an active state can guide attention. Further evidence has revealed that items with higher perceptual salience or items prioritised by a valid retro-cue in VWM tend to be in an active state. However, it is unclear which factor (perceptual salience or retro-cues) is more important for influencing the item state in VWM or whether the factors can act concurrently. Experiment 1 examined the role of perceptual salience by asking participants to hold two features with relatively different perceptual salience (colour vs. shape) in VWM while completing a visual search task. Guidance effects were found when either colour or both colour and shape in VWM matched one of the search distractors but not when shape matched. This demonstrated that the more salient feature in VWM can actively guide attention, while the less salient feature cannot. However, when shape in VWM was cued to be more relevant (prioritised) in Experiment 2, we found guidance effects in both colour-match and shape-match conditions. That is, both more salient but non-cued colour and less salient but cued shape could be active in VWM, such that attentional selection was affected by the matching colour or shape in the visual search task. This suggests that bottom-up perceptual salience and top-down retro-cues can jointly determine the active state in VWM.

Tracking visual search demands and memory load through pupil dilation

Continuously tracking cognitive demands via pupil dilation is a desirable goal for the monitoring and investigation of cognitive performance in applied settings where the exact time point of mental engagement in a task is often unknown. Yet, hitherto no experimentally validated algorithm exists for continuously estimating cognitive demands based on pupil size. Here, we evaluated the performance of a continuously operating algorithm that is agnostic of the onset of the stimuli and derives them by way of retrospectively modeling attentional pulses (i.e., onsets of processing). We compared the performance of this algorithm to a standard analysis of stimulus-locked pupil data. The pupil data were obtained while participants performed visual search (VS) and visual working memory (VWM) tasks with varying cognitive demands. In Experiment 1, VS was performed during the retention interval of the VWM task to assess interactive effects between search and memory load on pupil dilation. In Experiment 2, the tasks were performed separately. The results of the stimulus-locked pupil data demonstrated reliable increases in pupil dilation due to high VWM load. VS difficulty only affected pupil dilation when simultaneous memory demands were low. In the single task condition, increased VS difficulty resulted in increased pupil dilation. Importantly, online modeling of pupil responses was successful on three points. First, there was good correspondence between the modeled and stimulus locked pupil dilations. Second, stimulus onsets could be approximated from the derived attentional pulses to a reasonable extent. Third, cognitive demands could be classified above chance level from the modeled pupil traces in both tasks.

Visual Working Memory of Chinese Characters and Expertise: The Expert's Memory Advantage Is Based on Long-Term Knowledge of Visual Word Forms

People unfamiliar with Chinese characters show poorer visual working memory (VWM) performance for Chinese characters than do literates in Chinese. In a series of experiments, we investigated the reasons for this expertise advantage. Experiments 1 and 2 showed that the advantage of Chinese literates does not transfer to novel material. Experts had similar resolution as novices for material outside of their field of expertise, and the memory of novices and experts did not differ when detecting a big change, e.g., when a character’s color was changed. Memorizing appears to function as a rather abstract representation of word forms because memory for characters’ fonts was poor independently of expertise (Experiment 3), though still visual. Distractors that were highly similar conceptually did not increase memory errors, but visually similar distractors impaired memory (Experiment 4). We hypothesized that literates in Chinese represent characters in VWM as tokens of visual word forms made available by long-term memory. In Experiment 5, we provided novices with visual word form knowledge. Participants subsequently performed a change detection task with trained and novel characters in a functional magnetic resonance experiment. We analyzed set size- and training-dependent effects in the intraparietal sulcus (IPS) and the visual word form area. VWM for trained characters was better than for novel characters. Neural activity increased with set size and at a slower rate for trained than for novel characters. All conditions approached the same maximum, but novel characters reached the maximum at a smaller set size than trained characters. The time course of the bold response depended on set size and knowledge status. Starting from the same initial maximum, neural activity at small set sizes returned to baseline more quickly for trained characters than for novel characters. Additionally, high performers showed generally more neural activity in the IPS than low performers. We conclude that experts’ better performance in working memory (WM) is caused by the availability of visual long-term representations (word form types) that allow a sparse representation of the perceived stimuli and make even small changes big because they cause a type change that is easily detected.

Brain Processing of Complex Geometric Forms in a Visual Memory Task Increases P2 Amplitude

We study the cognitive processing of visual working memory in three different conditions of memory load and configuration change. Altering this features has been shown to alter the brain's processing in memory tasks. Most studies dealing with this issue have used the verbal-phonological modality. We use complex geometric polygons to assess visual working memory in a modified change detection task. Three different types of backgrounds were used to manipulate memory loading and 18 complex geometric polygons to manipulate stimuli configuration. The goal of our study was to test whether the memory load and configuration affect the correct-recall ratios. We expected that increasing visual items loading and changing configuration of items would induce differences in working memory performance. Brain activity related to the task was assessed through event-related potentials (ERP), during the test phase of each trial. Our results showed that visual items loading and changing of item configuration affect working memory on test phase on ERP component P2, but does not affect performance. However frontal related ERP component-P3-was minimally affected by visual memory loading or configuration changing, supporting that working memory is related to a filtering processing in posterior brain regions.

Tracking working memory maintenance with pupillometry

Phasic pupillary responses were used to track the active maintenance of information in working memory (WM). In seven experiments participants performed various change detection tasks while their pupils were continuously recorded. Across the experiments phasic pupillary responses increased as the number of maintained items increased up to around 4-5 items consistent with behavioral estimates of capacity. Combining data across experiments demonstrated that phasic pupillary responses were related to behavioral estimates of capacity. Furthermore, phasic pupillary responses demonstrated WM load-dependent relations only when active maintenance was required. When instructed to passively stare at the items or to drop items from WM, the pupil remained near baseline levels. These phasic pupillary responses also tracked the time course of maintenance demonstrating sustained responses early in the delay period, but declined thereafter. Finally, phasic pupillary responses tracked selection processes at encoding (filtering and pre-cues), but did not suggest evidence for item removal following retro-cues. These results are consistent with the notion that maintaining items in WM requires the allocation of effortful attention and further suggest that phasic pupillary responses can be used to track the active maintenance of items in WM.

Pupil dilation as an index of effort in cognitive control tasks: A review

Pupillometry research has experienced an enormous revival in the last two decades. Here we briefly review the surge of recent studies on task-evoked pupil dilation in the context of cognitive control tasks with the primary aim being to evaluate the feasibility of using pupil dilation as an index of effort exertion, rather than task demand or difficulty. Our review shows that across the three cognitive control domains of updating, switching, and inhibition, increases in task demands typically leads to increases in pupil dilation. Studies show a diverging pattern with respect to the relationship between pupil dilation and performance and we show how an effort account of pupil dilation can provide an explanation of these findings. We also discuss future directions to further corroborate this account in the context of recent theories on cognitive control and effort and their potential neurobiological substrates.

Retroactive Attentional Shifts Predict Performance in a Working Memory Task: Evidence by Lateralized EEG Patterns

Shifts of attention within working memory based on retroactive (retro-) cues were shown to facilitate performance in working memory tasks. Although posterior asymmetries in the EEG, such as the contralateral delay activity (CDA), have been used to study the active storage of lateralized working memory representations, results on the relation of such asymmetric effects to retro-cue benefits remain inconclusive. We recorded EEG in a retro-cue working memory task with lateralized items and a continuous performance response. Following either a selective or neutral retro-cue, participants adjusted the orientation of a central memory probe to the cued item. Selective retro-cues elicited an early posterior contralateral negativity (PCN), anterior directing attention negativity (ADAN) and a later modulation of CDA indicating that active storage was concentrated on the cued information. By dividing all trials into three within-condition performance quantiles, we could further show that high working memory accuracy was associated with a sustained increase of the CDA effect following the retro-cue. These results suggest that focusing resources on the active storage of relevant representations is an important factor regarding retro-cue benefits in working memory tasks.

Contralateral delay activity does not reflect behavioral feature load in visual working memory

An ongoing debate in visual working memory research is concentrated on whether visual working memory capacity is determined solely by the number of objects to be memorized, or additionally by the number of relevant features contained within objects. Using a novel change detection task that contained multi-feature objects we examined the effect of both object number and feature number on visual working memory capacity, change detection sensitivity, and posterior slow wave event-related brain potential (ERP) activity. Behaviorally, working memory capacity and sensitivity were modulated as a function of both the number of objects and the number of features memorized per object. However, the Contralateral Delay Activity (CDA) was only sensitive to the number of objects, but not to the number of features. This suggests that while both objects and features contribute to limitations in visual working memory capacity, the CDA is an insufficient mechanism to account for feature level representations.

Early selection versus late correction: Age-related differences in controlling working memory contents

We examined whether a reduced ability to ignore irrelevant information is responsible for the age-related decline of working memory (WM) functions. By means of event-related brain potentials, we will show that filtering is not out of service in older adults but shifted to a later processing stage. Participants performed a visual short-term memory task (change-detection task) in which targets were presented along with distractors. To allow early selection, a cue was presented in advance of each display, indicating where the targets were to appear. Despite this relatively easy selection criterion, older adults' filtering was delayed as indicated by the amplitude pattern of the contralateral delay activity. Importantly, WM-equated younger adults did not show a delay indicating that the delay is specific to older adults and not a general phenomenon that comes with low WM capacity. Moreover, the analysis of early visual potentials revealed qualitatively different perceptual/attentional processing between the age groups. Young adults exhibited stronger distractor sensitivity that in turn facilitated filtering. Older adults, in contrast, seemed to initially store distractors and to suppress them after the fact. These early selection versus late-correction modes suggest an age-related shift in the strategy to control the contents of WM. (PsycINFO Database Record