Training-induced Changes in the Dynamics of Attention as Reflected in Pupil Dilation

Open-DPSM: An open-source toolkit for modeling pupil size changes to dynamic visual inputs

Pupil size change is a widely adopted, sensitive indicator for sensory and cognitive processes. However, the interpretation of these changes is complicated by the influence of multiple low-level effects, such as brightness or contrast changes, posing challenges to applying pupillometry outside of extremely controlled settings. Building on and extending previous models, we here introduce Open Dynamic Pupil Size Modeling (Open-DPSM), an open-source toolkit to model pupil size changes to dynamically changing visual inputs using a convolution approach. Open-DPSM incorporates three key steps: (1) Modeling pupillary responses to both luminance and contrast changes; (2) Weighing of the distinct contributions of visual events across the visual field on pupil size change; and (3) Incorporating gaze-contingent visual event extraction and modeling. These steps improve the prediction of pupil size changes beyond the here-evaluated benchmarks. Open-DPSM provides Python functions, as well as a graphical user interface (GUI), enabling the extension of its applications to versatile scenarios and adaptations to individualized needs. By obtaining a predicted pupil trace using video and eye-tracking data, users can mitigate the effects of low-level features by subtracting the predicted trace or assess the efficacy of the low-level feature manipulations a priori by comparing estimated traces across conditions.

It's time for attentional control: Temporal expectation in the attentional blink

The attentional blink (AB) reveals a limitation in conscious processing of sequential targets. Although it is widely held that the AB derives from a structural bottleneck of central capacity, how the central processing is constrained is still unclear. As the AB reflects the dilemma of deploying attentional resources in the time dimension, research on temporal allocation provides an important avenue for understanding the mechanism. Here we reviewed studies regarding the role of temporal expectation in modulating the AB performance primarily based on two temporal processing strategies: interval-based and rhythm-based timings. We showed that both temporal expectations can help to organize limited resources among multiple attentional episodes, thereby mitigating the AB effect. As it turns out, scrutinizing on the AB from a temporal perspective is a promising way to comprehend the mechanisms behind the AB and conscious cognition. We also highlighted some unresolved issues and discussed potential directions for future research.

Immediate Temporal Information Modulates the Target Identification in the Attentional Blink

It has been shown that learned temporal information can be exploited to help facilitate the target identification in the attentional blink task. Here, we tested whether similar exploitation also worked on short-term temporal information, even when it did not reliably predict the target onset. In two experiments, we randomly manipulated either the interval between targets (T1 and T2; Experiment 1) or the temporal regularity of stimulus presentation (Experiment 2) in each trial. The results revealed evidence of effects of immediate temporal experience mainly on T2 performances but also occasionally on T1 performances. In general, the accuracy of T2 was enhanced when a longer inter-target interval was explicitly processed in the preceding trial (Experiment 1) or the temporal regularity, regardless of being explicitly or implicitly processed, was present in the stimulus stream, especially after T1 (Experiment 2). These results suggest that, under high temporal uncertainty, both interval and rhythmic cues can still be exploited to regulate the allocation of processing resources, thus, modulating the target identification in the attentional blink task, consistent with the view of flexible attentional allocation, and further highlighting the importance of the interplay between temporal processing and attentional control in the conscious visual perception.

Testing the skill-based approach: Consolidation strategy impacts attentional blink performance

Humans can learn simple new tasks very quickly. This ability suggests that people can reuse previously learned procedural knowledge when it applies to a new context. We have proposed a modeling approach based on this idea and used it to create a model of the attentional blink (AB). The main idea of the skill-based approach is that models are not created from scratch but, instead, built up from reusable pieces of procedural knowledge (skills). This approach not only provides an explanation for the fast learning of simple tasks but also shows much promise to improve certain aspects of cognitive modeling (e.g., robustness and generalizability). We performed two experiments, in order to collect empirical support for the model’s prediction that the AB will disappear when the two targets are consolidated as a single chunk. Firstly, we performed an unsuccessful replication of a study reporting that the AB disappears when participants are instructed to remember the targets as a syllable. However, a subsequent experiment using easily combinable stimuli supported the model’s prediction and showed a strongly reduced AB in a large group of participants. This result suggests that it is possible to avoid the AB with the right consolidation strategy. The skill-based approach allowed relating this finding to a general cognitive process, thereby demonstrating that incorporating this approach can be very helpful to generalize the findings of cognitive models, which otherwise tends to be rather difficult.

Training the attentional blink: subclinical depression decreases learning potential

The attentional blink (AB) reflects a temporal restriction of selective attention and is generally regarded as a very robust phenomenon. However, previous studies have found large individual differences in AB performance, and under some training conditions the AB can be reduced significantly. One factor that may account for individual differences in AB magnitude is the ability to accurately time attention. In the current study, we focus on the sensitivity for temporal information on the ability to control attention. Following a visual AB task, a time estimation task was presented in either the visual or auditory modality, followed by another visual AB task. It was found that the time estimation training in both the auditory and visual modality reduced AB magnitude. Although a reduction in AB magnitude was also observed when individuals were trained on a control task (either an auditory frequency or visual line length estimation task), the effect was significantly larger following the time estimation tasks. In addition, it was found that individuals who showed most improvement on the visual time estimation task, also showed the largest reduction in AB magnitude, which was not the case for individuals who were trained on the control tasks. Finally, a negative correlation was observed between depression scores (tested by Beck Depression Inventory-Short Form (BDI-SF) scores and the improvement in the AB and time estimation tasks. Our findings demonstrate clear links between timing ability and mechanisms to control attention and emotion.

Attention to a Moment in Time Impairs Episodic Distinctiveness during Rapid Serial Visual Presentation

Human attention is limited in the ability to select and segregate relevant distinct events from the continuous flow of external information while concurrently encoding their temporal succession. While it is well-known that orienting attention to one external target stimulus impairs the encoding of ensuing relevant external events, it is still unknown whether orienting attention to internally generated events can interfere with concurrent processing of external input. We addressed this issue by asking participants to identify a single target embedded among distractors in a non-spatial rapid serial visual presentation (RSVP) stream and to indicate whether that target appeared before or after an internally estimated midpoint of the stream. The results indicate that (a) such an internally generated temporal benchmark does not interfere with the identification of a subsequent physical target stimulus but (b) the two events cannot be accurately segregated when the physical target immediately follows the internally generated temporal event. These findings indicate that the asymmetrical distribution around the midpoint of order reversals reflects an impaired temporal discrimination ability. Orienting attention to a moment in time reduces episodic distinctiveness as much as orienting attention to external events.

Modeling pupil responses to rapid sequential events

Pupil size is an easily accessible, noninvasive online indicator of various perceptual and cognitive processes. Pupil measurements have the potential to reveal continuous processing dynamics throughout an experimental trial, including anticipatory responses. However, the relatively sluggish (~2 s) response dynamics of pupil dilation make it challenging to connect changes in pupil size to events occurring close together in time. Researchers have used models to link changes in pupil size to specific trial events, but such methods have not been systematically evaluated. Here we developed and evaluated a general linear model (GLM) pipeline that estimates pupillary responses to multiple rapid events within an experimental trial. We evaluated the modeling approach using a sample dataset in which multiple sequential stimuli were presented within 2-s trials. We found: (1) Model fits improved when the pupil impulse response function (PuRF) was fit for each observer. PuRFs varied substantially across individuals but were consistent for each individual. (2) Model fits also improved when pupil responses were not assumed to occur simultaneously with their associated trial events, but could have non-zero latencies. For example, pupil responses could anticipate predictable trial events. (3) Parameter recovery confirmed the validity of the fitting procedures, and we quantified the reliability of the parameter estimates for our sample dataset. (4) A cognitive task manipulation modulated pupil response amplitude. We provide our pupil analysis pipeline as open-source software (Pupil Response Estimation Toolbox: PRET) to facilitate the estimation of pupil responses and the evaluation of the estimates in other datasets.

No evidence for an attentional bias towards implicit temporal regularities

Action and perception are optimized by exploiting temporal regularities, and it has been suggested that the attentional system prioritizes information that contains some form of structure. Indeed, Zhao, Al-Aidroos, and Turk-Browne (Psychological Science, 24(5), 667–677, 2013) found that attention was biased towards the location and low-level visual features of shapes that appeared with a regular order but were irrelevant for the main search task. Here, we investigate whether this bias also holds for irrelevant metrical temporal regularities. In six experiments, participants were asked to perform search tasks. In Experiments 1a–d, sequences of squares, each presented at one of four locations, appeared in between the search trials. Crucially, in one location, the square appeared with a regular rhythm, whereas the timing in the other locations was random. In Experiments 2a and 2b, a sequence of centrally presented colored circles was shown in between the search trials, of which one specific color appeared regularly. We expected that, if attention is automatically biased towards these temporal regularities, reaction times would be faster if the target matches the location (Experiments 1a–d) or color (Experiments 2a–b) of the regular stimulus. However, no reaction time benefit was observed for these targets, suggesting that there was no attentional bias towards the regularity. In addition, we found no evidence for attentional entrainment to the rhythmic stimulus. These results suggest that people do not use implicit rhythmic temporal regularities to guide their attention in the same way as they use order regularities.

Adaptive Changes in the Dynamics of Visual Attention With Extended Practice

Previous research indicates that visual attention can adapt to temporal stimulus patterns utilizing the rapid serial visual presentation (RSVP) task. However, how the temporal dynamics of an attentional pulse adapt to temporal patterns has not been explored. We addressed this question by conducting an attentional component analysis on RSVP performance and explored whether changes in attentional dynamics were accompanied by explicit learning about predictable target timing. We utilized an RSVP task in which a target letter appeared either in two possible RSVP positions in fixed-timing conditions or in random positions over 1, 2, or 3 days of training. In a transfer phase, the target appeared in previously presented or new positions. Over 3 days of practice the target identification rate, efficacy, and precision of a putative attentional pulse increased. These changes reflected general learning in the RSVP task resulting in attentional dynamics more efficiently focused on the target. Although group performance effects did not support learning of fixed target positions, target identification rates and the measure of the efficacy of an attentional pulse at these positions were positively associated with explicit learning. The current study is the first to provide a detailed description of practice related adaptation of attentional dynamics and suggests that timing specific changes might be mediated by explicit temporal learning.

Pupil size is modulated by the size of equal-luminance gratings

Pupil size changes with light. For this reason, researchers studying the effect of attention, contextual processing, and arousal on the pupillary response have matched the mean luminance of their stimuli across conditions to eliminate the contribution of differences in light levels. Here, we argue that the match of mean luminance is not enough. In Experiment 1, we presented a circular sinewave grating on a gray background for 2 seconds. The area of the grating could be 3°, 6°, or 9°. The mean luminance of each grating was equal to the luminance of the gray background, such that regardless of the size of the grating there was no change in mean luminance between conditions. Participants were asked to fixate the center of the grating and passively view it. We found that in all size conditions, there was a pupil constriction starting at about 300 ms after stimulus onset, and the pupil constriction increased with the size of the grating. In Experiment 2, when a small grating was presented immediately after the presentation of a large grating (or vice versa), the pupil constriction changed accordingly. In Experiment 3, we replicated Experiment 1 but had the subjects perform an attention-demanding fixation task in one session, and passively view the stimuli in the other. We found that the main effect of task was not significant. In sum, our results show that stimulus size can modulate pupil size robustly and steadily even when the luminance is matched across the different stimuli.

Time to see the bigger picture: Individual differences in the attentional blink

If two to-be-identified targets are presented in close temporal succession, identification for the second target is typically impaired. This attentional blink (AB) phenomenon has long been considered as a robust, universal cognitive limitation. However, more recent studies have demonstrated that AB task performance greatly differs between individuals, with some individuals even showing no AB in certain paradigms. Several studies have focused on these individual differences in an attempt to reveal the mechanism underlying the AB, but an overview of this approach is currently missing. Here, by reviewing studies regarding individual differences in AB task performance, we investigate how individual differences have contributed to our understanding of the AB. We show that the individual differences AB literature provides reliable indications that the AB is a multifaceted phenomenon that presumably arises from a combination of factors; individuals with higher levels of executive working memory (WM) functioning and broad attentional focus perform better in the AB paradigm than individuals with lower executive functioning of WM and narrow attentional focus. As it turns out, seeing the bigger picture certainly seems helpful for AB task performance.

Pupillary response indexes the metrical hierarchy of unattended rhythmic violations

The perception of music is a complex interaction between what we hear and our interpretation. This is reflected in beat perception, in which a listener infers a regular pulse from a musical rhythm. Although beat perception is a fundamental human ability, it is still unknown whether attention to the music is necessary to establish the perception of stronger and weaker beats, or meter. In addition, to what extent beat perception is dependent on musical expertise is still a matter of debate. Here, we address these questions by measuring the pupillary response to omissions at different metrical positions in drum rhythms, while participants attended to another task. We found that the omission of the salient first beat elicited a larger pupil dilation than the omission of the less-salient second beat. This result shows that participants not only detected the beat without explicit attention to the music, but also perceived a metrical hierarchy of stronger and weaker beats. This suggests that hierarchical beat perception is an automatic process that requires no or minimal attentional resources. In addition, we found no evidence for the hypothesis that hierarchical beat perception is affected by musical expertise, suggesting that elementary beat perception might be independent from musical expertise. Finally, our results show that pupil dilation reflects surprise without explicit attention, demonstrating that the pupil is an accessible index to signatures of unattentive processing.

An Individual Differences Approach to Temporal Integration and Order Reversals in the Attentional Blink Task

Background

The reduced ability to identify a second target when it is presented in close temporal succession of a first target is called the attentional blink (AB). Studies have shown large individual differences in AB task performance, where lower task performance has been associated with more reversed order reports of both targets if these were presented in direct succession. In order to study the suggestion that reversed order reports reflect loss of temporal information, in the current study, we investigated whether individuals with a larger AB have a higher tendency to temporally integrate both targets into one visual event by using an AB paradigm containing symbol target stimuli.

Methodology/Principal Findings

Indeed, we found a positive relation between the tendency to temporally integrate information and individual AB magnitude. In contrast to earlier work, we found no relation between order reversals and individual AB magnitude. The occurrence of temporal integration was negatively related to the number of order reversals, indicating that individuals either integrated or separated and reversed information.

Conclusion

We conclude that individuals with better AB task performance use a shorter time window to integrate information, and therefore have higher preservation of temporal information. Furthermore, order reversals observed in paradigms with alphanumeric targets indeed seem to at least partially reflect temporal integration of both targets. Given the negative relation between temporal integration and ‘true’ order reversals observed with the current symbolic target set, these two behavioral outcomes seem to be two sides of the same coin.

Individual Differences in Temporal Selective Attention as Reflected in Pupil Dilation

BACKGROUND

Attention is restricted for the second of two targets when it is presented within 200-500 ms of the first target. This attentional blink (AB) phenomenon allows one to study the dynamics of temporal selective attention by varying the interval between the two targets (T1 and T2). Whereas the AB has long been considered as a robust and universal cognitive limitation, several studies have demonstrated that AB task performance greatly differs between individuals, with some individuals showing no AB whatsoever.

METHODOLOGY/PRINCIPAL FINDINGS

Here, we studied these individual differences in AB task performance in relation to differences in attentional timing. Furthermore, we investigated whether AB magnitude is predictive for the amount of attention allocated to T1. For both these purposes pupil dilation was measured, and analyzed with our recently developed deconvolution method. We found that the dynamics of temporal attention in small versus large blinkers differ in a number of ways. Individuals with a relatively small AB magnitude seem better able to preserve temporal order information. In addition, they are quicker to allocate attention to both T1 and T2 than large blinkers. Although a popular explanation of the AB is that it is caused by an unnecessary overinvestment of attention allocated to T1, a more complex picture emerged from our data, suggesting that this may depend on whether one is a small or a large blinker.

CONCLUSION

The use of pupil dilation deconvolution seems to be a powerful approach to study the temporal dynamics of attention, bringing us a step closer to understanding the elusive nature of the AB. We conclude that the timing of attention to targets may be more important than the amount of allocated attention in accounting for individual differences.

Adjustment to Subtle Time Constraints and Power Law Learning in Rapid Serial Visual Presentation

We investigated whether attention could be modulated through the implicit learning of temporal information in a rapid serial visual presentation (RSVP) task. Participants identified two target letters among numeral distractors. The stimulus-onset asynchrony immediately following the first target (SOA1) varied at three levels (70, 98, and 126 ms) randomly between trials or fixed within blocks of trials. Practice over 3 consecutive days resulted in a continuous improvement in the identification rate for both targets and attenuation of the attentional blink (AB), a decrement in target (T2) identification when presented 200-400 ms after another target (T1). Blocked SOA1s led to a faster rate of improvement in RSVP performance and more target order reversals relative to random SOA1s, suggesting that the implicit learning of SOA1 positively affected performance. The results also reveal "power law" learning curves for individual target identification as well as the reduction in the AB decrement. These learning curves reflect the spontaneous emergence of skill through subtle attentional modulations rather than general attentional distribution. Together, the results indicate that implicit temporal learning could improve high level and rapid cognitive processing and highlights the sensitivity and adaptability of the attentional system to subtle constraints in stimulus timing.

Sleep after practice reduces the attentional blink

The attentional blink (AB) is an impairment in detecting the second of two targets that appear in close temporal succession. We investigated the effect of practice and a nap on the magnitude of the AB deficit. We found evidence that sleep boosts practice-dependent reduction of the AB. Participants reported two target letters embedded in a rapid serial visual presentation display. After two morning sessions, half the participants took a polysomnographically recorded nap, while the others remained awake. Comparing two afternoon sessions to the two morning sessions, we observed a decreased AB only within the group who napped. The improvement was due to increased efficacy of the attentional selection of T2 (the probability of reporting a T2-relevant item). There was no change in selection's latency or temporal precision. The magnitude of improvement was positively associated with the duration of N2 sleep and the number of N2 sleep spindles. Our results suggest that sleep, particularly N2 sleep and sleep spindles, improves attentional selection in time.