Toward the influence of temporal attention on the selection of targets in a visual search task: An ERP study

Which ERP components are effective in measuring cognitive load in multimedia learning? A meta-analysis based on relevant studies

The open and generative nature of multimedia learning environments tends to cause cognitive overload in learners, and cognitive load is difficult for researchers to observe objectively because of its implicit and complex nature. Event-related potentials (ERP), a method of studying potential changes associated with specific events or stimuli by recording the electroencephalogram (EEG), has become an important method of measuring cognitive load in cognitive psychology. Although many studies have relied on ERP output measurements to compare different levels of cognitive load in multimedia learning, the results of the effect of cognitive load on ERP have been inconsistent. In this study, we used a meta-analysis of evidence-based research to quantitatively analyze 17 experimental studies to quantitatively evaluate which ERP component (amplitude) is most sensitive to cognitive load. Forty five effect sizes from 26 studies involving 360 participants were calculated. (1) The results of the studies analyzed in subgroups indicated high level effect sizes for P300 and P200 (2) Analyses of moderating variables for signal acquisition did not find that different methods of signal acquisition had a significant effect on the measurement of cognitive load (3) Analyses of moderating variables for task design found that a task system with feedback was more convenient for the measurement of cognitive load, and that designing for 3 levels of cognitive load was more convenient for the measurement of cognitive load than for 2 levels of cognitive load. (4) Analyses of continuous moderating variables for subject characteristics did not find significant effects of age, gender, or sample size on the results.

Tactile temporal predictions: The influence of conditional probability

Predicting the timing of incoming information allows brain to optimize information processing in dynamic environments. However, the effects of temporal predictions on tactile perception are not well established. In this study, two experiments were conducted to determine how temporal predictions interact with conditional probabilities in tactile perceptual processing. In Experiment 1, we explored the range of the interval between preceding ready cues and imperative targets in which temporal prediction effects can be observed. This prediction effect was observed for intervals of 500 and 1,000 ms. In Experiment 2, we investigated the benefits of temporal predictions on tactile perception while manipulating the conditional probability (setting the stimulus onset earlier or later than the predicted moment in short and long intervals). Our results revealed that this effect became stronger as the probability of the stimulus at the predicted time point increased under short-interval conditions. Together, our results show that the difficulty of transferring processing resources increases in temporally dynamic environments, suggesting a greater subjective cost associated with maladaptive responses to temporally uncertain events.

Null effects of temporal prediction on recognition memory but evidence for differential neural activity at encoding. A registered report

Previous research has demonstrated that rhythmic presentation of stimuli during encoding boosts subsequent recognition and is associated with distinct neural activity compared with when stimuli are presented in an arrhythmic manner. However, it is unclear whether the effect is driven by automatic entrainment to rhythm or non-rhythmic temporal prediction. This registered report presents an Electroencephalographic (EEG) study aimed at establishing the cognitive and neural mechanisms of the effect of temporal prediction on recognition. In a blocked design, stimulus onset during encoding was systematically manipulated in four conditions prior to recognition testing: rhythmic fixed (RF), rhythmic variable (RV), arrhythmic fixed (AF), and arrhythmic variable (AV). By orthogonally varying rhythm and temporal position we were able to assess their independent contributions to recognition enhancement. Our behavioural results did not replicate previous findings that show a difference in recognition memory based on temporal predictability at encoding. However, event-related potential (ERP) component analysis did show an early (N1) interaction effect of temporal position and rhythm, and later (N2 and Dm) effects driven by temporal position only. Taken together, we observed effects of temporal prediction at encoding, but these differences did not translate to later effects of memory, suggesting that effects of temporal prediction on recognition are less robust than previously thought.

Attention to space and time: Independent or interactive systems? A narrative review

While there is ample evidence for the ability to selectively attend to where in space and when in time a relevant event might occur, it remains poorly understood whether spatial and temporal attention operate independently or interactively to optimize behavior. To elucidate this important issue, we provide a narrative review of the literature investigating the relationship between the two. The studies were organized based on the attentional manipulation employed (endogenous vs. exogenous) and the type of task (detection vs. discrimination). Although the reviewed findings depict a complex scenario, three aspects appear particularly important in promoting independent or interactive effects of spatial and temporal attention: task demands, attentional manipulation, and their combination. Overall, the present review provides key insights into the relationship between spatial and temporal attention and identifies some critical gaps that need to be addressed by future research.

ERP Study of Mine Management System Warning Interface under Fatigue

Due to the large volume of monitoring data in mines, concentrating on and reviewing the data for a long period of time will easily cause fatigue. To study the influence of different visual codes of early-warning interfaces on the response of individuals who are fatigued, the changes in the subjective fatigue and corresponding frequency waves are compared before and after a fatigue-inducing task, as well as using event-related potential to study the behavioral data and EEG signals of subjects who participated in an oddball task on an early-warning interface. The results showed that all 14 subjects became fatigued after the fatigue-inducing task, and the amplitude of P200 when text is used in a fatigued state was the largest, with the longest latency. The subjects showed a slower reaction time and a reduced accuracy rate, thus indicating that in designing a warning interface, when text rather than color is used as a visual code, the operating load will be larger, mental load is increased, and attention resources are consumed. The experimental results provide the basis for the design and evaluation of early-warning interfaces of mine management systems.

Interaction of prior category knowledge and novel statistical patterns during visual search for real-world objects

Two aspects of real-world visual search are typically studied in parallel: category knowledge (e.g., searching for food) and visual patterns (e.g., predicting an upcoming street sign from prior street signs). Previous visual search studies have shown that prior category knowledge hinders search when targets and distractors are from the same category. Other studies have shown that task-irrelevant patterns of non-target objects can enhance search when targets appear in locations that previously contained these irrelevant patterns. Combining EEG (N2pc ERP component, a neural marker of target selection) and behavioral measures, the present study investigated how search efficiency is simultaneously affected by prior knowledge of real-world objects (food and toys) and irrelevant visual patterns (sequences of runic symbols) within the same paradigm. We did not observe behavioral differences between locating items in patterned versus random locations. However, the N2pc components emerged sooner when search items appeared in the patterned location, compared to the random location, with a stronger effect when search items were targets, as opposed to non-targets categorically related to the target. A multivariate pattern analysis revealed that neural responses during search trials in the same time window reflected where the visual patterns appeared. Our finding contributes to our understanding of how knowledge acquired prior to the search task (e.g., category knowledge) interacts with new content within the search task.

Temporal preparation accelerates spatial selection by facilitating bottom-up processing

Temporal preparation facilitates spatial selection in visual search. This selection benefit has not only been observed for targets, but also for task-irrelevant, salient distractors. This result suggests that temporal preparation influences bottom-up salience in spatial selection. To test this assumption, we conducted an event-related-potential (ERP) study in which we measured the joint effect of temporal preparation and target salience on the N2pc as an index of spatial selection and the N1 as an index of perceptual discrimination. To manipulate target salience, we employed two different setsizes (i.e., a small or large number of homogeneous distractors). To manipulate temporal preparation, we presented a warning signal before the search display and we varied the length of the interval (foreperiod) between warning signal and search display in different blocks of trials (constant foreperiod paradigm). Replicating previous results, we observed that the N1 and the N2pc arose earlier in case of good temporal preparation. Importantly, the beneficial effect on the N2pc onset latency was stronger when the target salience was initially low (i.e., small setsize). This result provides evidence that temporal preparation influences bottom-up processing and, thereby, facilitates spatial selection.

Reduction of temporal uncertainty facilitates stimulus-driven processes in spatial selection

Previous studies have shown that the reduction of temporal uncertainty facilitates target selection in visual search. We investigated whether this beneficial effect is caused by an effect on stimulus-driven processes or on goal-driven processes in spatial selection. To discriminate between these processes, we employed a visual search task in which participants searched for a shape target while ignoring a color singleton distractor. As an index of stimulus-driven processes, we measured the N2pc evoked by the singleton distractor (N_D). As indices of goal-driven processes, we measured the N2pc evoked by the target (N_T) and the distractor positivity (P_D) evoked by the singleton distractor, respectively. We observed that reducing temporal uncertainty modulated the amplitude of N_D and the onset latency of the N_T, but did not modulate the amplitude of the P_D. These results are consistent with the view that a reduction of temporal uncertainty influences non-selective, stimulus-driven processes in spatial selection.

Environmental rhythms orchestrate neural activity at multiple stages of processing during memory encoding: Evidence from event-related potentials

Accumulating evidence suggests that rhythmic temporal structures in the environment influence memory formation. For example, stimuli that appear in synchrony with the beat of background, environmental rhythms are better remembered than stimuli that appear out-of-synchrony with the beat. This rhythmic modulation of memory has been linked to entrained neural oscillations which are proposed to act as a mechanism of selective attention that prioritize processing of events that coincide with the beat. However, it is currently unclear whether rhythm influences memory formation by influencing early (sensory) or late (post-perceptual) processing of stimuli. The current study used stimulus-locked event-related potentials (ERPs) to investigate the locus of stimulus processing at which rhythm temporal cues operate in the service of memory formation. Participants viewed a series of visual objects that either appeared in-synchrony or out-of-synchrony with the beat of background music and made a semantic classification (living/non-living) for each object. Participants’ memory for the objects was then tested (in silence). The timing of stimulus presentation during encoding (in-synchrony or out-of-synchrony with the background beat) influenced later ERPs associated with post-perceptual selection and orienting attention in time rather than earlier ERPs associated with sensory processing. The magnitude of post-perceptual ERPs also differed according to whether or not participants demonstrated a mnemonic benefit for in-synchrony compared to out-of-synchrony stimuli, and was related to the magnitude of the rhythmic modulation of memory performance across participants. These results support two prominent theories in the field, the Dynamic Attending Theory and the Oscillation Selection Hypothesis, which propose that neural responses to rhythm act as a core mechanism of selective attention that optimize processing at specific moments in time. Furthermore, they reveal that in addition to acting as a mechanism of early attentional selection, rhythm influences later, post-perceptual cognitive processes as events are transformed into memory.

Temporal attention boosts perceptual effects of spatial attention and feature-based attention

Temporal attention, that is, the process of anticipating the occurrence of a stimulus at a given time point, has been shown to improve perceptual processing of visual stimuli. In the present study, we investigated whether and how temporal attention interacts with spatial attention and feature-based attention in visual selection. To monitor the influence of the three different attention dimensions on perceptual processing, we measured event-related potentials (ERPs). Our participants performed a visual search task, in which a colored singleton was presented amongst homogenous distractors. We manipulated spatial and feature-based attention by requiring participants to respond only to target singletons in a particular color and at a to-be-attended spatial location. We manipulated temporal attention by means of an explicit temporal cue that announced either validly or invalidly the occurrence of the search display. We obtained early ERP effects of spatial attention and feature-based attention at the validly cued but not at the invalidly cued time point. Taken together, our results suggest that temporal attention boosts early effects of spatial and feature-based attention.

Using Partial Directed Coherence to Study Alpha-Band Effective Brain Networks during a Visuospatial Attention Task

Previous studies have shown that the neural mechanisms underlying visual spatial attention rely on top-down control information from the frontal and parietal cortexes, which ultimately amplifies sensory processing of stimulus occurred at the attended location relative to those at unattended location. However, the modulations of effective brain networks in response to stimulus at attended and unattended location are not yet clear. In present study, we collected event-related potentials (ERPs) from 15 subjects during a visual spatial attention task, and a partial directed coherence (PDC) method was used to construct alpha-band effective brain networks of two conditions (targets at attended and nontargets at unattended location). Flow gain mapping, effective connectivity pattern, and graph measures including clustering coefficient (C), characteristic path length (L), global efficiency (E_global), and local efficiency (E_local) were compared between two conditions. Flow gain mapping showed that the frontal region seemed to serve as the main source of information transmission in response to targets at attended location while the parietal region served as the main source in nontarget condition. Effective connectivity pattern indicated that in response to targets, there existed obvious top-down connections from the frontal, temporal, and parietal cortexes to the visual cortex compared with in response to nontargets. Graph theory analysis was used to quantify the topographical properties of the brain networks, and results revealed that in response to targets, the brain networks were characterized by significantly smaller characteristic path length and larger global efficiency than in response to nontargets. Our findings suggested that smaller characteristic path length and larger global efficiency could facilitate global integration of information and provide a substrate for more efficient perceptual processing of targets at attended location compared with processing of nontargets at ignored location, which revealed the neural mechanisms underlying visual spatial attention from the perspective of effective brain networks and graph theory for the first time and opened new vistas to interpret a cognitive process.

Enhanced spatial focusing increases feature-based selection in unattended locations

Attention is a multifaceted phenomenon, which operates on features (e.g., colour or motion) and over space. A fundamental question is whether the attentional selection of features is confined to the spatially-attended location or operates independently across the entire visual field (global feature-based attention, GFBA). Studies providing evidence for GFBA often employ feature probes presented at spatially unattended locations, which elicit enhanced brain responses when they match a currently-attended target feature. However, the validity of this interpretation relies on consistent spatial focusing onto the target. If the probe were to temporarily attract spatial attention, the reported effects could reflect transient spatial selection processes, rather than GFBA. Here, using magnetoencephalographic recordings (MEG) in humans, we manipulate the strength and consistency of spatial focusing to the target by increasing the target discrimination difficulty (Experiment 1), and by demarcating the upcoming target’s location with a placeholder (Experiment 2), to see if GFBA effects are preserved. We observe that motivating stronger spatial focusing to the target did not diminish the effects of GFBA. Instead, aiding spatial pre-focusing with a placeholder enhanced the feature response at unattended locations. Our findings confirm that feature selection effects measured with spatially-unattended probes reflect a true location-independent neural bias.

Supplementation of gamma-aminobutyric acid (GABA) affects temporal, but not spatial visual attention

In a randomized, double-blind, and placebo-controlled experiment, the acute effects of gamma-aminobutyric acid (GABA) supplementation on temporal and spatial attention in young healthy adults were investigated. A hybrid two-target rapid serial visual presentation task was used to measure temporal attention and integration. Additionally, a visual search task was used to measure the speed and accuracy of spatial attention. While temporal attention depends primarily on the distribution of limited attentional resources across time, spatial attention represents the engagement and disengagement by relevant and irrelevant stimuli across the visual field. Although spatial attention was unaffected by GABA supplementation altogether, we found evidence supporting improved performance in the temporal attention task. The attentional blink was numerically, albeit not significantly, attenuated at Lag 3, and significantly fewer order errors were committed at Lag 1, compared to the placebo condition. No effect was found on temporal integration rates. Although there is controversy about whether oral GABA can cross the blood-brain barrier, our results offer preliminary evidence that GABA intake might help to distribute limited attentional resources more efficiently, and can specifically improve the identification and ordering of visual events that occur in close temporal succession.

Oculomotor measures reveal the temporal dynamics of preparing for search

Theories of visual search assume that selection is driven by an active template representation of the target object. Earlier studies suggest that template activation occurs prior to search, but the temporal dynamics of such preactivation remain unclear. Two experiments employed microsaccades to track both general preparation (i.e., anticipation of the search task as such) and template-specific preparation (i.e., anticipation of target selection) of visual search. Participants memorized a target color (i.e., the template) for an upcoming search task. During the delay period, we presented an irrelevant rapid serial visual presentation (RSVP) of lateralized colored disks. Crucially, at different time points into the RSVP, the template color was inserted, allowing us to measure attentional biases toward the template match as a function of time. Results showed a general suppression of saccades: the closer in time to the search display, the less saccades were produced. This suppression was stronger when a template-matching color was present compared to when absent. However, when microsaccades occurred, they were biased toward the template-matching color and more so just prior to the search display. We conclude that observers adapt search template activation to the anticipated moment of search, and that microsaccades reflect general as well as target-specific preparation effects.

Time-based expectations entail preparatory motor activity

Human behavior is guided by expectations that facilitate perception of upcoming events or reaction to them. In natural settings expectations are often implicitly based on time, e.g., when making a phone call one would expect to hear either a person answering (earlier) or a voicemail greeting (later). We investigated how time-based expectations can improve performance in the absence of explicit prior information on the pending stimulus or the associated response. Visual stimuli were presented after a characteristic short or long foreperiod, and a forced-choice categorization using either the left or the right hand was required. The electroencephalogram (EEG) revealed a decrease in central 9-12 Hz power over the course of the trial. Moreover, lateralized pre-motor potentials were observed which changed polarity after the short foreperiod. At stimulus onset, amplitudes of pre-motor potentials co-varied with performance, so that higher (more negative) amplitudes were associated with slower responses to unexpected targets. Altogether, the results suggest that implicit time-based expectations entail effector-specific preparatory brain activity, which is inhibited until the expected onset time of the event. Thus, time-based expectations prepare for action.

Time-Based Expectancy for Task Relevant Stimulus Features

When a particular target stimulus appears more frequently after a certain interval than after another one, participants adapt to such regularity, as evidenced by faster responses to frequent interval-target combinations than to infrequent ones. This phenomenon is known as time-based expectancy. Previous research has suggested that time-based expectancy is primarily motor-based, in the sense that participants learn to prepare a particular response after a specific interval. Perceptual time-based expectancy — in the sense of learning to perceive a certain stimulus after specific interval — has previously not been observed. We conducted a Two-Alternative-Forced-Choice experiment with four stimuli differing in shape and orientation. A subset of the stimuli was frequently paired with a certain interval, while the other subset was uncorrelated with interval. We varied the response relevance of the interval-correlated stimuli, and investigated under which conditions time-based expectancy transfers from trials with interval-correlated stimuli to trials with interval-uncorrelated stimuli. Transfer was observed only where transfer of perceptual expectancy and transfer of response expectancy predicted the same behavioral pattern, not when they predicted opposite patterns. The results indicate that participants formed time-based expectancy for stimuli as well as for responses. However, alternative interpretations are also discussed.