The warning stimulus as retrieval cue: The role of associative memory in temporal preparation

Interdependence of "What" and "When" in the Brain

From a brain's-eye-view, when a stimulus occurs and what it is are interrelated aspects of interpreting the perceptual world. Yet in practice, the putative perceptual inferences about sensory content and timing are often dichotomized and not investigated as an integrated process. We here argue that neural temporal dynamics can influence what is perceived, and in turn, stimulus content can influence the time at which perception is achieved. This computational principle results from the highly interdependent relationship of what and when in the environment. Both brain processes and perceptual events display strong temporal variability that is not always modeled; we argue that understanding-and, minimally, modeling-this temporal variability is key for theories of how the brain generates unified and consistent neural representations and that we ignore temporal variability in our analysis practice at the peril of both data interpretation and theory-building. Here, we review what and when interactions in the brain, demonstrate via simulations how temporal variability can result in misguided interpretations and conclusions, and outline how to integrate and synthesize what and when in theories and models of brain computation.

Sequential effect and temporal orienting in prestimulus oculomotor inhibition

When faced with unfamiliar circumstances, we often turn to our past experiences with similar situations to shape our expectations. This results in the well-established sequential effect, in which previous trials influence the expectations of the current trial. Studies have revealed that, in addition to the classical behavioral metrics, the inhibition of eye movement could be used as a biomarker to study temporal expectations. This prestimulus oculomotor inhibition is found a few hundred milliseconds prior to predictable events, with a stronger inhibition for predictable than unpredictable events. The phenomenon has been found to occur in various temporal structures, such as rhythms, cue-association, and conditional probability, yet it is still unknown whether it reflects local sequential information of the previous trial. To explore this, we examined the relationship between the sequential effect and the prestimulus oculomotor inhibition. Our results (N = 40) revealed that inhibition was weaker when the previous trial was longer than the current trial, in line with findings of behavioral metrics. These findings indicate that the prestimulus oculomotor inhibition covaries with expectation based on local sequential information, demonstrating the tight connection between this phenomenon and expectation and providing a novel measurement for studying sequential effects in temporal expectation.

How Does Temporal Blurring Alter Movement Timing?

Subjective uncertainty arises because the estimation of the timing of an event into the future is error prone. This impact of stimulus-bound uncertainty on movement preparation has often been investigated using reaction time tasks where a warning stimulus (WS) predicts the occurrence of a "go" signal. The timing of the "go" signal can be chosen from a particular probability distribution with a given variance or uncertainty. It has been repeatedly shown that reaction times covary with the shape of the used "go" signal distribution. This is interpreted as evidence for temporal preparation. Moreover, the variance of the response time should always increase with the duration of the delay between the WS and the "go" signal. This increasing variance has been interpreted as a consequence of the temporal "blurring" of future events (scalar expectancy). The present paper tested the validity of the temporal "blurring" hypothesis in humans with a simple oculomotor reaction time task where subjective and stimulus-bound uncertainties were increased. Subjective uncertainty about the timing of a "go" signal was increased by lengthening the delay between the WS and the "go" signal. Objective uncertainty was altered by increasing the variance of "go" signal timing. Contrary to temporal blurring hypotheses, the study has shown that increasing the delay between events did not significantly increase movement timing variability. These results suggest that temporal blurring could not be a property of movement timing in an implicit timing context.

Statistical learning of distractor locations is dependent on task context

Through statistical learning, humans can learn to suppress visual areas that often contain distractors. Recent findings suggest that this form of learned suppression is insensitive to context, putting into question its real-life relevance. The current study presents a different picture: we show context-dependent learning of distractor-based regularities. Unlike previous studies which typically used background cues to differentiate contexts, the current study manipulated task context. Specifically, the task alternated from block to block between a compound search and a detection task. In both tasks, participants searched for a unique shape, while ignoring a uniquely colored distractor item. Crucially, a different high-probability distractor location was assigned to each task context in the training blocks, and all distractor locations were made equiprobable in the testing blocks. In a control experiment, participants only performed a compound search task such that the contexts were made indistinguishable, but the high-probability locations changed in exactly the same way as in the main experiment. We analyzed response times for different distractor locations and show that participants can learn to suppress a location in a context-dependent way, but suppression from previous task contexts lingers unless a new high-probability location is introduced.

Warning signals only support the first action in a sequence

Acting upon target stimuli from the environment becomes faster when the targets are preceded by a warning (alerting) cue. Accordingly, alerting is often used to support action in safety-critical contexts (e.g., honking to alert others of a traffic situation). Crucially, however, the benefits of alerting for action have been established using laboratory tasks assessing only simple choice reactions. Real-world actions are considerably more complex and mainly consist of sensorimotor sequences of several sub-actions. Therefore, it is still unknown if the benefits of alerting for action transfer from simple choice reactions to such sensorimotor sequences. Here, we investigated how alerting affected performance in a sequential action task derived from the Trail-Making-Test, a well-established neuropsychological test of cognitive action control (Experiment 1). In addition to this task, participants performed a classic alerting paradigm including a simple choice reaction task (Experiment 2). Results showed that alerting sped up responding in both tasks, but in the sequential action task, this benefit was restricted to the first action of a sequence. This was the case, even when multiple actions were performed within a short time (Experiment 3), ruling out that the restriction of alerting to the first action was due to its short-lived nature. Taken together, these findings reveal the existence of an interface between phasic alertness and action control that supports the next action.

Statistical learning of spatiotemporal regularities dynamically guides visual attention across space

In dynamic environments, statistical learning of spatial and temporal regularities guides visual attention in space and time. In the current study, we explored whether and how combined spatiotemporal regularities regarding target events guide visual attention. In three experiments, participants performed the additional singleton task. They were asked to search for a target stimulus with a unique shape among five non-target distractors and respond to the orientation of a line inside the target. Unbeknownst to the participants, the moment in time that the search display was presented was predictive of the target location. Specifically, the target was more likely to be presented at one high-probability location after a short interval and at another high-probability location after a long interval. The results showed that participants' performance was better for high-probability locations than for low-probability locations. Moreover, visual search efficiency was greater when the target appeared at the high-probability location after its associated interval than when it occurred there after its nonassociated interval, regardless of whether the distribution of intervals was uniform (Experiment 1), exponential (Experiment 2), or anti-exponential (Experiment 3). Taken together, the results indicate that implicitly learned spatiotemporal regularities dynamically guide visual attention towards the probable target location.

On doing multi-act arithmetic: A multitrait-multimethod approach of performance dimensions in integrated multitasking

Here we present a systematic plan to the experimental study of test–retest reliability in the multitasking domain, adopting the multitrait-multimethod (MTMM) approach to evaluate the psychometric properties of performance in Düker-type speeded multiple-act mental arithmetic. These form of tasks capacitate the experimental analysis of integrated multi-step processing by combining multiple mental operations in flexible ways in the service of the overarching goal of completing the task. A particular focus was on scoring methodology, particularly measures of response speed variability. To this end, we present data of two experiments with regard to (a) test–retest reliability, (b) between-measures correlational structure, (c) and stability (test–retest practice effects). Finally, we compared participants with high versus low performance variability to assess ability-related differences in measurement precision (typically used as proxy to “simulate” patient populations), which is especially relevant in the applied fields of clinical neuropsychology. The participants performed two classic integrated multi-act arithmetic tasks, combining addition and verification (Exp. 1) and addition and comparison (Exp. 2). The results revealed excellent test–retest reliability for the standard and the variability measures. The analysis of between-measures correlational structure revealed the typical pattern of convergent and discriminant relationships, and also, that absolute response speed variability was highly correlated with average speed (r > 0.85), indicating that these measures mainly deliver redundant information. In contrast, speed-adjusted (relativized) variability revealed discriminant validity being correlated to a much lesser degree with average speed, indicating that this measure delivers additional information not already provided by the speed measure. Furthermore, speed-adjusted variability was virtually unaffected by test–retest practice, which makes this measure interesting in situations with repeated testing.

Revisiting variable-foreperiod effects: evaluating the repetition priming account

A warning signal preceding an imperative stimulus by a certain foreperiod can accelerate responses (foreperiod effect). When foreperiod is varied within a block, the foreperiod effect on reaction time (RT) is modulated by both the current and the prior foreperiods. Using a non-aging foreperiod distribution in a simple-reaction task, Capizzi et al. (Cognition, 134, 39-49, 2015) found equal sequential effects for different foreperiods, which they credited to repetition priming. The multiple-trace theory of Los et al. (Frontiers in Psychology, 5, Article 1058, 2014) attributes the slope of the foreperiod-RT function to the foreperiod distribution. We conducted three experiments that examined these predicted relations. Experiment 1 tested Capizzi et al.'s prediction in a choice-reaction task and found an increasing foreperiod-RT function but a larger sequential effect at the shorter foreperiod. Experiment 2 used two distinct short foreperiods with the same foreperiod distribution and found a decreasing foreperiod-RT function. By increasing the difference between the foreperiods used in Experiment 2, Experiment 3 yielded a larger sequential effect overall. The experiments provide evidence that, with a non-aging foreperiod distribution, the variable-foreperiod paradigm yields unequal sequential-effect sizes at the different foreperiods, consistent with the multiple-trace theory but contrary to Capizzi et al.'s repetition-priming account. The foreperiod-RT functions are similar to those of the fixed-foreperiod paradigm, which is not predicted by the multiple trace theory.

Surprisingly inflexible: Statistically learned suppression of distractors generalizes across contexts

The present study investigates the flexibility of statistically learned distractor suppression between different contexts. Participants performed the additional singleton task searching for a unique shape, while ignoring a uniquely colored distractor. Crucially, we created two contexts within the experiments, and each context was assigned its own high-probability distractor location, so that the location where the distractor was most likely to appear depended on the context. Experiment 1 signified context through the color of the background. In Experiment 2, we aimed to more strongly differentiate between the contexts using an auditory or visual cue to indicate the upcoming context. In Experiment 3, context determined the appropriate response ensuring that participants engaged the context in order to be able to perform the task. Across all experiments, participants learned to suppress both high-probability locations, even if they were not aware of these spatial regularities. However, these suppression effects occurred independent of context, as the pattern of suppression reflected a de-prioritization of both high-probability locations which did not change with the context. We employed Bayesian analyses to statistically quantify the absence of context-dependent suppression effects. We conclude that statistically learned distractor suppression is robust and generalizes across contexts.

Implicitly learning when to be ready: From instances to categories

There is growing appreciation for the role of long-term memory in guiding temporal preparation in speeded reaction time tasks. In experiments with variable foreperiods between a warning stimulus (S1) and a target stimulus (S2), preparation is affected by foreperiod distributions experienced in the past, long after the distribution has changed. These effects from memory can shape preparation largely implicitly, outside of participants' awareness. Recent studies have demonstrated the associative nature of memory-guided preparation. When distinct S1s predict different foreperiods, they can trigger differential preparation accordingly. Here, we propose that memory-guided preparation allows for another key feature of learning: the ability to generalize across acquired associations and apply them to novel situations. Participants completed a variable foreperiod task where S1 was a unique image of either a face or a scene on each trial. Images of either category were paired with different distributions with predominantly shorter versus predominantly longer foreperiods. Participants displayed differential preparation to never-before seen images of either category, without being aware of the predictive nature of these categories. They continued doing so in a subsequent Transfer phase, after they had been informed that these contingencies no longer held. A novel rolling regression analysis revealed at a fine timescale how category-guided preparation gradually developed throughout the task, and that explicit information about these contingencies only briefly disrupted memory-guided preparation. These results offer new insights into temporal preparation as the product of a largely implicit process governed by associative learning from past experiences.

Transfer effects in auditory temporal preparation occur using an unfilled but not filled foreperiod

How quickly participants respond to a “go” after a “warning” signal is partly determined by the time between the two signals (the foreperiod) and the distribution of foreperiods. According to Multiple Trace Theory of Temporal Preparation (MTP), participants use memory traces of previous foreperiods to prepare for the upcoming go signal. If the processes underlying temporal preparation reflect general encoding and memory principles, transfer effects (the carryover effect of a previous block’s distribution of foreperiods to the current block) should be observed regardless of the sensory modality in which signals are presented. Despite convincing evidence for transfer effects in the visual domain, only weak evidence for transfer effects has been documented in the auditory domain. Three experiments were conducted to examine whether such differences in results are due to the modality of the stimulus or other procedural factors. In each experiment, two groups of participants were exposed to different foreperiod distributions in the acquisition phase and to the same foreperiod distribution in the transfer phase. Experiment 1 used a choice-reaction time (RT) task, and the warning signal remained on until the go signal, but there was no evidence for transfer effects. Experiments 2 and 3 used a simple- and choice-RT task, respectively, and there was silence between the warning and go signals. Both experiments revealed evidence for transfer effects, which suggests that transfer effects are most evident when there is no auditory stimulation between the warning and go signals.