Combining spatial and temporal expectations to improve visual perception

Temporal expectancies and rhythmic cueing in touch: The influence of spatial attention

Attention resources can be allocated in both space and time. Exogenous temporal attention can be driven by rhythmic events in our environment which automatically entrain periods of attention. Temporal expectancies can also be generated by the elapse of time, leading to foreperiod effects (the longer between a cue and imperative target, the faster the response). This study investigates temporal attention in touch and the influence of spatial orienting. In experiment 1, participants used bilateral tactile cues to orient endogenous spatial attention to the left or right hand where a unilateral tactile target was presented. This facilitated response times for attended over unattended targets. In experiment 2, the cue was unilateral and non-predictive of the target location resulting in inhibition of return. Importantly, the cue was rhythmic and targets were presented early, in synchrony or late in relation to the rhythmic cue. A foreperiod effect was observed in experiment 1 that was independent from any spatial attention effects. In experiment 2, in synchrony were slower compared to out of synchrony targets but only for cued and not uncued targets, suggesting the rhythm generates periods of exogenous inhibition. Taken together, temporal and spatial attention interact in touch, but only when both types of attention are exogenous. If the task requires endogenous spatial orienting, space and time are independent.

Organizational principles of multidimensional predictions in human auditory attention

Anticipating the future rests upon our ability to exploit contextual cues and to formulate valid internal models or predictions. It is currently unknown how multiple predictions combine to bias perceptual information processing, and in particular whether this is determined by physiological constraints, behavioral relevance (task demands), or past knowledge (perceptual expertise). In a series of behavioral auditory experiments involving musical experts and non-musicians, we investigated the respective and combined contribution of temporal and spectral predictions in multiple detection tasks. We show that temporal and spectral predictions alone systematically increase perceptual sensitivity, independently of task demands or expertise. When combined, however, spectral predictions benefit more to non-musicians and dominate over temporal ones, and the extent of the spectrotemporal synergistic interaction depends on task demands. This suggests that the hierarchy of dominance primarily reflects the tonotopic organization of the auditory system and that expertise or attention only have a secondary modulatory influence.

Not All Predictions Are Equal: "What" and "When" Predictions Modulate Activity in Auditory Cortex through Different Mechanisms

Using predictions based on environmental regularities is fundamental for adaptive behavior. While it is widely accepted that predictions across different stimulus attributes (e.g., time and content) facilitate sensory processing, it is unknown whether predictions across these attributes rely on the same neural mechanism. Here, to elucidate the neural mechanisms of predictions, we combine invasive electrophysiological recordings (human electrocorticography in 4 females and 2 males) with computational modeling while manipulating predictions about content ("what") and time ("when"). We found that "when" predictions increased evoked activity over motor and prefrontal regions both at early (∼180 ms) and late (430-450 ms) latencies. "What" predictability, however, increased evoked activity only over prefrontal areas late in time (420-460 ms). Beyond these dissociable influences, we found that "what" and "when" predictability interactively modulated the amplitude of early (165 ms) evoked responses in the superior temporal gyrus. We modeled the observed neural responses using biophysically realistic neural mass models, to better understand whether "what" and "when" predictions tap into similar or different neurophysiological mechanisms. Our modeling results suggest that "what" and "when" predictability rely on complementary neural processes: "what" predictions increased short-term plasticity in auditory areas, whereas "when" predictability increased synaptic gain in motor areas. Thus, content and temporal predictions engage complementary neural mechanisms in different regions, suggesting domain-specific prediction signaling along the cortical hierarchy. Encoding predictions through different mechanisms may endow the brain with the flexibility to efficiently signal different sources of predictions, weight them by their reliability, and allow for their encoding without mutual interference.SIGNIFICANCE STATEMENT Predictions of different stimulus features facilitate sensory processing. However, it is unclear whether predictions of different attributes rely on similar or different neural mechanisms. By combining invasive electrophysiological recordings of cortical activity with experimental manipulations of participants' predictions about content and time of acoustic events, we found that the two types of predictions had dissociable influences on cortical activity, both in terms of the regions involved and the timing of the observed effects. Further, our biophysical modeling analysis suggests that predictability of content and time rely on complementary neural processes: short-term plasticity in auditory areas and synaptic gain in motor areas, respectively. This suggests that predictions of different features are encoded with complementary neural mechanisms in different brain regions.

Alpha-band functional connectivity during cued versus implicit modality-specific anticipatory attention: EEG-source coherence analysis

The anticipation of future events based on a background experience is one of the main components of any goal-directed behavior. Anticipatory attention can be either voluntary (explicit) or involuntary (implicit). We presumed that these two types of anticipatory attention differed in terms of cortical functional organization. We examined this assumption with an experimental model consisting of three experimental sessions (cued attention, implicit learning, and baseline) that were equal in terms of stimuli, motor responses, and cognitive task. Participants were asked to discriminate the temporal order of stimuli within a pair presented in either the visual or auditory sensory modality. Prestimulus functional connectivity was assessed via alpha-band coherence computed in the source space for preselected regions of interests. Functional links between the cortices of the frontoparietal control system increased during the cued attention condition and did not increase during the implicit anticipation condition. The buildup of implicit anticipation was accompanied by the strengthening of functional links between the intraparietal, ventral premotor, and presupplementary motor areas. It was discovered that both cued and implicit types of anticipation were underlain by functional modality-specific cortical links.

Phasic and sustained interactions of multisensory interplay and temporal expectation

Every moment organisms are confronted with complex streams of information which they use to generate a reliable mental model of the world. There is converging evidence for several optimization mechanisms instrumental in integrating (or segregating) incoming information; among them are multisensory interplay (MSI) and temporal expectation (TE). Both mechanisms can account for enhanced perceptual sensitivity and are well studied in isolation; how these two mechanisms interact is currently less well-known. Here, we tested in a series of four psychophysical experiments for TE effects in uni- and multisensory contexts with different levels of modality-related and spatial uncertainty. We found that TE enhanced perceptual sensitivity for the multisensory relative to the best unisensory condition (i.e. multisensory facilitation according to the max-criterion). In the latter TE effects even vanished if stimulus-related spatial uncertainty was increased. Accordingly, computational modelling indicated that TE, modality-related and spatial uncertainty predict multisensory facilitation. Finally, the analysis of stimulus history revealed that matching expectation at trial n-1 selectively improves multisensory performance irrespective of stimulus-related uncertainty. Together, our results indicate that benefits of multisensory stimulation are enhanced by TE especially in noisy environments, which allows for more robust information extraction to boost performance on both short and sustained time ranges.

The costs and benefits of temporal predictability: impaired inhibition of prepotent responses accompanies increased activation of task-relevant responses

While the benefit of temporal predictability on sensorimotor processing is well established, it is still unknown whether this is due to efficient execution of an appropriate response and/or inhibition of an inappropriate one. To answer this question, we examined the effects of temporal predictability in tasks that required selective (Simon task) or global (Stop-signal task) inhibitory control of prepotent responses. We manipulated temporal expectation by presenting cues that either predicted (temporal cues) or not (neutral cues) when the target would appear. In the Simon task, performance was better when target location (left/right) was compatible with the hand of response and performance was improved further still if targets were temporally cued. However, Conditional Accuracy Functions revealed that temporal predictability selectively increased the number of fast, impulsive errors. Temporal cueing had no effect on selective response inhibition, as measured by the dynamics of the interference effect (delta plots) in the Simon task. By contrast, in the Stop-signal task, Stop-signal reaction time, a covert measure of a more global form of response inhibition, was significantly longer in temporally predictive trials. Therefore, when the time of target onset could be predicted in advance, it was harder to stop the impulse to respond to the target. Collectively, our results indicate that temporal cueing compounded the interfering effects of a prepotent response on task performance. We suggest that although temporal predictability enhances activation of task-relevant responses, it impairs inhibition of prepotent responses.

Attention flexibly trades off across points in time

Sensory signals continuously enter the brain, raising the question of how perceptual systems handle this constant flow of input. Attention to an anticipated point in time can prioritize visual information at that time. However, how we voluntarily attend across time when there are successive task-relevant stimuli has been barely investigated. We developed a novel experimental protocol that allowed us to assess, for the first time, both the benefits and costs of voluntary temporal attention when perceiving a short sequence of two or three visual targets with predictable timing. We found that when humans directed attention to a cued point in time, their ability to perceive orientation was better at that time but also worse earlier and later. These perceptual tradeoffs across time are analogous to those found across space for spatial attention. We concluded that voluntary attention is limited, and selective, across time.

Anticipatory neural dynamics of spatial-temporal orienting of attention in younger and older adults

Spatial and temporal expectations act synergistically to facilitate visual perception. In the current study, we sought to investigate the anticipatory oscillatory markers of combined spatial-temporal orienting and to test whether these decline with ageing. We examined anticipatory neural dynamics associated with joint spatial-temporal orienting of attention using magnetoencephalography (MEG) in both younger and older adults. Participants performed a cued covert spatial-temporal orienting task requiring the discrimination of a visual target. Cues indicated both where and when targets would appear. In both age groups, valid spatial-temporal cues significantly enhanced perceptual sensitivity and reduced reaction times. In the MEG data, the main effect of spatial orienting was the lateralised anticipatory modulation of posterior alpha and beta oscillations. In contrast to previous reports, this modulation was not attenuated in older adults; instead it was even more pronounced. The main effect of temporal orienting was a bilateral suppression of posterior alpha and beta oscillations. This effect was restricted to younger adults. Our results also revealed a striking interaction between anticipatory spatial and temporal orienting in the gamma-band (60-75 Hz). When considering both age groups separately, this effect was only clearly evident and only survived statistical evaluation in the older adults. Together, these observations provide several new insights into the neural dynamics supporting separate as well as combined effects of spatial and temporal orienting of attention, and suggest that different neural dynamics associated with attentional orienting appear differentially sensitive to ageing.

Additive and interactive effects of spatial attention and expectation on perceptual decisions

Spatial attention and expectation are two critical top-down mechanisms controlling perceptual inference. Based on previous research it remains unclear whether their influence on perceptual decisions is additive or interactive. We developed a novel multisensory approach that orthogonally manipulated spatial attention (i.e. task-relevance) and expectation (i.e. signal probability) selectively in audition and evaluated their effects on observers' responses in vision. Critically, while experiment 1 manipulated expectation directly via the probability of task-relevant auditory targets across hemifields, experiment 2 manipulated it indirectly via task-irrelevant auditory non-targets. Surprisingly, our results demonstrate that spatial attention and signal probability influence perceptual decisions either additively or interactively. These seemingly contradictory results can be explained parsimoniously by a model that combines spatial attention, general and spatially selective response probabilities as predictors with no direct influence of signal probability. Our model provides a novel perspective on how spatial attention and expectation facilitate effective interactions with the environment.

Predictive cues for auditory stream formation in humans and monkeys

Auditory perception is improved when stimuli are predictable, and this effect is evident in a modulation of the activity of neurons in the auditory cortex as shown previously. Human listeners can better predict the presence of duration deviants embedded in stimulus streams with fixed interonset interval (isochrony) and repeated duration pattern (regularity), and neurons in the auditory cortex of macaque monkeys have stronger sustained responses in the 60-140 ms post-stimulus time window under these conditions. Subsequently, the question has arisen whether isochrony or regularity in the sensory input contributed to the enhancement of the neuronal and behavioural responses. Therefore, we varied the two factors isochrony and regularity independently and measured the ability of human subjects to detect deviants embedded in these sequences as well as measuring the responses of neurons the primary auditory cortex of macaque monkeys during presentations of the sequences. The performance of humans in detecting deviants was significantly increased by regularity. Isochrony enhanced detection only in the presence of the regularity cue. In monkeys, regularity increased the sustained component of neuronal tone responses in auditory cortex while isochrony had no consistent effect. Although both regularity and isochrony can be considered as parameters that would make a sequence of sounds more predictable, our results from the human and monkey experiments converge in that regularity has a greater influence on behavioural performance and neuronal responses.

Task relevance modulates the behavioural and neural effects of sensory predictions

The brain is thought to generate internal predictions to optimize behaviour. However, it is unclear whether predictions signalling is an automatic brain function or depends on task demands. Here, we manipulated the spatial/temporal predictability of visual targets, and the relevance of spatial/temporal information provided by auditory cues. We used magnetoencephalography (MEG) to measure participants’ brain activity during task performance. Task relevance modulated the influence of predictions on behaviour: spatial/temporal predictability improved spatial/temporal discrimination accuracy, but not vice versa. To explain these effects, we used behavioural responses to estimate subjective predictions under an ideal-observer model. Model-based time-series of predictions and prediction errors (PEs) were associated with dissociable neural responses: predictions correlated with cue-induced beta-band activity in auditory regions and alpha-band activity in visual regions, while stimulus-bound PEs correlated with gamma-band activity in posterior regions. Crucially, task relevance modulated these spectral correlates, suggesting that current goals influence PE and prediction signalling.

As natural environments change, animals need to continuously learn and update predictions about their current context to optimize behaviour. According to predictive coding, a general principle of brain function is the propagation of both neural predictions from hierarchically higher to lower brain regions and of the ensuing prediction-errors back up the cortical hierarchy. We show that the neural activity that signals internal predictions and prediction-errors depends on the current task or goals. We applied magnetoencephalography and computational modelling of behavioural data to a study in which human participants could generate spatial and temporal predictions about upcoming stimuli, while performing spatial or temporal tasks. We found that current context (task relevance) modulated the influence of predictions on behavioural and neural responses. At the level of behavioural responses, only the task-relevant predictions led to improvement in task performance. At the level of neural responses, we found that predictions and prediction-errors correlated with activity in different brain regions and in dissociable frequency bands—reflecting synchronized neural activity. Crucially, these specific neural signatures of prediction and prediction-error signalling were strongly modulated by their contextual relevance. Thus, our results show that current goals influence prediction and prediction-error signalling in the brain.

Temporal alignment of anticipatory motor cortical beta lateralisation in hidden visual-motor sequences

Performance improves when participants respond to events that are structured in repeating sequences, suggesting that learning can lead to proactive anticipatory preparation. Whereas most sequence‐learning studies have emphasised spatial structure, most sequences also contain a prominent temporal structure. We used MEG to investigate spatial and temporal anticipatory neural dynamics in a modified serial reaction time (SRT) task. Performance and brain activity were compared between blocks with learned spatial‐temporal sequences and blocks with new sequences. After confirming a strong behavioural benefit of spatial‐temporal predictability, we show lateralisation of beta oscillations in anticipation of the response associated with the upcoming target location and show that this also aligns to the expected timing of these forthcoming events. This effect was found both when comparing between repeated (learned) and new (unlearned) sequences, as well as when comparing targets that were expected after short vs. long intervals within the repeated (learned) sequence. Our findings suggest that learning of spatial‐temporal structure leads to proactive and dynamic modulation of motor cortical excitability in anticipation of both the location and timing of events that are relevant to guide action.

The role of multisensory interplay in enabling temporal expectations

Temporal regularities can guide our attention to focus on a particular moment in time and to be especially vigilant just then. Previous research provided evidence for the influence of temporal expectation on perceptual processing in unisensory auditory, visual, and tactile contexts. However, in real life we are often exposed to a complex and continuous stream of multisensory events. Here we tested - in a series of experiments - whether temporal expectations can enhance perception in multisensory contexts and whether this enhancement differs from enhancements in unisensory contexts. Our discrimination paradigm contained near-threshold targets (subject-specific 75% discrimination accuracy) embedded in a sequence of distractors. The likelihood of target occurrence (early or late) was manipulated block-wise. Furthermore, we tested whether spatial and modality-specific target uncertainty (i.e. predictable vs. unpredictable target position or modality) would affect temporal expectation (TE) measured with perceptual sensitivity (d^') and response times (RT). In all our experiments, hidden temporal regularities improved performance for expected multisensory targets. Moreover, multisensory performance was unaffected by spatial and modality-specific uncertainty, whereas unisensory TE effects on d^' but not RT were modulated by spatial and modality-specific uncertainty. Additionally, the size of the temporal expectation effect, i.e. the increase in perceptual sensitivity and decrease of RT, scaled linearly with the likelihood of expected targets. Finally, temporal expectation effects were unaffected by varying target position within the stream. Together, our results strongly suggest that participants quickly adapt to novel temporal contexts, that they benefit from multisensory (relative to unisensory) stimulation and that multisensory benefits are maximal if the stimulus-driven uncertainty is highest. We propose that enhanced informational content (i.e. multisensory stimulation) enables the robust extraction of temporal regularities which in turn boost (uni-)sensory representations.

Endogenous temporal attention in the absence of stimulus-driven cues emerges in the second year of life

Anticipating both where and when an object will appear is a critical ability for adaptation. Research in the temporal domain in adults indicate that dissociable mechanisms relate to endogenous attention driven by the properties of the stimulus themselves (e.g. rhythmic, sequential, or trajectory cues) and driven by symbolic cues. In infancy, we know that the capacity to endogenously orient attention progressively develops through infancy. However, the above-mentioned distinction has not yet been explored since previous studies involved stimulus-driven cues. The current study tested 12- and 15-month-olds in an adaptation of the anticipatory eye movement procedure to determine whether infants were able to anticipate a specific location and temporal interval predicted only by symbolic pre-cues. In the absence of stimulus-driven cues, results show that only 15-month-olds could show anticipatory behavior based on the temporal information provided by the symbolic cues. Distinguishing stimulus-driven expectations from those driven by symbolic cues allowed dissecting more clearly the developmental progression of temporal endogenous attention.

Lateralization of spatial rather than temporal attention underlies the left hemifield advantage in rapid serial visual presentation

In bilateral rapid serial visual presentation (RSVP), the second of two targets, T1 and T2, is better identified in the left visual field (LVF) than in the right visual field (RVF). This LVF advantage may reflect hemispheric asymmetry in temporal attention or/and in spatial orienting of attention. Participants performed two tasks: the "standard" bilateral RSVP task (Exp.1) and its unilateral variant (Exp.1 & 2). In the bilateral task, spatial location was uncertain, thus target identification involved stimulus-driven spatial orienting. In the unilateral task, the targets were presented block-wise in the LVF or RVF only, such that no spatial orienting was needed for target identification. Temporal attention was manipulated in both tasks by varying the T1-T2 lag. The results showed that the LVF advantage disappeared when involvement of stimulus-driven spatial orienting was eliminated, whereas the manipulation of temporal attention had no effect on the asymmetry. In conclusion, the results do not support the hypothesis of hemispheric asymmetry in temporal attention, and provide further evidence that the LVF advantage reflects right hemisphere predominance in stimulus-driven orienting of spatial attention. These conclusions fit evidence that temporal attention is implemented by bilateral parietal areas and spatial attention by the right-lateralized ventral frontoparietal network.

Learning temporal context shapes prestimulus alpha oscillations and improves visual discrimination performance

Time is an inseparable component of every physical event that we perceive, yet it is not clear how the brain processes time or how the neuronal representation of time affects our perception of events. Here we asked subjects to perform a visual discrimination task while we changed the temporal context in which the stimuli were presented. We collected electroencephalography (EEG) signals in two temporal contexts. In predictable blocks stimuli were presented after a constant delay relative to a visual cue, and in unpredictable blocks stimuli were presented after variable delays relative to the visual cue. Four subsecond delays of 83, 150, 400, and 800 ms were used in the predictable and unpredictable blocks. We observed that predictability modulated the power of prestimulus alpha oscillations in the parieto-occipital sites: alpha power increased in the 300-ms window before stimulus onset in the predictable blocks compared with the unpredictable blocks. This modulation only occurred in the longest delay period, 800 ms, in which predictability also improved the behavioral performance of the subjects. Moreover, learning the temporal context shaped the prestimulus alpha power: modulation of prestimulus alpha power grew during the predictable block and correlated with performance enhancement. These results suggest that the brain is able to learn the subsecond temporal context of stimuli and use this to enhance sensory processing. Furthermore, the neural correlate of this temporal prediction is reflected in the alpha oscillations.NEW & NOTEWORTHY It is not well understood how the uncertainty in the timing of an external event affects its processing, particularly at subsecond scales. Here we demonstrate how a predictable timing scheme improves visual processing. We found that learning the predictable scheme gradually shaped the prestimulus alpha power. These findings indicate that the human brain is able to extract implicit subsecond patterns in the temporal context of events.

Temporal Expectations Guide Dynamic Prioritization in Visual Working Memory through Attenuated α Oscillations

Although working memory is generally considered a highly dynamic mnemonic store, popular laboratory tasks used to understand its psychological and neural mechanisms (such as change detection and continuous reproduction) often remain relatively “static,” involving the retention of a set number of items throughout a shared delay interval. In the current study, we investigated visual working memory in a more dynamic setting, and assessed the following: (1) whether internally guided temporal expectations can dynamically and reversibly prioritize individual mnemonic items at specific times at which they are deemed most relevant; and (2) the neural substrates that support such dynamic prioritization. Participants encoded two differently colored oriented bars into visual working memory to retrieve the orientation of one bar with a precision judgment when subsequently probed. To test for the flexible temporal control to access and retrieve remembered items, we manipulated the probability for each of the two bars to be probed over time, and recorded EEG in healthy human volunteers. Temporal expectations had a profound influence on working memory performance, leading to faster access times as well as more accurate orientation reproductions for items that were probed at expected times. Furthermore, this dynamic prioritization was associated with the temporally specific attenuation of contralateral α (8–14 Hz) oscillations that, moreover, predicted working memory access times on a trial-by-trial basis. We conclude that attentional prioritization in working memory can be dynamically steered by internally guided temporal expectations, and is supported by the attenuation of α oscillations in task-relevant sensory brain areas.

SIGNIFICANCE STATEMENT In dynamic, everyday-like, environments, flexible goal-directed behavior requires that mental representations that are kept in an active (working memory) store are dynamic, too. We investigated working memory in a more dynamic setting than is conventional, and demonstrate that expectations about when mnemonic items are most relevant can dynamically and reversibly prioritize these items in time. Moreover, we uncover a neural substrate of such dynamic prioritization in contralateral visual brain areas and show that this substrate predicts working memory retrieval times on a trial-by-trial basis. This places the experimental study of working memory, and its neuronal underpinnings, in a more dynamic and ecologically valid context, and provides new insights into the neural implementation of attentional prioritization within working memory.

Early behavioural facilitation by temporal expectations in complex visual-motor sequences

•

We show incidental spatial-temporal sequence learning in an adapted SRT task.

•

Incidentally acquired temporal expectations have the largest effect for short intervals.

•

The facilitation for short intervals mirrors explicit temporal orienting results.

In daily life, temporal expectations may derive from incidental learning of recurring patterns of intervals. We investigated the incidental acquisition and utilisation of combined temporal-ordinal (spatial/effector) structure in complex visual-motor sequences using a modified version of a serial reaction time (SRT) task. In this task, not only the series of targets/responses, but also the series of intervals between subsequent targets was repeated across multiple presentations of the same sequence. Each participant completed three sessions. In the first session, only the repeating sequence was presented. During the second and third session, occasional probe blocks were presented, where a new (unlearned) spatial-temporal sequence was introduced. We first confirm that participants not only got faster over time, but that they were slower and less accurate during probe blocks, indicating that they incidentally learned the sequence structure. Having established a robust behavioural benefit induced by the repeating spatial-temporal sequence, we next addressed our central hypothesis that implicit temporal orienting (evoked by the learned temporal structure) would have the largest influence on performance for targets following short (as opposed to longer) intervals between temporally structured sequence elements, paralleling classical observations in tasks using explicit temporal cues. We found that indeed, reaction time differences between new and repeated sequences were largest for the short interval, compared to the medium and long intervals, and that this was the case, even when comparing late blocks (where the repeated sequence had been incidentally learned), to early blocks (where this sequence was still unfamiliar). We conclude that incidentally acquired temporal expectations that follow a sequential structure can have a robust facilitatory influence on visually-guided behavioural responses and that, like more explicit forms of temporal orienting, this effect is most pronounced for sequence elements that are expected at short inter-element intervals.

Neural mechanisms of rhythm-based temporal prediction: Delta phase-locking reflects temporal predictability but not rhythmic entrainment

Predicting the timing of upcoming events enables efficient resource allocation and action preparation. Rhythmic streams, such as music, speech, and biological motion, constitute a pervasive source for temporal predictions. Widely accepted entrainment theories postulate that rhythm-based predictions are mediated by synchronizing low-frequency neural oscillations to the rhythm, as indicated by increased phase concentration (PC) of low-frequency neural activity for rhythmic compared to random streams. However, we show here that PC enhancement in scalp recordings is not specific to rhythms but is observed to the same extent in less periodic streams if they enable memory-based prediction. This is inconsistent with the predictions of a computational entrainment model of stronger PC for rhythmic streams. Anticipatory change in alpha activity and facilitation of electroencephalogram (EEG) manifestations of response selection are also comparable between rhythm- and memory-based predictions. However, rhythmic sequences uniquely result in obligatory depression of preparation-related premotor brain activity when an on-beat event is omitted, even when it is strategically beneficial to maintain preparation, leading to larger behavioral costs for violation of prediction. Thus, while our findings undermine the validity of PC as a sign of rhythmic entrainment, they constitute the first electrophysiological dissociation, to our knowledge, between mechanisms of rhythmic predictions and of memory-based predictions: the former obligatorily lead to resonance-like preparation patterns (that are in line with entrainment), while the latter allow flexible resource allocation in time regardless of periodicity in the input. Taken together, they delineate the neural mechanisms of three distinct modes of preparation: continuous vigilance, interval-timing-based prediction and rhythm-based prediction.

"Nonspatial" Attentional Deficits Interact with Spatial Position in Neglect

A widely debated question concerns whether or not spatial and nonspatial components of visual attention interact in attentional performance. Spatial neglect is a common consequence of brain injury where individuals fail to respond to stimuli presented on their contralesional side. It has been argued that, beyond the spatial bias, these individuals also tend to exhibit nonspatial perceptual deficits. Here we demonstrate that the "nonspatial" deficits affecting the temporal dynamics of attentional deployment are in fact modulated by spatial position. Specifically, we observed that the pathological attentional blink of chronic neglect is enhanced when stimuli are presented on the contralesional side of the trunk while keeping retinal and head-centered coordinates constant. We did not find this pattern in right brain-damaged patients without neglect or in patients who had recovered from neglect. Our work suggests that the nonspatial attentional deficits observed in neglect are heavily modulated by egocentric spatial position. This provides strong evidence against models that suggest independent modules for spatial and nonspatial attentional functions while also providing strong evidence that trunk position plays an important role in neglect.

Automated symbolic orienting is not modulated by explicit temporal attention

Recent studies show that spatial attention is uniquely engaged by the selection history of a stimulus. One example of this process is Automated Symbolic Orienting, which is thought to reflect overlearned spatial links between a behaviorally relevant stimulus and a target event. However, since automated symbolic effects have been found to vary with temporal expectancies about when a target might occur, it is possible that this spatial effect may also depend on processing resources associated with voluntary temporal attention. To test this idea, here we elicited automated symbolic orienting and voluntary temporal attention in isolation and in combination. Across all conditions, both types of orienting remained typical without interacting. Thus, typical automated symbolic orienting is not modulated by participants' explicit utilization of temporal information; however, and as we have shown previously, typical ASO does appear to require the presence of an implicit temporal structure within a task.

Power and Phase of Alpha Oscillations Reveal an Interaction between Spatial and Temporal Visual Attention

Oscillatory brain rhythms can bias attention via phase and amplitude changes, which modulate sensory activity, biasing information to be processed or ignored. Alpha band (7-14 Hz) oscillations lateralize with spatial attention and rhythmically inhibit visual activity and awareness through pulses of inhibition. Here we show that human observers' awareness of spatially unattended targets is dependent on both alpha power and alpha phase at target onset. Following a predictive directional cue, alpha oscillations were entrained bilaterally using repetitive visual stimuli. Subsequently, we presented either spatially cued or uncued targets at SOAs either validly or invalidly predicted in time by the entrainers. Temporal validity maximally modulated perceptual performance outside the spatial focus of attention and was associated with both increased alpha power and increased neural entrainment of phase in the hemisphere processing spatially unattended information. The results demonstrate that alpha oscillations represent a pulsating inhibition, which impedes visual processing for unattended space.

Temporal orienting of attention can be preserved in normal aging

Being able to orient our attention to moments in time is crucial for optimizing behavioral performance. In young adults, flexible cue-based temporal expectations have been shown to modulate perceptual functions and enhance behavioral performance. Recent studies with older individuals have reported significant deficits in cued temporal orienting. To investigate the extent of these deficits, the authors conducted 3 studies in healthy old and young adults. For each study, participants completed 2 tasks: a reaction time (RT) task that emphasized speeded responding and a nonspeeded rapid-serial-visual-presentation task that emphasized visual discrimination. Auditory cues indicated the likelihood of a target item occurring after a short or long temporal interval (foreperiod; 75% validity). In the first study, cues indicating a short or a long foreperiod were manipulated across blocks. The second study was designed to replicate and extend the first study by manipulating the predictive temporal cues on a trial-by-trial basis. The third study extended the findings by including neutral cues so that it was possible to separate cueing validity benefits and invalidity costs. In all 3 studies, cued temporal expectation conferred significant performance advantages for target stimuli occurring after the short foreperiod for both old and young participants. Contrary to previous findings, these results suggest that the ability to allocate attention to moments in time can be preserved in healthy aging. Further research is needed to ascertain whether similar neural networks are used to orient attention in time as we age, and/or whether compensatory mechanisms are at work in older individuals. (PsycINFO Database Record

Temporal Regularity of the Environment Drives Time Perception

It's reasonable to assume that a regularly paced sequence should be perceived as regular, but here we show that perceived regularity depends on the context in which the sequence is embedded. We presented one group of participants with perceptually regularly paced sequences, and another group of participants with mostly irregularly paced sequences (75% irregular, 25% regular). The timing of the final stimulus in each sequence could be varied. In one experiment, we asked whether the last stimulus was regular or not. We found that participants exposed to an irregular environment frequently reported perfectly regularly paced stimuli to be irregular. In a second experiment, we asked participants to judge whether the final stimulus was presented before or after a flash. In this way, we were able to determine distortions in temporal perception as changes in the timing necessary for the sound and the flash to be perceived synchronous. We found that within a regular context, the perceived timing of deviant last stimuli changed so that the relative anisochrony appeared to be perceptually decreased. In the irregular context, the perceived timing of irregular stimuli following a regular sequence was not affected. These observations suggest that humans use temporal expectations to evaluate the regularity of sequences and that expectations are combined with sensory stimuli to adapt perceived timing to follow the statistics of the environment. Expectations can be seen as a-priori probabilities on which perceived timing of stimuli depend.

Temporal Prediction in lieu of Periodic Stimulation

Predicting not only what will happen, but also when it will happen is extremely helpful for optimizing perception and action. Temporal predictions driven by periodic stimulation increase perceptual sensitivity and reduce response latencies. At the neurophysiological level, a single mechanism has been proposed to mediate this twofold behavioral improvement: the rhythmic entrainment of slow cortical oscillations to the stimulation rate. However, temporal regularities can occur in aperiodic contexts, suggesting that temporal predictions per se may be dissociable from entrainment to periodic sensory streams. We investigated this possibility in two behavioral experiments, asking human participants to detect near-threshold auditory tones embedded in streams whose temporal and spectral properties were manipulated. While our findings confirm that periodic stimulation reduces response latencies, in agreement with the hypothesis of a stimulus-driven entrainment of neural excitability, they further reveal that this motor facilitation can be dissociated from the enhancement of auditory sensitivity. Perceptual sensitivity improvement is unaffected by the nature of temporal regularities (periodic vs aperiodic), but contingent on the co-occurrence of a fulfilled spectral prediction. Altogether, the dissociation between predictability and periodicity demonstrates that distinct mechanisms flexibly and synergistically operate to facilitate perception and action.

Endogenous temporal and spatial orienting: Evidence for two distinct attentional mechanisms

The requirement to orient attention in space and time usually occurs simultaneously. Previous reports were indecisive regarding possible interactions between temporal and spatial orienting. The present study examined whether temporal and spatial orienting can operate simultaneously and independently in the framework of a detection task. Participants completed three consecutive target detection tasks: in the first two tasks a central cue provided predictive information regarding either the temporal delay of the target or its spatial location. In a third task the temporal and spatial cues from the first two tasks were combined into a single cue. Temporal and spatial information provided by the combined cue could be valid or invalid for each type of information separately. Results from the combined temporal-spatial task revealed that at a short cue-to-target interval temporal validity effects were significant at the attended and unattended spatial locations and were not modulated by spatial validity conditions. Spatial validity effects were also significant and comparable between the valid and invalid temporal conditions. Moreover, temporal and spatial validity effects in the combined task were equivalent to those attained in the separate tasks. At a long cue-to-target delay, spatial validity effects were significant and were not modulated by temporal validity but there were no temporal validity effects. Overall, the results suggest that participants were able to extract temporal and spatial information provided by a single cue simultaneously and independently. We conclude that temporal and spatial endogenous orienting function orthogonally in a task that does not require demanding perceptual discrimination.

Temporal Attention as a Scaffold for Language Development

Language is one of the most fascinating abilities that humans possess. Infants demonstrate an amazing repertoire of linguistic abilities from very early on and reach an adult-like form incredibly fast. However, language is not acquired all at once but in an incremental fashion. In this article we propose that the attentional system may be one of the sources for this developmental trajectory in language acquisition. At birth, infants are endowed with an attentional system fully driven by salient stimuli in their environment, such as prosodic information (e.g., rhythm or pitch). Early stages of language acquisition could benefit from this readily available, stimulus-driven attention to simplify the complex speech input and allow word segmentation. At later stages of development, infants are progressively able to selectively attend to specific elements while disregarding others. This attentional ability could allow them to learn distant non-adjacent rules needed for morphosyntactic acquisition. Because non-adjacent dependencies occur at distant moments in time, learning these dependencies may require correctly orienting attention in the temporal domain. Here, we gather evidence uncovering the intimate relationship between the development of attention and language. We aim to provide a novel approach to human development, bridging together temporal attention and language acquisition.

Time in Cortical Circuits

Time is central to cognition. However, the neural basis for time-dependent cognition remains poorly understood. We explore how the temporal features of neural activity in cortical circuits and their capacity for plasticity can contribute to time-dependent cognition over short time scales. This neural activity is linked to cognition that operates in the present or anticipates events or stimuli in the near future. We focus on deliberation and planning in the context of decision making as a cognitive process that integrates information across time. We progress to consider how temporal expectations of the future modulate perception. We propose that understanding the neural basis for how the brain tells time and operates in time will be necessary to develop general models of cognition.

SIGNIFICANCE STATEMENT

Time is central to cognition. However, the neural basis for time-dependent cognition remains poorly understood. We explore how the temporal features of neural activity in cortical circuits and their capacity for plasticity can contribute to time-dependent cognition over short time scales. We propose that understanding the neural basis for how the brain tells time and operates in time will be necessary to develop general models of cognition.

Visual Benefits in Apparent Motion Displays: Automatically Driven Spatial and Temporal Anticipation Are Partially Dissociated

Many behaviourally relevant sensory events such as motion stimuli and speech have an intrinsic spatio-temporal structure. This will engage intentional and most likely unintentional (automatic) prediction mechanisms enhancing the perception of upcoming stimuli in the event stream. Here we sought to probe the anticipatory processes that are automatically driven by rhythmic input streams in terms of their spatial and temporal components. To this end, we employed an apparent visual motion paradigm testing the effects of pre-target motion on lateralized visual target discrimination. The motion stimuli either moved towards or away from peripheral target positions (valid vs. invalid spatial motion cueing) at a rhythmic or arrhythmic pace (valid vs. invalid temporal motion cueing). Crucially, we emphasized automatic motion-induced anticipatory processes by rendering the motion stimuli non-predictive of upcoming target position (by design) and task-irrelevant (by instruction), and by creating instead endogenous (orthogonal) expectations using symbolic cueing. Our data revealed that the apparent motion cues automatically engaged both spatial and temporal anticipatory processes, but that these processes were dissociated. We further found evidence for lateralisation of anticipatory temporal but not spatial processes. This indicates that distinct mechanisms may drive automatic spatial and temporal extrapolation of upcoming events from rhythmic event streams. This contrasts with previous findings that instead suggest an interaction between spatial and temporal attention processes when endogenously driven. Our results further highlight the need for isolating intentional from unintentional processes for better understanding the various anticipatory mechanisms engaged in processing behaviourally relevant stimuli with predictable spatio-temporal structure such as motion and speech.

Preserved memory-based orienting of attention with impaired explicit memory in healthy ageing

It is increasingly recognised that spatial contextual long-term memory (LTM) prepares neural activity for guiding visuo-spatial attention in a proactive manner. In the current study, we investigated whether the decline in explicit memory observed in healthy ageing would compromise this mechanism. We compared the behavioural performance of younger and older participants on learning new contextual memories, on orienting visual attention based on these learnt contextual associations, and on explicit recall of contextual memories. We found a striking dissociation between older versus younger participants in the relationship between the ability to retrieve contextual memories versus the ability to use these to guide attention to enhance performance on a target-detection task. Older participants showed significant deficits in the explicit retrieval task, but their behavioural benefits from memory-based orienting of attention were equivalent to those in young participants. Furthermore, memory-based orienting correlated significantly with explicit contextual LTM in younger adults but not in older adults. These results suggest that explicit memory deficits in ageing might not compromise initial perception and encoding of events. Importantly, the results also shed light on the mechanisms of memory-guided attention, suggesting that explicit contextual memories are not necessary.

Expectations developed over multiple timescales facilitate visual search performance

Our perception of the world is strongly influenced by our expectations, and a question of key importance is how the visual system develops and updates its expectations through interaction with the environment. We used a visual search task to investigate how expectations of different timescales (from the last few trials to hours to long-term statistics of natural scenes) interact to alter perception. We presented human observers with low-contrast white dots at 12 possible locations equally spaced on a circle, and we asked them to simultaneously identify the presence and location of the dots while manipulating their expectations by presenting stimuli at some locations more frequently than others. Our findings suggest that there are strong acuity differences between absolute target locations (e.g., horizontal vs. vertical) and preexisting long-term biases influencing observers' detection and localization performance, respectively. On top of these, subjects quickly learned about the stimulus distribution, which improved their detection performance but caused increased false alarms at the most frequently presented stimulus locations. Recent exposure to a stimulus resulted in significantly improved detection performance and significantly more false alarms but only at locations at which it was more probable that a stimulus would be presented. Our results can be modeled and understood within a Bayesian framework in terms of a near-optimal integration of sensory evidence with rapidly learned statistical priors, which are skewed toward the very recent history of trials and may help understanding the time scale of developing expectations at the neural level.

Temporal prediction errors modulate task-switching performance

We have previously shown that temporal prediction errors (PEs, the differences between the expected and the actual stimulus’ onset times) modulate the effective connectivity between the anterior cingulate cortex and the right anterior insular cortex (rAI), causing the activity of the rAI to decrease. The activity of the rAI is associated with efficient performance under uncertainty (e.g., changing a prepared behavior when a change demand is not expected), which leads to hypothesize that temporal PEs might disrupt behavior-change performance under uncertainty. This hypothesis has not been tested at a behavioral level. In this work, we evaluated this hypothesis within the context of task switching and concurrent temporal predictions. Our participants performed temporal predictions while observing one moving ball striking a stationary ball which bounced off with a variable temporal gap. Simultaneously, they performed a simple color comparison task. In some trials, a change signal made the participants change their behaviors. Performance accuracy decreased as a function of both the temporal PE and the delay. Explaining these results without appealing to ad hoc concepts such as “executive control” is a challenge for cognitive neuroscience. We provide a predictive coding explanation. We hypothesize that exteroceptive and proprioceptive minimization of PEs would converge in a fronto-basal ganglia network which would include the rAI. Both temporal gaps (or uncertainty) and temporal PEs would drive and modulate this network respectively. Whereas the temporal gaps would drive the activity of the rAI, the temporal PEs would modulate the endogenous excitatory connections of the fronto-striatal network. We conclude that in the context of perceptual uncertainty, the system is not able to minimize perceptual PE, causing the ongoing behavior to finalize and, in consequence, disrupting task switching.

Developmental Trajectories of Internally and Externally Driven Temporal Prediction

The ability to generate temporal prediction (TP) is fundamental to our survival since it allows us to selectively orient our attention in time in order to prioritize relevant environmental information. Studies on adult participants showed that externally and internally driven mechanisms can be engaged to establish TP, both resulting in better behavioural performance. However, few studies on children have investigated the ability to engage internally and externally driven TP, especially in relation to how these mechanisms change across development. In this study, 111 participants (88 children between six and eleven years of age, and 23 adults) were tested by means of a simple reaction time paradigm, in which temporal cueing and neutral conditions were orthogonally manipulated to induce externally and internally driven TP mechanisms, as well as an interaction between the two. Sequential effects (SEs) relative to both tasks were also investigated. Results showed that all children participating in the study were able to implement both external and internal TP in an independent fashion. However, children younger than eight years were not able to combine both strategies. Furthermore, in the temporal cueing blocks they did not show the typically-observed asymmetric SE pattern. These results suggest that children can flexibly use both external and internal TP mechanisms to optimise their behaviour, although their successful combined use develops only after eight years of age.

Top-down control of the phase of alpha-band oscillations as a mechanism for temporal prediction

The physiological state of the brain before an incoming stimulus has substantial consequences for subsequent behavior and neural processing. For example, the phase of ongoing posterior alpha-band oscillations (8-14 Hz) immediately before visual stimulation has been shown to predict perceptual outcomes and downstream neural activity. Although this phenomenon suggests that these oscillations may phasically route information through functional networks, many accounts treat these periodic effects as a consequence of ongoing activity that is independent of behavioral strategy. Here, we investigated whether alpha-band phase can be guided by top-down control in a temporal cueing task. When participants were provided with cues predictive of the moment of visual target onset, discrimination accuracy improved and targets were more frequently reported as consciously seen, relative to unpredictive cues. This effect was accompanied by a significant shift in the phase of alpha-band oscillations, before target onset, toward each participant's optimal phase for stimulus discrimination. These findings provide direct evidence that forming predictions about when a stimulus will appear can bias the phase of ongoing alpha-band oscillations toward an optimal phase for visual processing, and may thus serve as a mechanism for the top-down control of visual processing guided by temporal predictions.

Neuronal oscillations as a mechanistic substrate of auditory temporal prediction

Neuronal oscillations are comprised of rhythmic fluctuations of excitability that are synchronized in ensembles of neurons and thus function as temporal filters that dynamically organize sensory processing. When perception relies on anticipatory mechanisms, ongoing oscillations also provide a neurophysiological substrate for temporal prediction. In this article, we review evidence for this account with a focus on auditory perception. We argue that such "oscillatory temporal predictions" can selectively amplify neuronal sensitivity to inputs that occur in a predicted, task-relevant rhythm and optimize temporal selection. We elaborate this argument for a prototypic example, speech processing, where information is present at multiple time scales, with delta, theta, and low-gamma oscillations being specifically and simultaneously engaged, enabling multiplexing. We then consider the origin of temporal predictions, specifically the idea that the motor system is involved in the generation of such prior information. Finally, we place temporal predictions in the general context of internal models, discussing how they interact with feature-based or spatial predictions. We propose that complementary predictions interact synergistically according to a dominance hierarchy, shaping perception in the form of a multidimensional filter mechanism.

Intersensory selective attention and temporal orienting operate in parallel and are instantiated in spatially distinct sensory and motor cortices

Knowledge about the sensory modality in which a forthcoming event might occur permits anticipatory intersensory attention. Information as to when exactly an event occurs enables temporal orienting. Intersensory and temporal attention mechanisms are often deployed simultaneously, but as yet it is unknown whether these processes operate interactively or in parallel. In this human electroencephalography study, we manipulated intersensory attention and temporal orienting in the same paradigm. A continuous stream of bisensory visuo-tactile inputs was presented, and a preceding auditory cue indicated to which modality participants should attend (visual or tactile). Temporal orienting was manipulated blockwise by presenting stimuli either at regular or irregular intervals. Using linear beamforming, we examined neural oscillations at virtual channels in sensory and motor cortices. Both attentional processes simultaneously modulated the power of anticipatory delta- and beta-band oscillations, as well as delta-band phase coherence. Modulations in sensory cortices reflected intersensory attention, indicative of modality-specific gating mechanisms. Modulations in motor and partly in somatosensory cortex reflected temporal orienting, indicative of a supramodal preparatory mechanism. We found no evidence for interactions between intersensory attention and temporal orienting, suggesting that these two mechanisms act in parallel and largely independent of each other in sensory and motor cortices.

Children can implicitly, but not voluntarily, direct attention in time

Children are able to use spatial cues to orient their attention to discrete locations in space from around 4 years of age. In contrast, no research has yet investigated the ability of children to use informative cues to voluntarily predict when an event will occur in time. The spatial and temporal attention task was used to determine whether children were able to voluntarily orient their attention in time, as well as in space: symbolic spatial and temporal cues predicted where or when an imperative target would appear. Thirty typically developing children (average age 11 yrs) and 32 adults (average age 27 yrs) took part. Confirming previous findings, adults made use of both spatial and temporal cues to optimise behaviour, and were significantly slower to respond to invalidly cued targets in either space or time. Children were also significantly slowed by invalid spatial cues, demonstrating their use of spatial cues to guide expectations. In contrast, children’s responses were not slowed by invalid temporal cues, suggesting that they were not using the temporal cue to voluntarily orient attention through time. Children, as well as adults, did however demonstrate signs of more implicit forms of temporal expectation: RTs were faster for long versus short cue-target intervals (the variable foreperiod effect) and slower when the preceding trial’s cue-target interval was longer than that on the current trial (sequential effects). Overall, our results suggest that although children implicitly made use of the temporally predictive information carried by the length of the current and previous trial’s cue-target interval, they could not deliberately use symbolic temporal cues to speed responses. The developmental trajectory of the ability to voluntarily use symbolic temporal cues is therefore delayed, relative both to the use of symbolic (arrow) spatial cues, and to the use of implicit temporal information.

Predictive motor control of sensory dynamics in auditory active sensing

Neuronal oscillations present potential physiological substrates for brain operations that require temporal prediction. We review this idea in the context of auditory perception. Using speech as an exemplar, we illustrate how hierarchically organized oscillations can be used to parse and encode complex input streams. We then consider the motor system as a major source of rhythms (temporal priors) in auditory processing, that act in concert with attention to sharpen sensory representations and link them across areas. We discuss the circuits that could mediate this audio-motor interaction, notably the potential role of the somatosensory system. Finally, we reposition temporal predictions in the context of internal models, discussing how they interact with feature-based or spatial predictions. We argue that complementary predictions interact synergistically according to the organizational principles of each sensory system, forming multidimensional filters crucial to perception.

High-speed incoming infrared target detection by fusion of spatial and temporal detectors

This paper presents a method for detecting high-speed incoming targets by the fusion of spatial and temporal detectors to achieve a high detection rate for an active protection system (APS). The incoming targets have different image velocities according to the target-camera geometry. Therefore, single-target detector-based approaches, such as a 1D temporal filter, 2D spatial filter and 3D matched filter, cannot provide a high detection rate with moderate false alarms. The target speed variation was analyzed according to the incoming angle and target velocity. The speed of the distant target at the firing time is almost stationary and increases slowly. The speed varying targets are detected stably by fusing the spatial and temporal filters. The stationary target detector is activated by an almost zero temporal contrast filter (TCF) and identifies targets using a spatial filter called the modified mean subtraction filter (M-MSF). A small motion (sub-pixel velocity) target detector is activated by a small TCF value and finds targets using the same spatial filter. A large motion (pixel-velocity) target detector works when the TCF value is high. The final target detection is terminated by fusing the three detectors based on the threat priority. The experimental results of the various target sequences show that the proposed fusion-based target detector produces the highest detection rate with an acceptable false alarm rate.

Independent effects of bottom-up temporal expectancy and top-down spatial attention. An audiovisual study using rhythmic cueing

Selective attention to a spatial location has shown enhanced perception and facilitate behavior for events at attended locations. However, selection relies not only on where but also when an event occurs. Recently, interest has turned to how intrinsic neural oscillations in the brain entrain to rhythms in our environment, and, stimuli appearing in or out of sync with a rhythm have shown to modulate perception and performance. Temporal expectations created by rhythms and spatial attention are two processes which have independently shown to affect stimulus processing but it remains largely unknown how, and if, they interact. In four separate tasks, this study investigated the effects of voluntary spatial attention and bottom-up temporal expectations created by rhythms in both unimodal and crossmodal conditions. In each task the participant used an informative cue, either color or pitch, to direct their covert spatial attention to the left or right, and respond as quickly as possible to a target. The lateralized target (visual or auditory) was then presented at the attended or unattended side. Importantly, although not task relevant, the cue was a rhythm of either flashes or beeps. The target was presented in or out of sync (early or late) with the rhythmic cue. Results showed participants were faster responding to spatially attended compared to unattended targets in all tasks. Moreover, there was an effect of rhythmic cueing upon response times in both unimodal and crossmodal conditions. Responses were faster to targets presented in sync with the rhythm compared to when they appeared too early in both crossmodal tasks. That is, rhythmic stimuli in one modality influenced the temporal expectancy in the other modality, suggesting temporal expectancies created by rhythms are crossmodal. Interestingly, there was no interaction between top-down spatial attention and rhythmic cueing in any task suggesting these two processes largely influenced behavior independently.