Accounting for memory mechanisms in interval timing: a review

Motor variability modulates calibration of precisely timed movements

Interacting with the environment often requires precisely timed movements, challenging the brain to minimize the detrimental impact of neural noise. Recent research demonstrates that the brain exploits the variability of its temporal estimates and recalibrates perception accordingly. Time-critical movements, however, contain a sensory measurement and a motor stage. The brain must have knowledge of both in order to avoid maladapted behavior. By manipulating sensory and motor variability, we show that the sensorimotor system recalibrates sensory and motor uncertainty separately. Serial dependencies between observed interval durations in the previous and motor reproductions in the current trial were weighted by the variability of movements. These serial dependencies generalized across different effectors, but not to a visual discrimination task. Our results suggest that the brain has accurate knowledge about contributions of motor uncertainty to errors in temporal movements. This knowledge about motor uncertainty seems to be processed separately from knowledge about sensory uncertainty.

Modality-specific sensory and decisional carryover effects in duration perception

BACKGROUND

The brain uses recent history when forming perceptual decisions. This results in carryover effects in perception. Although separate sensory and decisional carryover effects have been shown in many perceptual tasks, their existence and nature in temporal processing are unclear. Here, we investigated whether and how previous stimuli and previous choices affect subsequent duration perception, in vision and audition.

RESULTS

In a series of three experiments, participants were asked to classify visual or auditory stimuli into "shorter" or "longer" duration categories. In experiment 1, visual and auditory stimuli were presented in separate blocks. Results showed that current duration estimates were repelled away from the previous trial's stimulus duration, but attracted towards the previous choice, in both vision and audition. In experiment 2, visual and auditory stimuli were pseudorandomly presented in one block. We found that sensory and decisional carryover effects occurred only when previous and current stimuli were from the same modality. Experiment 3 further investigated the stimulus dependence of carryover effects within each modality. In this experiment, visual stimuli with different shape topologies (or auditory stimuli with different audio frequencies) were pseudorandomly presented in one visual (or auditory) block. Results demonstrated sensory carryover (within each modality) despite task-irrelevant differences in visual shape topology or audio frequency. By contrast, decisional carryover was reduced (but still present) across different visual topologies and completely absent across different audio frequencies.

CONCLUSIONS

These results suggest that serial dependence in duration perception is modality-specific. Moreover, repulsive sensory carryover effects generalize within each modality, whereas attractive decisional carryover effects are contingent on contextual details.

Duration discrimination: A diffusion decision modeling approach

The human ability to discriminate the duration of two subsequently presented stimuli is often studied with tasks that involve a comparison between a standard stimulus (with fixed duration) and comparison stimuli (with varying durations). The performance in such tasks is influenced by the presentation order of these successively presented stimuli. The so-called Type A effect refers to the impact of presentation order on the point of subjective equality. The Type B effect describes effects of presentation order on the just-noticeable-difference. Cognitive models that account for these context effects assume that participants' duration estimation is influenced by the history of previously encountered stimuli. For example, the internal reference model assumes that the magnitude of a "typical" stimulus is represented by an internal reference. This internal reference evolves throughout an experiment and is updated on every trial. Different recent models have in common that they describe how the internal reference is computed but are agnostic to the decision process itself. In this study, we develop a new model that incorporates the mechanisms of perceptual discrimination models into a diffusion model. The diffusion model focuses on the dynamics of the decision process itself and accounts for choice and response times based on a set of latent cognitive variables. We show that our model accurately predicts the accuracy and response time distribution in a classical duration discrimination task. Further, model parameters were sensitive to the Type A and B effect. The proposed model opens up new opportunities for studying human discrimination performance (e.g., individual differences).

Training the attentional blink: subclinical depression decreases learning potential

The attentional blink (AB) reflects a temporal restriction of selective attention and is generally regarded as a very robust phenomenon. However, previous studies have found large individual differences in AB performance, and under some training conditions the AB can be reduced significantly. One factor that may account for individual differences in AB magnitude is the ability to accurately time attention. In the current study, we focus on the sensitivity for temporal information on the ability to control attention. Following a visual AB task, a time estimation task was presented in either the visual or auditory modality, followed by another visual AB task. It was found that the time estimation training in both the auditory and visual modality reduced AB magnitude. Although a reduction in AB magnitude was also observed when individuals were trained on a control task (either an auditory frequency or visual line length estimation task), the effect was significantly larger following the time estimation tasks. In addition, it was found that individuals who showed most improvement on the visual time estimation task, also showed the largest reduction in AB magnitude, which was not the case for individuals who were trained on the control tasks. Finally, a negative correlation was observed between depression scores (tested by Beck Depression Inventory-Short Form (BDI-SF) scores and the improvement in the AB and time estimation tasks. Our findings demonstrate clear links between timing ability and mechanisms to control attention and emotion.

Time changes: Timing contexts support event segmentation in associative memory

We tend to mentally segment a series of events according to perceptual contextual changes, such that items from a shared context are more strongly associated in memory than items from different contexts. It is also known that timing context provides a scaffold to structure experiences in memory, but its role in event segmentation has not been investigated. We adapted a previous paradigm, which was used to investigate event segmentation using visual contexts, to study the effects of changes in timing contexts on event segmentation in associative memory. In two experiments, we presented lists of 36 items in which the interstimulus intervals (ISIs) changed after a series of six items ranging between 0.5 and 4 s in 0.5 s steps. After each list, participants judged which one of two test items were shown first (temporal order judgment) for items that were either drawn from the same context (within an ISI) or from consecutive contexts (across ISIs). Further, participants judged from memory whether the ISI associated to an item lasted longer than a standard interval (2.25 s) that was not previously shown (temporal source memory). Experiment 2 further included a time-item encoding task. Results revealed an effect of timing context changes in temporal order judgments, with faster responses (Experiment 1) or higher accuracy (Experiment 2) when items were drawn from the same context, as opposed to items drawn from across contexts. Further, in both experiments, we found that participants were well able to provide temporal source memory judgments based on recalled durations. Finally, replicated across experiments, we found subjective duration bias, as estimated by psychometric curve fitting parameters of the recalled durations, correlated negatively with within-context temporal order judgments. These findings show that changes in timing context support event segmentation in associative memory.

Conceptually plausible Bayesian inference in interval timing

In a world that is uncertain and noisy, perception makes use of optimization procedures that rely on the statistical properties of previous experiences. A well-known example of this phenomenon is the central tendency effect observed in many psychophysical modalities. For example, in interval timing tasks, previous experiences influence the current percept, pulling behavioural responses towards the mean. In Bayesian observer models, these previous experiences are typically modelled by unimodal statistical distributions, referred to as the prior. Here, we critically assess the validity of the assumptions underlying these models and propose a model that allows for more flexible, yet conceptually more plausible, modelling of empirical distributions. By representing previous experiences as a mixture of lognormal distributions, this model can be parametrized to mimic different unimodal distributions and thus extends previous instantiations of Bayesian observer models. We fit the mixture lognormal model to published interval timing data of healthy young adults and a clinical population of aged mild cognitive impairment patients and age-matched controls, and demonstrate that this model better explains behavioural data and provides new insights into the mechanisms that underlie the behaviour of a memory-affected clinical population.

A common dynamic prior for time in duration discrimination

Estimation of time depends heavily on both global and local statistical context. Durations that are short relative to the global distribution are systematically overestimated; durations that are locally preceded by long durations are also overestimated. Context effects are prominent in duration discrimination tasks, where a standard duration and a comparison duration are presented on each trial. In this study, we compare and test two models that posit a dynamically updating internal reference that biases time estimation on global and local scales in duration discrimination tasks. The internal reference model suggests that the internal reference operates during postperceptual stages and only interacts with the first presented duration. In contrast, a Bayesian account of time estimation implies that any perceived duration updates the internal reference and therefore interacts with both the first and second presented duration. We implemented both models and tested their predictions in a duration discrimination task where the standard duration varied from trial to trial. Our results are in line with a Bayesian perspective on time estimation. First, the standard systematically biased estimation of the comparison, such that shorter standards increased the likelihood of reporting that the comparison was shorter. Second, both the previous standard and comparison systematically biased time estimation of subsequent trials in the same direction. Third, more precise observers showed smaller biases. In sum, our findings suggest a common dynamic prior for time that is updated by each perceived duration and where the relative weighting of old and new observations is determined by their relative precision.

Eliciting contextual temporal calibration: The effect of bottom-up and top-down information in reproduction tasks

Bayesian integration assumes that a current observation is integrated with previous observations. An example in the temporal domain is the central tendency effect: when a range of durations is presented, a regression towards the mean is observed. Furthermore, a context effect emerges if a partially overlapping lower and a higher range of durations is presented in a blocked design, with the overlapping durations pulled towards the mean duration of the block. We determine under which conditions this context effect is observed, and whether explicit cues strengthen the effect. Each block contained either two or three durations, with one duration present in both blocks. We provided either no information at the start of each block about the nature of that block, provided written ("short" / "long" or "A" / "B") categorizations, or operationalized pitch (low vs high) to reflect the temporal context. We demonstrate that (1) the context effect emerges as long as sufficiently distinct durations are presented; (2) the effect is not modulated by explicit instructions or other cues; (3) just a single additional duration is sufficient to produce a context effect. Taken together, these results provide information on the most efficient operationalization to evoke the context effect, allowing for highly economical experimental designs, and highlights the automaticity by which priors are constructed.

Lingering Sound: Event-Related Phase-Amplitude Coupling and Phase-Locking in Fronto-Temporo-Parietal Functional Networks During Memory Retrieval of Music Melodies

Brain oscillations and connectivity have emerged as promising measures of evaluating memory processes, including encoding, maintenance, and retrieval, as well as the related executive function. Although many studies have addressed the neural mechanisms underlying working memory, most of these studies have focused on the visual modality. Neurodynamics and functional connectivity related to auditory working memory are yet to be established. In this study, we explored the dynamic of high density (128-channel) electroencephalography (EEG) in a musical delayed match-to-sample task (DMST), in which 36 participants were recruited and were instructed to recognize and distinguish the target melodies from similar distractors. Event-related spectral perturbations (ERSPs), event-related phase-amplitude couplings (ERPACs), and phase-locking values (PLVs) were used to determine the corresponding brain oscillations and connectivity. First, we observed that low-frequency oscillations in the frontal, temporal, and parietal regions were increased during the processing of both target and distracting melodies. Second, the cross-frequency coupling between low-frequency phases and high-frequency amplitudes was elevated in the frontal and parietal regions when the participants were distinguishing between the target from distractor, suggesting that the phase-amplitude coupling could be an indicator of neural mechanisms underlying memory retrieval. Finally, phase-locking, an index evaluating brain functional connectivity, revealed that there was fronto-temporal phase-locking in the theta band and fronto-parietal phase-locking in the alpha band during the recognition of the two stimuli. These findings suggest the existence of functional connectivity and the phase-amplitude coupling in the neocortex during musical memory retrieval, and provide a highly resolved timeline to evaluate brain dynamics. Furthermore, the inter-regional phase-locking and phase-amplitude coupling among the frontal, temporal and parietal regions occurred at the very beginning of musical memory retrieval, which might reflect the precise timing when cognitive resources were involved in the retrieval of targets and the rejection of similar distractors. To the best of our knowledge, this is the first EEG study employing a naturalistic task to study auditory memory processes and functional connectivity during memory retrieval, results of which can shed light on the use of natural stimuli in studies that are closer to the real-life applications of cognitive evaluations, mental treatments, and brain-computer interface.

Robust Temporal Averaging of Time Intervals Between Action and Sensation

Perception of the time interval between one’s own action (a finger tapping) and the associated sensory feedback (a visual flash or an auditory beep) is critical for precise and flexible control of action and behavioral decision. Previous studies have examined temporal averaging for multiple time intervals and its role for perceptual organization and crossmodal integration. In the present study, we extended the temporal averaging from sensory stimuli to the coupling of action and its sensory feedback. We investigated whether and how temporal averaging could be achieved with respect to the multiple intervals in a sequence of action-sensory feedback events, and hence affect the subsequent timing behavior. In unimodal task, participants voluntarily tapped their index finger at a constant pace while receiving auditory feedback (beeps) with varied intervals as well as variances throughout the sequence. In crossmodal task, for a given sequence, each tap was accompanied randomly with either visual flash or auditory beep as sensory feedback. When the sequence was over, observers produced a subsequent tap with either auditory or visual stimulus, which enclose a probe interval. In both tasks, participants were required to make a two alternative forced choice (2AFC), to indicate whether the target interval is shorter or longer than the mean interval between taps and their associated sensory events in the preceding sequence. In both scenarios, participants’ judgments of the probe interval suggested that they had internalized the mean interval associated with specific bindings of action and sensation, showing a robust temporal averaging process for the interval between action and sensation.

The dynamic effect of context on interval timing in children and adults

Human reproductions of time intervals are often biased toward previously perceived durations, resulting in a central tendency effect. The aim of the current study was to compare this effect of temporal context on time reproductions within children and adults. Children aged from 5 to 7 years, as well as adults, performed a ready-set-go reproduction task with a short and a long duration distribution. A central tendency effect was observed both in children and adults, with no age-difference in the effect of global context on temporal performance. However, the analysis of the effect of local context (trial-by-trial) indicated that younger children relied more on the duration (objective duration) presented in the most recent trial than adults. In addition, statistical analyses of the influence on temporal performance of recently reproduced durations by subjects (subjective duration) revealed that temporal reproductions in adults were influenced by performance drifts, i.e., their evaluation of their temporal error, while children simply relied on the value of reproduced durations on the recent trials. We argue that the central tendency effect was larger in young children due to their noisier internal representation of durations: A noisy system led participants to base their estimation on experienced duration rather than on the evaluation of their judgment.

1-s Productions: A Validation of an Efficient Measure of Clock Variability

Objective: Clock variance is an important statistic in many clinical and developmental studies. Existing methods require a large number of trials for accurate clock variability assessment, which is problematic in studies using clinical or either young or aged participants. Furthermore, these existing methods often implicitly convolute clock and memory processes, making it difficult to disentangle whether the clock or memory system are driving the observed deviations. Here we assessed whether 20 repeated productions of a well-engrained interval (1 s), a task that does not incorporate memory updating nor the processing of feedback, could provide an accurate assessment of clock variability.

Method: Sixty-eight undergraduate students completed two tasks: a 1-s production task in which they were asked to produce a 1-s duration by ending a tone by a keypress, and a multi-duration reproduction task. Durations presented in the reproduction task were tones lasting 1.17, 1.4 and 1.68 s. No feedback was presented in either task, and the order of presentation was counterbalanced between participants.

Results: The observed central tendency in the reproduction task was better explained by models including the measures of clock variability derived from the 1-s production task than by models without it. Three clock variability measures were calculated for each participant [standard deviation, root mean squared residuals (RMSRs) from an estimated linear slope, and RMSR scaled by mean production duration]. The model including the scaled RMSR was preferred over the alternative models, and no notable effects of the order of task presentation were observed. These results suggest that: (1) measures of variability should account for drift; (2) the presentation of another timing task before a 1-s production task did not influence the assessment of the clock variability; and (3) the observed variability adheres to the scalar property and predicts temporal performance, and is thus a usable index of clock variability.

Conclusion: This study shows that just 20 repeated productions of 1 s provide a reliable index of clock variability. As administering this task is fast and easy, it could prove to be useful in a large variety of developmental and clinical populations.

Time-Interval Emphasis in an Aeronautical Dual-Task Context: A Countermeasure to Task Absorption

OBJECTIVE

We tested a training method intended to prevent unsafe aeronautical behavior (i.e., too much time spent gazing inside the cockpit) induced by the modern cockpit, by teaching individuals to perform a task complementing the see-and-avoid mandatory safety task within a limited time interval.

BACKGROUND

Aeronautical activities led crews to perform several tasks simultaneously in an ergonomic environment under constant change. See and avoid remains one of the main safety tasks during visual flight. However, modern cockpits induce absorption and impair performance of this safety task. Many laboratory studies showed the relevance of training methods for managing dual-task situations and estimating time intervals.

METHOD

A specific virtual environment was developed to expose participants to a dual-task situation in which time-interval emphasis was provided in real time. Two types of emphasis training were tested: a permissive one that allowed participants to pursue the inside-cockpit task beyond the time limit and a nonpermissive one that did not.

RESULTS

The best time-interval acquisition, with retention up to 24 hr later, was observed in the nonpermissive condition, but task performances immediately after the training sessions were equivalent across conditions.

CONCLUSION

Time-emphasis training appears to be an efficient means of promoting absorption resistance while preserving task performance. Transferability of time-interval estimation skills has yet to be tested.

APPLICATION

Most areas of application for absorption resistance (aviation, shipping, rail, road, etc.) could benefit from this type of training to manage multitask situations.

Decay of internal reference information in duration discrimination: Intertrial interval modulates the Type B effect

Psychophysical evidence suggests that human perception of a stimulus is assimilated towards previous stimuli. The internal reference model (IRM) explains such assimilation through an internal reference (IR), which integrates past and present stimulus representations and thus might be conceived as a form of perceptual memory. In this study, we investigated whether the IR decays with time, as previously shown for perceptual memory representations in general. One specific prediction of IRM is higher discrimination sensitivity when a constant standard precedes rather than follows a variable comparison in a two-alternative forced-choice (2AFC) discrimination task. Furthermore, the magnitude of this so-called negative Type B effect should decrease with decreasing weighting of past stimulus information in the integration process. Therefore, decay of the IR should result in a reduced Type B effect. To examine this prediction, we carried out a 2AFC duration discrimination experiment with a short (1,600 ms) and a long (3,200 ms) intertrial interval (ITI). As expected, a reduced negative Type B effect was observed at the long compared with the short ITI, consistent with the idea that humans rely on the immediate past when evaluating current sensory input, however, less so when the IR incorporating the perceptual short-term memory representation of these past stimuli has already decayed.

Joint action coordination in expert-novice pairs: Can experts predict novices' suboptimal timing?

Previous research has established that skilled joint action partners use predictive models to achieve temporal coordination, for instance, when playing a music duet. But how do joint action partners with different skill levels achieve coordination? Can experts predict the suboptimal timing of novices? What kind of information allows them to predict novices’ timing? To address these questions, we asked skilled pianists to perform duets with piano novices. We varied whether, prior to performing duets, experts were familiar with novices’ performances of their individual parts of the duets and whether experts had access to the musical scores including the novices’ part of the duet. Familiarity with the score led to better coordination when the score implied a difficult passage. Familiarity with novices’ performances led to better joint action coordination for the remaining parts of the duet. Together, the results indicate that experts are surprisingly flexible in predicting novices’ suboptimal timing.

Neural markers of memory consolidation do not predict temporal estimates of encoded items

In contrast to the paradigms used in most laboratory experiments on interval timing, everyday tasks often involve tracking multiple, concurrent intervals without an explicit starting signal. As these characteristics are problematic for most existing clock-based models of interval timing, here we explore an alternative notion that suggests that time perception and working memory encoding might be closely connected. In this integrative model, the consolidation of a new item in working memory initiates cortical oscillations that also signal the onset of a time interval. The objective of this study was to test whether memory consolidation indeed acts as the starting signal of interval timing. Participants performed an attentional blink task in which they not only reported the targets, but also the estimated target onsets, allowing us to calculate estimated lag. In the attentional blink task, the second target (T2) in a rapid serial visual presentation is often not reported when it follows quickly after the first target (T1). However, if this fast T2 is reported, memory consolidation of T2 is presumably delayed. Consequently, if memory consolidation determines interval onset, we would expect a later estimated onset when consolidation is delayed. Furthermore, as the P3 ERP component is assumed to reflect memory consolidation, we expect that the estimated onsets and subjective lag are functions of the P3 latencies. The behavioral data show that the presumed delay in memory consolidation did not lead to later estimated onsets. In addition, the EEG results suggest that there was no relationship between P3 latency and subjective lag or estimated onset. Overall, our results suggest that there is no direct link between the encoding of items in working memory and sub-second interval timing of these items in the attentional blink task.

The effect of attention and working memory on the estimation of elapsed time

Psychological models of time perception involve attention and memory: while attention typically regulates the flow of events, memory maintains timed events or intervals. The precise, and possibly distinct, roles of attention and memory in time perception remain debated. In this behavioral study, we tested 48 participants in a prospective duration estimation task while they fully attended to time or performed a working memory (WM) task. We report that paying attention to time lengthened perceived duration in the range of seconds to minutes, whereas diverting attention away from time shortened perceived duration. The overestimation due to attending to time did not scale with durations. To the contrary, increasing WM load systematically decreased subjective duration and this effect scaled with durations. Herein, we discuss the dissociation between attention and WM in timing and scalar variability from the perspective of Bayesian models of time estimations.

Temporal Context Influences the Perceived Duration of Everyday Actions: Assessing the Ecological Validity of Lab-Based Timing Phenomena

Timing is key to accurate performance, for example when learning a new complex sequence by mimicry. However, most timing research utilizes artificial tasks and simple stimuli with clearly marked onset and offset cues. Here we address the question whether existing interval timing findings generalize to real-world timing tasks. In this study, animated video clips of a person performing different everyday actions were presented and participants had to reproduce the main action's duration. Although reproduced durations are more variable then observed in laboratory studies, the data adheres to two interval timing laws: Relative timing sensitivity is constant across durations (scalar property), and the subjective duration of a previous action influenced the current action's perceived duration (temporal context effect). Taken together, this demonstrates that laboratory findings generalize, and paves the way for studying interval timing as a component of complex, everyday cognitive performance.

Optimal Perceived Timing: Integrating Sensory Information with Dynamically Updated Expectations

The environment has a temporal structure, and knowing when a stimulus will appear translates into increased perceptual performance. Here we investigated how the human brain exploits temporal regularity in stimulus sequences for perception. We find that the timing of stimuli that occasionally deviate from a regularly paced sequence is perceptually distorted. Stimuli presented earlier than expected are perceptually delayed, whereas stimuli presented on time and later than expected are perceptually accelerated. This result suggests that the brain regularizes slightly deviant stimuli with an asymmetry that leads to the perceptual acceleration of expected stimuli. We present a Bayesian model for the combination of dynamically-updated expectations, in the form of a priori probability of encountering future stimuli, with incoming sensory information. The asymmetries in the results are accounted for by the asymmetries in the distributions involved in the computational process.

Cognitive Aging and Time Perception: Roles of Bayesian Optimization and Degeneracy

This review outlines the basic psychological and neurobiological processes associated with age-related distortions in timing and time perception in the hundredths of milliseconds-to-minutes range. The difficulty in separating indirect effects of impairments in attention and memory from direct effects on timing mechanisms is addressed. The main premise is that normal aging is commonly associated with increased noise and temporal uncertainty as a result of impairments in attention and memory as well as the possible reduction in the accuracy and precision of a central timing mechanism supported by dopamine-glutamate interactions in cortico-striatal circuits. Pertinent to these findings, potential interventions that may reduce the likelihood of observing age-related declines in timing are discussed. Bayesian optimization models are able to account for the adaptive changes observed in time perception by assuming that older adults are more likely to base their temporal judgments on statistical inferences derived from multiple trials than on a single trial's clock reading, which is more susceptible to distortion. We propose that the timing functions assigned to the age-sensitive fronto-striatal network can be subserved by other neural networks typically associated with finely-tuned perceptuo-motor adjustments, through degeneracy principles (different structures serving a common function).