Effects of pitch distance and likelihood on the perceived duration of deviant auditory events

Improving memory for unusual events with wakeful reactivation

Memory consists of multiple processes, from encoding information, consolidating it into short- and long- term memory, and later retrieving relevant information. Targeted memory reactivation is an experimental method during which sensory components of a multisensory representation (such as sounds or odors) are ‘reactivated’, facilitating the later retrieval of unisensory attributes. We examined whether novel and unpredicted events benefit from reactivation to a greater degree than normal stimuli. We presented participants with everyday objects, and ‘tagged’ these objects with sounds (e.g., animals and their matching sounds) at different screen locations. ‘Oddballs’ were created by presenting unusual objects and sounds (e.g., a unicorn with a heartbeat sound). During a short reactivation phase, participants listened to a replay of normal and oddball sounds. Participants were then tested on their memory for visual and spatial information in the absence of sounds. Participants were better at remembering the oddball objects compared to normal ones. Importantly, participants were also better at recalling the locations of oddball objects whose sounds were reactivated, compared to objects whose sounds that were not presented again. These results suggest that episodic memory benefits from associating objects with unusual cues, and that reactivating those cues strengthen the entire multisensory representation, resulting in enhanced memory for unisensory attributes.

Changes in face category induce stronger duration distortion in the temporal oddball paradigm

A novel stimulus embedded in a sequence of repeated stimuli is often perceived to be longer in duration. Studies have indicated the involvement of repetition suppression in this duration distortion, but it remains unclear which processing stages are important. The present study examined whether high-level visual category processing contributes to the oddball's duration distortion. In Experiment 1, we presented a novel face image in either human, monkey, or cat category after a repetition of an identical human face image in the temporal oddball paradigm. We found that the duration distortion of the last stimulus increased when the face changed across different categories, than when it changed within the same category. However, the effect of category change disappeared when globally scrambled and locally scrambled face images were used in Experiments 2 and 3, respectively, suggesting that the difference in duration distortion cannot be attributed to low-level visual properties of the images. Furthermore, in Experiment 4, we again used intact face images and found that category changes can influence the duration distortion even when a series of different human faces was presented before the last stimulus. These findings indicate that high-level visual category processing plays an important role in the duration distortion of oddballs. This study supports the idea that visual processing at higher visual stages is involved in duration perception. (219 words).

Interval timing in a hierarchical violation-of-expectation task: Dissociable effects of local and global predictions

Predictability associated with an event influences its perceived time. The two forms of predictions that are often discussed and have a dissociable influence on perceived time are repetition and expectation. However, predictions based on expectation can be seen at multiple levels, potentially leading to an inconsistency in the pattern in which expectation influences perceived time. Therefore, the present study aimed to investigate how different levels of predictions impact perceived time. In two separate experiments utilizing visual and auditory stimuli, we used a hierarchical violation-of-expectation paradigm that can dissociate two types of predictions based on local and global rules. Results from analysis of variance computed with local and global predictions revealed a pattern of local and global predictions having a distinct influence on perceived time. More specifically, while the local predictions that consider the immediate stimulus exposure reduced the perceived time, the global predictions that consider the overall regularities of a given context increased the perceived time. These results integrate well with the recent theoretical models rooted in a predictive coding framework that emphasizes the opposing effects of the first order and second order predictions on perceived time.

Neurophysiological responses of globus pallidus internus during the auditory oddball task in Parkinson's disease

Parkinson's disease can be associated with significant cognitive impairment that may lead to dementia. Deep brain stimulation (DBS) of the subthalamic nucleus is an effective therapy for motor symptoms but is associated with cognitive decline. DBS of globus pallidus internus (GPi) poses less risk of cognitive decline so may be the preferred target. A research priority is to identify biomarkers of cognitive decline in this population, but efforts are hampered by a lack of understanding of the role of the different basal ganglia nuclei, such as the globus pallidus, in cognitive processing. During deep brain stimulation (DBS) surgery, we monitored single units, beta oscillatory LFP activity as well as event related potentials (ERPs) from the globus pallidus internus (GPi) of 16 Parkinson's disease patients, while they performed an auditory attention task. We used an auditory oddball task, during which one standard tone is presented at regular intervals and a second deviant tone is presented with a low probability that the subject is requested to count and report at the end of the task. All forms of neuronal activity studied were selective modulated by the attended tones. Of 62 neurons studied, the majority (51 or 82%) responded selectively to the deviant tone. Beta oscillatory activity showed an overall desynchronization during both types of attended tones interspersed by bursts of beta activity giving rise to peaks at a latency of around 200 ms after tone onset. cognitive ERPs recorded in GPi were selective to the attended tone and the right-side cERP was larger than the left side. The averages of trials showing a difference in beta oscillatory activity between deviant and standard also had a significant difference in cERP amplitude. In one block of trials, the random occurrence of 3 deviant tones in short succession silenced the activity of the GPi neuron being recorded. Trial blocks where a clear difference in LFP beta was seen were twice as likely to yield a correct tone count (25 vs 11). The data demonstrate strong modulation of GPi neuronal activity during the auditory oddball task. Overall, this study demonstrates an involvement of GPi in processing of non-motor cognitive tasks such as working memory and attention, and suggests that direct effects of DBS in non-motor GPi may contribute to cognitive changes observed post-operatively.

Oddball onset timing: Little evidence of early gating of oddball stimuli from tapping, reacting, and producing

Oddballs, rare or novel stimuli, appear to last longer than non-oddballs. This illusion is often attributed to the perceived time that an oddball occupies being longer than that of a non-oddball. However, it is also possible that oddball stimuli are perceived to onset earlier than non-oddballs; they are “gated” earlier in time and thus the perceived duration of those stimuli are longer. In the current article, we directly investigate this proposal by asking participants to react to, produce durations initiated with, and tap along to either oddball or standard stimuli. Tapping provided some support for earlier perceived onset of an oddball in the visual modality. However, both reaction time and duration production experiments provided evidence against an oddball being gated earlier than a standard stimulus. Contrarily, these experiments showed an oddball resulted in longer reaction times and productions, respectively. Taken together, these three experiments indicate it is unlikely that the expansion of time attributed to oddball presentation is purely due to the earlier gating of oddball stimuli. In fact, the first two experiments provide some evidence that the effect of an oddball must compensate for the later gating of these stimuli.

Non-magnitude sources of bias on duration judgements for blank intervals: conceptual relatedness of interval markers reduces subjective interval duration

We report three experiments in which the events flanking a temporal interval were either related or unrelated, based on overlap in the letter identity of single letters (Experiment 1), in the conceptual congruency of color words and colored rectangles (Experiment 2), or in the conceptual congruency of sentence stems and their terminal words (Experiment 3). In all cases, we observed a bias for participants to judge the duration of temporal intervals as shorter when the flanking events were related. We draw an analogy between these temporal judgement distortions and those reported elsewhere (Alards-Tomalin et al. in J Exp Psychol Learn Mem Cogn 40(2):555-566, 2014) that revealed that the similarity in the relative magnitude of flanking events generate the same type of bias on duration judgements. The observation that non-magnitude dimensions of relatedness between flanking events can also bias duration judgements raise questions about the applicability of two influential theoretical frameworks for understanding the distorting effects that non-temporal stimulus dimensions can have on duration judgments, A Theory of Magnitude (Buetl and Walsh in Philos Trans R Soc B Biol Sci 12:1831-1840, 2009, Walsh in Trends Cogn Sci 7:483-488, 2003) and the Conceptual Metaphor Theory (e.g., Lakoff and Johnson in Philosophy in the flesh: the embodied mind and its challenge to western thought. Basic Books, New York, 1999). In our general discussion, we consider a number of alternative frameworks that may account for these findings.

The simultaneous oddball: Oddball presentation does not affect simultaneity judgments

The oddball duration illusion describes how a rare or nonrepeated stimulus is perceived as lasting longer than a common or repeated stimulus. It has been argued that the oddball duration illusion could emerge because of an earlier perceived onset of an oddball stimulus. However, most methods used to assess the perceived duration of an oddball stimulus are ill suited to detect onset effects. Therefore, in the current article, I tested the perceived onset of oddball and standard stimuli using a simultaneity judgment task. In Experiments 1 and 2, repetition and rarity of the target stimulus were varied, and participants were required to judge whether the target stimulus and another stimulus were concurrent. In Experiment 3, I tested whether a brief initial stimulus could act as a conditioning stimulus in the oddball duration illusion. This was to ensure an oddball duration illusion could have occurred given the short duration of stimuli in the first two experiments. In both the first two experiments, I found moderate support for no onset-based difference between oddball and nonoddball stimuli. In Experiment 3, I found that a short conditioning stimulus could still lead to the oddball duration illusion occurring, removing this possible explanation for the null result. Experiment 4 showed that an oddball duration illusion could emerge given the rarity of the stimulus and a concurrent sound. In sum, the current article found evidence against an onset-based explanation of the oddball duration illusion.

Temporal productions in a variable environment: timing starts from stimulus identification rather than onset

Timing an interval is integral in everyday life, from crossing a street or boiling an egg to playing sports and chatting with friends. In the current article, participants were asked to produce durations ranging from 500 to 1250 ms by either terminating an automatically initiated duration, or by maintaining a key press. When participants expected this production to start was manipulated using a variable foreperiod. Further, between subjects, the durations required for production were either variable or constant within a block. Together, these manipulations set up a temporally-and event-uncertain environment. When participants both initiated and terminated an interval, the uncertainty of the environment did not systematically affect productions. However, when productions were only terminated, productions were longer and given more uncertainty. While the effects of timing onset could be attributed to when a participant registers a stimulus, the effects of uncertainty with regards to what duration would be required for production indicates that participants appear to register what a stimulus is prior to initiating their timing. This finding indicates that timing may relate to when a stimulus is identified, rather than when it is first perceived. Alternatively, perhaps the onset of timing is postponed by event uncertainty.

Perceived Duration: The Interplay of Top-Down Attention and Task-Relevant Information

Perception of time is susceptible to distortions; among other factors, it has been suggested that the perceived duration of a stimulus is affected by the observer’s expectations. It has been hypothesized that the duration of an oddball stimulus is overestimated because it is unexpected, whereas repeated stimuli have a shorter perceived duration because they are expected. However, recent findings suggest instead that fulfilled expectations about a stimulus elicit an increase in perceived duration, and that the oddball effect occurs because the oddball is a target stimulus, not because it is unexpected. Therefore, it has been suggested that top-down attention is sometimes sufficient to explain this effect, and sometimes only necessary, with an additional contribution from saliency. However, how the expectedness of a target stimulus and its salient features affect its perceived duration is still an open question. In the present study, participants’ expectations about and the saliency of target stimuli were orthogonally manipulated with stimuli presented on a short (Experiment 1) or long (Experiment 2) temporal scale. Four repetitive standard stimuli preceded each target stimulus in a task in which participants judged whether the target was longer or shorter in duration than the standards. Engagement of top-down attention to target stimuli increased their perceived duration to the same extent irrespective of their expectedness. A small but significant additional contribution to this effect from the saliency of target stimuli was dependent on the temporal scale of stimulus presentation. In Experiment 1, saliency only significantly increased perceived duration in the case of expected target stimuli. In contrast, in Experiment 2, saliency exerted a significant effect on the overestimation elicited by unexpected target stimuli, but the contribution of this variable was eliminated in the case of expected target stimuli. These findings point to top-down attention as the primary cognitive mechanism underlying the perceptual extraction and processing of task-relevant information, which may be strongly correlated with perceived duration. Furthermore, the scalar properties of timing were observed, favoring the pacemaker-accumulator model of timing as the underlying timing mechanism.

Illusory perception of auditory filled duration is task- and context-dependent

In filled-duration illusion, a continuous (long) tone or an auditory sequence with multiple clicks is typically perceived as longer than the same physical duration (i.e., empty interval) enclosed by two auditory clicks. The auditory sequence is composed of multiple empty intervals. However, the individual empty interval in an auditory sequence, compared with the empty interval presented alone, could be biased in duration perception. In the current study, we implemented five experiments to reveal that the time perception of a single empty interval versus that of (mean) empty intervals in an auditory sequence depends on the task demands and contextual information. Specifically, we observed that the empty interval (140 ms) was perceived as longer than the same physical inter-stimulus interval in a sound sequence (Experiments 1 and 3). However, the empty interval (140 ms) was perceived as shorter than a continuous beep (i.e., filled duration of 140 ms) (Experiment 2). We observed a robust compression effect, in which the target empty interval (bounded by two oddball clicks) was perceived as shorter than the other physically equivalent intervals in a sound sequence (Experiment 4). In addition to the 'compression', perception of the target empty interval was assimilated by the other, task-irrelevant empty intervals in the sound sequence (Experiment 5). We explained the observed contextually modulated temporal illusions within a Bayesian inference framework.

Perceived duration of auditory oddballs: test of a novel pitch-window hypothesis

Unexpected oddball stimuli embedded within a series of otherwise identical standard stimuli tend to be overestimated in duration. The present study tested a pitch-window explanation of the auditory oddball effect on perceived duration in two experiments. For both experiments, participants listened to isochronous sequences consisting of a series of 400 Hz fixed-duration standard tones with an embedded oddball tone that differed in pitch and judged whether the variable-duration oddball was shorter or longer than the standard. Participants were randomly assigned to either a wide or narrow pitch-window condition, in which an anchor oddball was presented with high likelihood at either a far pitch (850 Hz) or a near pitch (550 Hz), respectively. In both pitch-window conditions, probe oddballs were presented with low likelihood at pitches that were either within or outside the frequency range established by the standard and anchor tones. Identical 700 Hz probe oddballs were perceived to be shorter in duration in the wide pitch-window condition than in the narrow pitch-window condition (Experiments 1 and 2), even when matching the overall frequency range of oddballs across conditions (Experiment 2). Results support the proposed pitch-window hypothesis, but are inconsistent with both enhanced processing and predictive coding accounts of the oddball effect.

Further Evidence That the Effects of Repetition on Subjective Time Depend on Repetition Probability

Repeated stimuli typically have shorter apparent duration than novel stimuli. Most explanations for this effect have attributed it to the repeated stimuli being more expected or predictable than the novel items, but an emerging body of work suggests that repetition and expectation exert distinct effects on time perception. The present experiment replicated a recent study in which the probability of repetition was varied between blocks of trials. As in the previous work, the repetition effect was smaller when repeats were common (and therefore more expected) than when they were rare. These results add to growing evidence that, contrary to traditional accounts, expectation increases apparent duration whereas repetition compresses subjective time, perhaps via a low-level process like adaptation. These opposing processes can be seen as instances of a more general “processing principle,” according to which subjective time is a function of the perceptual strength of the stimulus representation, and therefore depends on a confluence of “bottom-up” and “top-down” variables.

Triple dissociation of duration perception regulating mechanisms: Top-down attention is inherent

The brain constantly adjusts perceived duration based on the recent event history. One such lab phenomenon is subjective time expansion induced in an oddball paradigm ("oddball chronostasis"), where the duration of a distinct item (oddball) appears subjectively longer when embedded in a series of other repeated items (standards). Three hypotheses have been separately proposed but it remains unresolved which or all of them are true: 1) attention prolongs oddball duration, 2) repetition suppression reduces standards duration, and 3) accumulative temporal preparation (anticipation) expedites the perceived item onset so as to lengthen its duration. We thus conducted critical systematic experiments to dissociate the relative contribution of all hypotheses, by orthogonally manipulating sequences types (repeated, ordered, or random) and target serial positions. Participants' task was to judge whether a target lasts shorter or longer than its reference. The main finding was that a random item sequence still elicited significant chronostasis even though each item was odd. That is, simply being a target draws top-down attention and induces chronostasis. In Experiments 1 (digits) and 2 (orientations), top-down attention explained about half of the effect while saliency/adaptation explained the other half. Additionally, for non-repeated (ordered and random) sequence types, a target with later serial position still elicited stronger chronostasis, favoring a temporal preparation over a repetition suppression account. By contrast, in Experiment 3 (colors), top-down attention was likely the sole factor. Consequently, top-down attention is necessary and sometimes sufficient to explain oddball chronostasis; saliency/adaptation and temporal preparation are contingent factors. These critical boundary conditions revealed in our study serve as quantitative constraints for neural models of duration perception.

Duration compression induced by visual and phonological repetition of Chinese characters

Our prior experience heavily influences our subjective time. One of such phenomena is repetition compression, that is, repeated stimuli are perceived shorter than novel stimuli. However, most of the studies on repetition compression used identical stimuli, leaving the question whether similar repetition effects could take place in phonological and semantic level repetition. We used Chinese characters to manipulate different levels of repetition in a duration discrimination task. We replicated earlier findings that repetition of visual identical characters shortened the apparent duration and found the repetition compression was spatially independent. Phonological repetition also caused the duration compression though the effect was weaker than the visual repetition. However, we observed no duration compression during the semantic repetition. The results suggest that repetition compression is mediated by visual and phonological representation of a stimulus in an early stage in processing hierarchy. We explained our findings according to the framework of predictive coding.

Attentional entrainment and perceived event duration

This study considered the contribution of dynamic attending theory (DAT) and attentional entrainment to systematic distortions in perceived event duration. Three experiments were conducted using an auditory oddball paradigm, in which listeners judged the duration of a deviant (oddball) stimulus embedded within a series of identical (standard) stimuli. To test for a role of attentional entrainment in perceived oddball duration, oddballs were presented at either temporally expected (on time) or unexpectedly early or late time points relative to extrapolation of the context rhythm. Consistent with involvement of attentional entrainment in perceived duration, duration judgements about the oddball were least distorted when the oddball occurred on time with respect to the entrained rhythm, whereas durations of early and late oddballs were perceived to be shorter and longer, respectively. This pattern of results was independent of the absolute time interval preceding the oddball. Moreover, as expected, an irregularly timed sequence context weakened observed differences between oddballs with on-time and late onsets. Combined with other recent work on the role of temporal preparation in duration distortions, the present findings allot at least a portion of the oddball effect to increased attention to events that are more expected, rather than on their unexpected nature per se.

Perceived duration is reduced by repetition but not by high-level expectation

A repeated stimulus is judged as briefer than a novel one. It has been suggested that this duration illusion is an example of a more general phenomenon-namely that a more expected stimulus is judged as briefer than a less expected one. To test this hypothesis, we manipulated high-level expectation through the probability of a stimulus sequence, through the regularity of the preceding stimuli in a sequence, or through whether a stimulus violates an overlearned sequence. We found that perceived duration is not reduced by these types of expectation. Repetition of stimuli, on the other hand, consistently reduces perceived duration across our experiments. In addition, the effect of stimulus repetition is constrained to the location of the repeated stimulus. Our findings suggest that estimates of subsecond duration are largely the result of low-level sensory processing.

The influence of stimulus repetition on duration judgments with simple stimuli

Two experiments investigated the effects of stimulus repetition vs. stimulus novelty on perceived duration. In a reminder task, a standard and a comparison stimulus were presented consecutively in each trial, and the comparison was either a repetition of the standard or a different stimulus. Pseudowords (Experiment 1) or strings of consonants (Experiment 2) were used as stimuli and the inter-stimulus interval (ISI) between the standard and the comparison was either constant or variable. Participants were asked to judge whether the comparison was shorter or longer than the standard. In both experiments, we observed shorter judged durations for repeated than for novel comparisons whereas the manipulation of the ISI had no pronounced effects on duration judgments. The finding of shorter duration judgments for repeated as compared to novel nonwords replicates the results of a previous study (Matthews, 2011) which employed highly complex stimulus material. The present study shows that changes of simple, semantically meaningless stimuli are sufficient to result in a shorter perceived duration of repeated as compared to novel stimuli.

Introducing a control condition in the classic oddball paradigm: Oddballs are overestimated in duration not only because of their oddness

Three experiments assessed whether participants perceive infrequent stimuli (oddballs) to last longer than frequent ones (standards). The classical oddball paradigm was modified so that participants judged the duration of a target stimulus which could either be a standard or an oddball. The target was always the fifth stimulus in the stream and all stimuli were presented at predefined spatial positions. These modifications enabled a direct comparison of duration judgments for oddballs and standards. In Experiments 1 and 2 not only the duration of oddballs but also the duration of standards was overestimated by virtually the same amount. In other words, the overestimation of oddballs was not due to their oddness but reflected a different temporal dilation such as the negative time-order error. In Experiment 3, all stimuli were presented at the same spatial position. Again, both oddballs and standards were overestimated, however, oddballs more so. The present results highlight the importance of comparing the judged duration of oddballs and standards when evaluating the size of the genuine oddball effect. Furthermore, the findings suggest that the occurrence of temporal oddball effects can depend on spatial features of stimulus presentation.

Statistical context shapes stimulus-specific adaptation in human auditory cortex

Stimulus-specific adaptation is the phenomenon whereby neural response magnitude decreases with repeated stimulation. Inconsistencies between recent nonhuman animal recordings and computational modeling suggest dynamic influences on stimulus-specific adaptation. The present human electroencephalography (EEG) study investigates the potential role of statistical context in dynamically modulating stimulus-specific adaptation by examining the auditory cortex-generated N1 and P2 components. As in previous studies of stimulus-specific adaptation, listeners were presented with oddball sequences in which the presentation of a repeated tone was infrequently interrupted by rare spectral changes taking on three different magnitudes. Critically, the statistical context varied with respect to the probability of small versus large spectral changes within oddball sequences (half of the time a small change was most probable; in the other half a large change was most probable). We observed larger N1 and P2 amplitudes (i.e., release from adaptation) for all spectral changes in the small-change compared with the large-change statistical context. The increase in response magnitude also held for responses to tones presented with high probability, indicating that statistical adaptation can overrule stimulus probability per se in its influence on neural responses. Computational modeling showed that the degree of coadaptation in auditory cortex changed depending on the statistical context, which in turn affected stimulus-specific adaptation. Thus the present data demonstrate that stimulus-specific adaptation in human auditory cortex critically depends on statistical context. Finally, the present results challenge the implicit assumption of stationarity of neural response magnitudes that governs the practice of isolating established deviant-detection responses such as the mismatch negativity.

Duration estimation entails predicting when

The estimation of duration can be affected by context and surprise. Using MagnetoEncephaloGraphy (MEG), we tested whether increased neural activity during surprise and following neural suppression in two different contexts supported subjective time dilation (Eagleman and Pariyadath, 2009; Pariyadath and Eagleman, 2012). Sequences of three 300 ms frequency-modulated (FM, control) or pure tones (test) were presented and followed by a fourth FM varying in duration. In test, the last FM was perceived as significantly longer than veridical duration (Tse et al., 2004) but did not differ from the perceived duration in control. Several novel and distinct neural signatures were observed in duration estimation: first, neural suppression of standard stimuli was observed for the onset but not for the offset auditory evoked responses. Second, ramping activity increased with veridical duration in control whereas at the same latency in test, the amplitude of the midlatency response increased with the distance of deviant durations. Third, in both conditions, the amplitude of the offset auditory evoked responses accounted well for participants' performance: the longer the perceived duration, the larger the offset response. Fourth, neural duration demarcated by the peak latencies of the onset and ramping evoked activities indexed a systematic time compression that reliably predicted subjective time perception. Our findings suggest that interval timing undergoes time compression by capitalizing on the predicted offset of an auditory event.