Time Order as Psychological Bias

Precision-based causal inference modulates audiovisual temporal recalibration

Cross-modal temporal recalibration guarantees stable temporal perception across ever-changing environments. Yet, the mechanisms of cross-modal temporal recalibration remain unknown. Here, we conducted an experiment to measure how participants' temporal perception was affected by exposure to audiovisual stimuli with consistent temporal delays. Consistent with previous findings, recalibration effects plateaued with increasing audiovisual asynchrony and varied by which modality led during the exposure phase. We compared six observer models that differed in how they update the audiovisual temporal bias during the exposure phase and whether they assume modality-specific or modality-independent precision of arrival latency. The causal-inference observer shifts the audiovisual temporal bias to compensate for perceived asynchrony, which is inferred by considering two causal scenarios: when the audiovisual stimuli have a common cause or separate causes. The asynchrony-contingent observer updates the bias to achieve simultaneity of auditory and visual measurements, modulating the update rate by the likelihood of the audiovisual stimuli originating from a simultaneous event. In the asynchrony-correction model, the observer first assesses whether the sensory measurement is asynchronous; if so, she adjusts the bias proportionally to the magnitude of the measured asynchrony. Each model was paired with either modality-specific or modality-independent precision of arrival latency. A Bayesian model comparison revealed that both the causal-inference process and modality-specific precision in arrival latency are required to capture the nonlinearity and asymmetry observed in audiovisual temporal recalibration. Our findings support the hypothesis that audiovisual temporal recalibration relies on the same causal-inference processes that govern cross-modal perception.

Quantifying accuracy and precision from continuous response data in studies of spatial perception and crossmodal recalibration

The ability to detect the absolute location of sensory stimuli can be quantified with either error-based metrics derived from single-trial localization errors or regression-based metrics derived from a linear regression of localization responses on the true stimulus locations. Here we tested the agreement between these two approaches in estimating accuracy and precision in a large sample of 188 subjects who localized auditory stimuli from different azimuthal locations. A subsample of 57 subjects was subsequently exposed to audiovisual stimuli with a consistent spatial disparity before performing the sound localization test again, allowing us to additionally test which of the different metrics best assessed correlations between the amount of crossmodal spatial recalibration and baseline localization performance. First, our findings support a distinction between accuracy and precision. Localization accuracy was mainly reflected in the overall spatial bias and was moderately correlated with precision metrics. However, in our data, the variability of single-trial localization errors (variable error in error-based metrics) and the amount by which the eccentricity of target locations was overestimated (slope in regression-based metrics) were highly correlated, suggesting that intercorrelations between individual metrics need to be carefully considered in spatial perception studies. Secondly, exposure to spatially discrepant audiovisual stimuli resulted in a shift in bias toward the side of the visual stimuli (ventriloquism aftereffect) but did not affect localization precision. The size of the aftereffect shift in bias was at least partly explainable by unspecific test repetition effects, highlighting the need to account for inter-individual baseline differences in studies of spatial learning.

Neural oscillations reflect the individual differences in the temporal perception of audiovisual speech

Multisensory integration occurs within a limited time interval between multimodal stimuli. Multisensory temporal perception varies widely among individuals and involves perceptual synchrony and temporal sensitivity processes. Previous studies explored the neural mechanisms of individual differences for beep-flash stimuli, whereas there was no study for speech. In this study, 28 subjects (16 male) performed an audiovisual speech/ba/simultaneity judgment task while recording their electroencephalography. We examined the relationship between prestimulus neural oscillations (i.e. the pre-pronunciation movement-related oscillations) and temporal perception. The perceptual synchrony was quantified using the Point of Subjective Simultaneity and temporal sensitivity using the Temporal Binding Window. Our results revealed dissociated neural mechanisms for individual differences in Temporal Binding Window and Point of Subjective Simultaneity. The frontocentral delta power, reflecting top-down attention control, is positively related to the magnitude of individual auditory leading Temporal Binding Windows (auditory Temporal Binding Windows; LTBWs), whereas the parieto-occipital theta power, indexing bottom-up visual temporal attention specific to speech, is negatively associated with the magnitude of individual visual leading Temporal Binding Windows (visual Temporal Binding Windows; RTBWs). In addition, increased left frontal and bilateral temporoparietal occipital alpha power, reflecting general attentional states, is associated with increased Points of Subjective Simultaneity. Strengthening attention abilities might improve the audiovisual temporal perception of speech and further impact speech integration.

Altering Time Perception in Virtual Reality Through Multimodal Visual-Tactile Kappa Effect

The perception of time is highly subjective and intertwined with space perception. In a well-known perceptual illusion, called Kappa effect, the distance between consecutive stimuli is modified to induce time distortions in the perceived inter-stimulus interval that are proportional to the distance between the stimuli. However, to the best of our knowledge, this effect has not been characterized and exploited in virtual reality (VR) within a multisensory elicitation framework. This paper investigates the Kappa effect elicited by concurrent visual-tactile stimuli delivered to the forearm, through a multimodal VR interface. This paper compares the outcomes of an experiment in VR with the results of the same experiment performed in the "physical world", where a multimodal interface was applied to participants' forearm to deliver controlled visual-tactile stimuli. Our results suggest that a multimodal Kappa effect can be elicited both in VR and in the physical world relying on concurrent visual-tactile stimulation. Moreover, our results confirm the existence of a relation between the ability of participants in discriminating the duration of time intervals and the magnitude of the experienced Kappa effect. These outcomes can be exploited to modulate the subjective perception of time in VR, paving the path toward more personalised human-computer interaction.

Spontaneous alpha-band oscillations reflect individual differences in audiovisual temporal perception

Light, and sound are persistently out of sync for subjective temporal perception called point of subjective simultaneity (PSS). It is stable within individuals but variable among individuals. Previous studies found that spontaneous alpha power, functioning in attention-related brain states, predicts individual PSS in the temporal order judgment (TOJ) task. However, the neural mechanisms underlying individual differences in audiovisual PSS have not been elucidated in the simultaneity judgment (SJ) task. A hypothesis that the spontaneous alpha band power might reflect the individual subjective temporal bias was proposed. We designed an SJ task EEG experiment where subjects judged whether the beep-flash stimuli are synchronous to test the above hypothesis. We primarily explored the correlation between the alpha-band power differences (visual- and auditory-leading conditions) with individual PSS. We used the V50A (~50% proportion of synchronous responses) to represent visual-leading conditions while A50V represents auditory-leading ones. We found the higher alpha power difference (V50A - A50V) predicted larger individual PSS. This study extends previous results and found that individual difference effects in the alpha band power also exist in the SJ task. The results suggested that alpha power might be associated with a spontaneous attentional state and reflect individuals' subjective temporal bias.

Bringing Bayes and Shannon to the Study of Behavioural and Neurobiological Timing and Associative Learning

Bayesian parameter estimation and Shannon’s theory of information provide tools for analysing and understanding data from behavioural and neurobiological experiments on interval timing—and from experiments on Pavlovian and operant conditioning, because timing plays a fundamental role in associative learning. In this tutorial, we explain basic concepts behind these tools and show how to apply them to estimating, on a trial-by-trial, reinforcement-by-reinforcement and response-by-response basis, important parameters of timing behaviour and of the neurobiological manifestations of timing in the brain. These tools enable quantification of relevant variables in the trade-off between acting as an ideal observer should act and acting as an ideal agent should act, which is also known as the trade-off between exploration (information gathering) and exploitation (information utilization) in reinforcement learning. They enable comparing the strength of the evidence for a measurable association to the strength of the behavioural evidence that the association has been perceived. A GitHub site and an OSF site give public access to well-documented Matlab and Python code and to raw data to which these tools have been applied.

Temporal Order Judgments in Schizophrenia and Bipolar Disorders – Explicit and Implicit Measures

Ordering events in time is essential for the understanding of causal relationships between successive events. Incorrect causal links can lead to false beliefs and an altered perception of reality. These symptoms belong to psychosis, which is present in schizophrenia (SZ) spectrum and bipolar (BP) disorder. Experimental results show that patients with SZ have an altered perception of temporal order, while there are no data in patients with BP. We investigated the ability of patients with SZ, BP, and controls to judge the order of stimuli with a 100-ms Stimulus Onset Asynchrony (SOA), and how such large asynchronies facilitate temporal order judgments for small asynchronies. Explicit temporal order effects suggest that patients with SZ perform worse at a long SOA (100 ms) as compared to controls, whereas patients with BP show no difference compared to controls or to patients with SZ. Implicit order effects reveal improved performances in case of identical as compared to different relative order between two successive trials for all groups, with no differences between the groups. We replicated explicit order impairments in patients with SZ compared to controls, while implicit effects appear to be preserved. This difficulty for patients to consciously order stimuli in time might be understood under the light of the loosening-of-associations phenomenon well described in SZ. Further, we showed that patients with BP do not reveal such an explicit order impairment which is consistent with phenomenological descriptions, suggesting a difference in time experience in patients with SZ and BP.

Neural-latency noise places limits on human sensitivity to the timing of events

The brain-time account posits that the physical timing of sensory-evoked neural activity determines the perceived timing of corresponding sensory events. A canonical model formalises this account for tasks such as simultaneity and order judgements: Signals arrive at a decision centre in an order, and at a temporal offset, shaped by neural propagation times. This model assumes that the noise affecting people's temporal judgements is primarily neural-latency noise, i.e. variation in propagation times across trials, but this assumption has received little scrutiny. Here, we recorded EEG alongside simultaneity judgements from 50 participants in response to combinations of visual, auditory and tactile stimuli. Bootstrapping of ERP components was used to estimate neural-latency noise, and simultaneity judgements were modelled to estimate the precision of timing judgements. We obtained the predicted correlation between neural and behavioural measures of latency noise, supporting a fundamental feature of the canonical model of perceived timing.

Self, Me and I in the repertoire of spontaneously occurring altered states of Selfhood: eight neurophenomenological case study reports

This study investigates eight case reports of spontaneously emerging, brief episodes of vivid altered states of Selfhood (ASoSs) that occurred during mental exercise in six long-term meditators by using a neurophenomenological electroencephalography (EEG) approach. In agreement with the neurophenomenological methodology, first-person reports were used to identify such spontaneous ASoSs and to guide the neural analysis, which involved the estimation of three operational modules of the brain self-referential network (measured by EEG operational synchrony). The result of such analysis demonstrated that the documented ASoSs had unique neurophenomenological profiles, where several aspects or components of Selfhood (measured neurophysiologically and phenomenologically) are affected and expressed differently, but still in agreement with the neurophysiological three-dimensional construct model of the complex experiential Selfhood proposed in our earlier work (Fingelkurts et al. in Conscious Cogn 86:103031. 10.1016/j.concog.2020.103031, 2020).

The time course of synaesthetic colour perception

Grapheme-colour synaesthesia is a neurodevelopmental condition wherein perception of numbers and letters consistently and involuntarily elicits concurrent experiences of colour photisms. Accumulating evidence suggests that heterogeneity in the visuospatial phenomenology of synaesthesia is attributable to the operation of top-down processes underlying photisms experienced as representations in associator synaesthetes and bottom-up processes subserving photisms experienced as spatially localized in projector synaesthetes. An untested corollary of this hypothesis is that bottom-up mechanisms will actuate earlier photism perception in projector than associator synaesthetes. We tested this prediction in a pre-registered study wherein associators and projectors completed adaptive temporal order judgement tasks for graphemes, colours, and photisms. In corroboration of the hypothesis of differential photism access across subtypes, projectors displayed lower photism colour thresholds than associators whereas the two subtypes did not significantly differ in veridical colour thresholds. Synaesthetes did not differ in grapheme or colour thresholds relative to non-synaesthete controls. These results are consistent with the proposal of differential neural mechanisms underlying photism perception in subtypes of grapheme-colour synaesthesia and warrant renewed attention to heterogeneity in the mechanisms and phenomenology of this condition.

Sensory experience during early sensitive periods shapes cross-modal temporal biases

Typical human perception features stable biases such as perceiving visual events as later than synchronous auditory events. The origin of such perceptual biases is unknown. To investigate the role of early sensory experience, we tested whether a congenital, transient loss of pattern vision, caused by bilateral dense cataracts, has sustained effects on audio-visual and tactile-visual temporal biases and resolution. Participants judged the temporal order of successively presented, spatially separated events within and across modalities. Individuals with reversed congenital cataracts showed a bias towards perceiving visual stimuli as occurring earlier than auditory (Expt. 1) and tactile (Expt. 2) stimuli. This finding stood in stark contrast to normally sighted controls and sight-recovery individuals who had developed cataracts later in childhood: both groups exhibited the typical bias of perceiving vision as delayed compared to audition. These findings provide strong evidence that cross-modal temporal biases depend on sensory experience during an early sensitive period.

Alpha Activity Reflects the Magnitude of an Individual Bias in Human Perception

Biases in sensory perception can arise from both experimental manipulations and personal trait-like features. These idiosyncratic biases and their neural underpinnings are often overlooked in studies on the physiology underlying perception. A potential candidate mechanism reflecting such idiosyncratic biases could be spontaneous alpha band activity, a prominent brain rhythm known to influence perceptual reports in general. Using a temporal order judgment task, we here tested the hypothesis that alpha power reflects the overcoming of an idiosyncratic bias. Importantly, to understand the interplay between idiosyncratic biases and contextual (temporary) biases induced by experimental manipulations, we quantified this relation before and after temporal recalibration. Using EEG recordings in human participants (male and female), we find that prestimulus frontal alpha power correlates with the tendency to respond relative to an own idiosyncratic bias, with stronger α leading to responses matching the bias. In contrast, alpha power does not predict response correctness. These results also hold after temporal recalibration and are specific to the alpha band, suggesting that alpha band activity reflects, directly or indirectly, processes that help to overcome an individual's momentary bias in perception. We propose that combined with established roles of parietal α in the encoding of sensory information frontal α reflects complementary mechanisms influencing perceptual decisions.SIGNIFICANCE STATEMENT The brain is a biased organ, frequently generating systematically distorted percepts of the world, leading each of us to evolve in our own subjective reality. However, such biases are often overlooked or considered noise when studying the neural mechanisms underlying perception. We show that spontaneous alpha band activity predicts the degree of biasedness of human choices in a time perception task, suggesting that alpha activity indexes processes needed to overcome an individual's idiosyncratic bias. This result provides a window onto the neural underpinnings of subjective perception, and offers the possibility to quantify or manipulate such priors in future studies.

Judging Relative Onsets and Offsets of Audiovisual Events

This study assesses the fidelity with which people can make temporal order judgments (TOJ) between auditory and visual onsets and offsets. Using an adaptive staircase task administered to a large sample of young adults, we find that the ability to judge temporal order varies widely among people, with notable difficulty created when auditory events closely follow visual events. Those findings are interpretable within the context of an independent channels model. Visual onsets and offsets can be difficult to localize in time when they occur within the temporal neighborhood of sound onsets or offsets.

Individual Alpha Frequency Predicts Perceived Visuotactile Simultaneity

Temporal encoding is a key feature in multisensory processing that leads to the integration versus segregation of perceived events over time. Whether or not two events presented at different offsets are perceived as simultaneous varies widely across the general population. Such tolerance to temporal delays is known as the temporal binding window (TBW). It has been recently suggested that individual oscillatory alpha frequency (IAF) peak may represent the electrophysiological correlate of TBW, with IAF also showing a wide variability in the general population (8–12 Hz). In our work, we directly tested this hypothesis by measuring each individual's TBW during a visuotactile simultaneity judgment task while concurrently recording their electrophysiological activity. We found that the individual's TBW significantly correlated with their left parietal IAF, such that faster IAF accounted for narrower TBW. Furthermore, we found that higher prestimulus alpha power measured over the same left parietal regions accounted for more veridical responses of non-simultaneity, which may be explained either by accuracy in perceptual simultaneity or, alternatively, in line with recent proposals by a shift in response bias from more conservative (high alpha power) to more liberal (low alpha power). We propose that the length of an alpha cycle constrains the temporal resolution within which perceptual processes take place.

Dissociating the sequential dependency of subjective temporal order from subjective simultaneity

The physical simultaneity between two events can differ from our point of subjective simultaneity (PSS). Studies using simultaneity judgments (SJ) and temporal order judgments (TOJ) tasks have shown that whether two events are reported as simultaneous is highly context-dependent. It has been recently suggested that the interval between the two events in the previous trial can modulate judgments both in SJ and TOJ tasks, an effect named rapid recalibration. In this work, we investigated rapid recalibration in SJ and TOJ tasks and tested whether centering the range of presented intervals on perceived simultaneity modulated this effect. We found a rapid recalibration effect in TOJ, but not in SJ. Moreover, we found that centering the intervals on objective or subjective simultaneity did not change the pattern of results. Interestingly, we also found no correlations between an individual’s PSS in TOJ and in SJ tasks, which corroborates other studies in suggesting that these two psychophysical measures may capture different processes.

Group analyses can hide heterogeneity effects when searching for a general model: Evidence based on a conflict monitoring task

In experimental psychology, a unique model of general processing is often sought to represent the behaviors of all individuals. We address the question of whether seeking this objective - a unique model - is the most fruitful scientific strategy by studying a specific case example. In order to approach an answer to such a question, we compared the conventional approach in experimental psychology with analyses at the individual level by applying a specific mathematical modeling approach. A sample of 1159 individuals completed an experimental task based on managing conflict (a type of Simon task). Key findings revealed that at least four models are required to properly account for individuals' performance. Interestingly, four out of ten participants failed to show stimulus-response congruency effects in the experimental task, whereas the remaining 60% followed distinguishable theoretical models (consistent with conflict-monitoring theory and/or priming and episodic memory effects). The reported findings suggest that individuals' psychological characteristics might help to explain some of the reproducibility issues that are currently of great concern in psychology. These findings, along with further recent research, support the view that general and differential psychological approaches work better together for addressing relevant theoretical issues in psychological research.

Robustness of individual differences in temporal interference effects

Magnitudes or quantities of the different dimensions that define a stimulus (e.g., space, speed or numerosity) influence the perceived duration of that stimulus, a phenomenon known as (temporal) interference effects. This complicates studying the neurobiological foundation of the perception of time, as any signatures of temporal processing are tainted by interfering dimensions. In earlier work, in which judgements on either time or numerosity were made while EEG was recorded, we used Maximum Likelihood Estimation (MLE) to estimate, for each participant separately, the influence of temporal and numerical information on making duration or numerosity judgements. We found large individual differences in the estimated magnitudes, but ML-estimates allowed us to partial out interference effects. However, for such analyses, it is essential that estimates are meaningful and stable. Therefore, in the current study, we examined the reliability of the MLE procedure by comparing the interference magnitudes estimated in two sessions, spread a week apart. In addition to the standard paradigm, we also presented task variants in which the interfering dimension was manipulated, to assess which aspects of the numerosity dimension exert the largest influence on temporal processing. The results indicate that individual interference magnitudes are stable, both between sessions and over tasks. Further, the ML-estimates of the time-numerosity judgement tasks were predictive of performance in a standard temporal judgement task. Thus, how much temporal information participants use in time estimations tasks seems to be a stable trait that can be captured with the MLE procedure. ML-estimates are, however, not predictive of performance in other interference-tasks, here operationalized by a numerical Stroop task. Taken together, the MLE procedure is a reliable tool to quantify individual differences in magnitude interference effects and can therefore reliably inform the analysis of neuroimaging data when contrasts are needed between the accumulation of a temporal and an interfering dimension.

Temporal Audiovisual Motion Prediction in 2D- vs. 3D-Environments

Predicting motion is essential for many everyday life activities, e.g., in road traffic. Previous studies on motion prediction failed to find consistent results, which might be due to the use of very different stimulus material and behavioural tasks. Here, we directly tested the influence of task (detection, extrapolation) and stimulus features (visual vs. audiovisual and three-dimensional vs. non-three-dimensional) on temporal motion prediction in two psychophysical experiments. In both experiments a ball followed a trajectory toward the observer and temporarily disappeared behind an occluder. In audiovisual conditions a moving white noise (congruent or non-congruent to visual motion direction) was presented concurrently. In experiment 1 the ball reappeared on a predictable or a non-predictable trajectory and participants detected when the ball reappeared. In experiment 2 the ball did not reappear after occlusion and participants judged when the ball would reach a specified position at two possible distances from the occluder (extrapolation task). Both experiments were conducted in three-dimensional space (using stereoscopic screen and polarised glasses) and also without stereoscopic presentation. Participants benefitted from visually predictable trajectories and concurrent sounds during detection. Additionally, visual facilitation was more pronounced for non-3D stimulation during detection task. In contrast, for a more complex extrapolation task group mean results indicated that auditory information impaired motion prediction. However, a post hoc cross-validation procedure (split-half) revealed that participants varied in their ability to use sounds during motion extrapolation. Most participants selectively profited from either near or far extrapolation distances but were impaired for the other one. We propose that interindividual differences in extrapolation efficiency might be the mechanism governing this effect. Together, our results indicate that both a realistic experimental environment and subject-specific differences modulate the ability of audiovisual motion prediction and need to be considered in future research.

Prestimulus Alpha Oscillations and the Temporal Sequencing of Audiovisual Events

Perceiving the temporal order of sensory events typically depends on participants' attentional state, thus likely on the endogenous fluctuations of brain activity. Using magnetoencephalography, we sought to determine whether spontaneous brain oscillations could disambiguate the perceived order of auditory and visual events presented in close temporal proximity, that is, at the individual's perceptual order threshold (Point of Subjective Simultaneity [PSS]). Two neural responses were found to index an individual's temporal order perception when contrasting brain activity as a function of perceived order (i.e., perceiving the sound first vs. perceiving the visual event first) given the same physical audiovisual sequence. First, average differences in prestimulus auditory alpha power indicated perceiving the correct ordering of audiovisual events irrespective of which sensory modality came first: a relatively low alpha power indicated perceiving auditory or visual first as a function of the actual sequence order. Additionally, the relative changes in the amplitude of the auditory (but not visual) evoked responses were correlated with participant's correct performance. Crucially, the sign of the magnitude difference in prestimulus alpha power and evoked responses between perceived audiovisual orders correlated with an individual's PSS. Taken together, our results suggest that spontaneous oscillatory activity cannot disambiguate subjective temporal order without prior knowledge of the individual's bias toward perceiving one or the other sensory modality first. Altogether, our results suggest that, under high perceptual uncertainty, the magnitude of prestimulus alpha (de)synchronization indicates the amount of compensation needed to overcome an individual's prior in the serial ordering and temporal sequencing of information.

Conscious awareness and time perception

This paper examines Zhou, Pöppel, and Bao's 2014 proposal ("In the Jungle of Time: The Concept of Identity as a Way Out," https://doi.org/10.3389/fpsyg.2014.00844) to unify the psychology of time through a biological principle concerning identity and homeostasis. Although the present analysis largely agrees with their proposal, it argues that a Dual Model of time (see Montemayor's 2013 monograph, Minding Time: A Philosophical and Theoretical Approach to the Psychology of Time) is needed to account for two important roles in time cognition, one related to navigation and the other to conscious awareness. In a Dual Model, the homeostatic principle plays a critical role with respect to conscious awareness, but a different principle is needed for integrating the metric constraints on navigation.

Differences between endogenous attention to spatial locations and sensory modalities

Vibell et al. (J Cogn Neurosci 19:109-120, 2007) reported that endogenously attending to a sensory modality (vision or touch) modulated perceptual processing, in part, by the relative speeding-up of neural activation (i.e., as a result of prior entry). However, it was unclear whether it was the fine temporal discrimination required by the temporal-order judgment task that was used, or rather, the type of attentional modulation (spatial locations or sensory modalities) that was responsible for the shift in latencies that they observed. The present study used a similar experimental design to evaluate whether spatial attention would also yield similar latency effects suggestive of prior entry in the early visual P1 potentials. Intriguingly, while the results demonstrate similar neural latency shifts attributable to spatial attention, they started at a somewhat later stage than seen in Vibell et al.'s study. These differences are consistent with different neural mechanisms underlying attention to a specific sensory modality versus to a spatial location.