Cross-dimensional magnitude interaction is modulated by representational noise: evidence from space-time interaction

Memory traces of duration and location in the right intraparietal sulcus

Time and space form an integral part of every human experience, and for the neuronal representation of these perceptual dimensions, previous studies point to the involvement of the right-hemispheric intraparietal sulcus and structures in the medial temporal lobe. Here we used multi-voxel pattern analysis (MVPA) to investigate long-term memory traces for temporal and spatial stimulus features in those areas. Participants were trained on four images associated with short versus long durations and with left versus right locations. Our results demonstrate stable representations of both temporal and spatial information in the right posterior intraparietal sulcus. Building upon previous findings of stable neuronal codes for directly perceived durations and locations, these results show that the reactivation of long-term memory traces for temporal and spatial features can be decoded from neuronal activation patterns in the right parietal cortex.

Ebbinghaus, Müller-Lyer, and Ponzo: Three examples of bidirectional space-time interference

Previous studies have shown interference between illusory size and perceived duration. The present study replicated this space-time interference in three classic visual-spatial illusions, the Ebbinghaus, the Müller-Lyer, and the Ponzo illusion. The results showed bidirectional interference between illusory size and duration for all three illusions. That is, subjectively larger stimuli were judged to be presented longer, and stimuli that were presented longer were judged to be larger. Thus, cross-dimensional interference between illusory size and duration appears to be a robust phenomenon and to generalize across a wide range of visual size illusions. This space-time interference most likely arises at the memory level and supports the theoretical notion of a common representational metric for space and time.

Time and visual-spatial illusions: Evidence for cross-dimensional interference between duration and illusory size

Time and space are intimately related to each other. Previous evidence has shown that stimulus size can affect perceived duration even when size differences are illusory. In the present study, we investigated the effect of visual-spatial illusions on duration judgments in a temporal reproduction paradigm. Specifically, we induced the Ebbinghaus illusion (Exp. 1) and the horizontal-vertical illusion (Exp. 2) during the encoding phase of the target interval or the reproduction phase. The results showed (a) that illusory size affects temporal processing similarly to the way physical size does, (b) that the effect is independent of whether the illusion appeared during encoding or reproduction, and (c) that the interference between size and temporal processing is bidirectional. These results suggest a rather late locus of size-time interference in the processing stream.

On the interplay between time and space perception in discontinuous stimulus displays

The present study examined whether and how the mutual perceptual biases of temporal and spatial information, known as the kappa and the tau effects, depend on the duration and spatial extent of sensory stimulation as well as on the magnitude of spatio-temporal discrepancy. Three small circles were presented in succession at different spatial positions. The time points of presentation and the spatial position of the second circle systematically varied. Participants judged either whether the temporal interval between the first and the second circle was longer than the interval between the second and the third circle (Experiment 1) or whether the spatial distance between the first and the second circle was larger than the distance between the second and the third circle (Experiment 2), or both in separate blocks of trials (Experiment 3). The impact of spatial information on temporal perception (i.e., the kappa effect) increased with velocity of motion presumably imputed by the participants to the static displays and decreased with spatio-temporal discrepancy. No inverse biases (i.e., no tau effects) were observed. These results are considered as an indication that integration of spatial and temporal signals follow the same basic principles as multisensory integration of redundant signals, such as those from vision and touch.

Encoding, working memory, or decision: how feedback modulates time perception

The hypothesis that individuals can accurately represent temporal information within approximately 3 s is the premise of several theoretical models and empirical studies in the field of temporal processing. The significance of accurately representing time within 3 s and the universality of the overestimation contrast dramatically. To clarify whether this overestimation arises from an inability to accurately represent time or a response bias, we systematically examined whether feedback reduces overestimation at the 3 temporal processing stages of timing (encoding), working memory, and decisions proposed by the scalar timing model. Participants reproduced the time interval between 2 circles with or without feedback, while the electroencephalogram (EEG) was synchronously recorded. Behavioral results showed that feedback shortened reproduced times and significantly minimized overestimation. EEG results showed that feedback significantly decreased the amplitude of contingent negative variation (CNV) in the decision stage but did not modulate the CNV amplitude in the encoding stage or the P2-P3b amplitudes in the working memory stage. These results suggest that overestimation arises from response bias when individuals convert an accurate representation of time into behavior. Our study provides electrophysiological evidence to support the conception that short intervals under approximately 3 s can be accurately represented as "temporal gestalt."

Near-optimal integration of the magnitude information of time and numerosity

Magnitude information is often correlated in the external world, providing complementary information about the environment. As if to reflect this relationship, the perceptions of different magnitudes (e.g. time and numerosity) are known to influence one another. Recent studies suggest that such magnitude interaction is similar to cue integration, such as multisensory integration. Here, we tested whether human observers could integrate the magnitudes of two quantities with distinct physical units (i.e. time and numerosity) as abstract magnitude information. The participants compared the magnitudes of two visual stimuli based on time, numerosity, or both. Consistent with the predictions of the maximum-likelihood estimation model, the participants integrated time and numerosity in a near-optimal manner; the weight of each dimension was proportional to their relative reliability, and the integrated estimate was more reliable than either the time or numerosity estimate. Furthermore, the integration approached a statistical optimum as the temporal discrepancy of the acquisition of each piece of information became smaller. These results suggest that magnitude interaction arises through a similar computational mechanism to cue integration. They are also consistent with the idea that different magnitudes are processed by a generalized magnitude system.

Exploring spatiotemporal interactions: On the superiority of time over space

Space and time mutually influence each other such that space affects time estimation (space-on-time effect), and conversely (time-on-space effect). These reciprocal interferences suggest that space and time are intrinsically linked in the human mind. Yet, recent evidence for an asymmetrical advantage for space over time challenges the classical theoretical interpretation. In the present study, we tested whether the superiority of space over time in magnitude interference depends on the cognitive resources engaged in the spatial task. We conducted three experiments in which participants performed judgments on temporal intervals and spatial distances in separate blocks. In each trial, two dots were successively flashed at various locations, and participants were to judge whether the duration or distance between the dots was short or long. To manipulate cognitive demands in the spatial task, distances varied across experiments (highly discriminable for the non-demanding spatial task in Experiment 1 and scarcely discriminable for the demanding spatial task in Experiment 2). Importantly, this manipulation tended to enhance perceptual sensitivity (as indexed by Weber Ratios) but slowed down the decision process (as indexed by response times) in the demanding experiment. Our results provide evidence for robust space-on-time and time-on-space effects (Experiments 1 and 2). More crucially, the involvement of cognitive resources in a demanding spatial task causes a massive time-on-space effect: Spatial judgments are indeed more influenced by irrelevant temporal information than the reverse (Experiments 2 and 3). Overall, the flexibility of spatiotemporal interferences has direct theoretical implications and questions the origins of space-time interaction.

Number-time interaction: Search for a common magnitude system in a cross-modal setting

A theory of magnitude (ATOM) suggests that a generalized magnitude system in the brain processes magnitudes such as space, time, and numbers. Numerous behavioral and neurocognitive studies have provided support to ATOM theory. However, the evidence for common magnitude processing primarily comes from the studies in which numerical and temporal information are presented visually. Our current understanding of such cross-dimensional magnitude interactions is limited to visual modality only. However, it is still unclear whether the ATOM-framework accounts for the integration of cross-modal magnitude information. To examine the cross-modal influence of numerical magnitude on temporal processing of the tone, we conducted three experiments using a temporal bisection task. We presented the numerical magnitude information in the visual domain and the temporal information in the auditory either simultaneously with duration judgment task (Experiment-1), before duration judgment task (Experiment-2), and before duration judgment task but with numerical magnitude also being task-relevant (Experiment-3). The results suggest that the numerical information presented in the visual domain affects temporal processing of the tone only when the numerical magnitudes were task-relevant and available while making a temporal judgment (Experiments-1 and 3). However, numerical information did not interfere with temporal information when presented temporally separated from the duration information (Experiments-2). The findings indicate that the influence of visual numbers on temporal processing in cross-modal settings may not arise from the common magnitude system but instead from general cognitive mechanisms like attention and memory.

Cross-dimensional magnitude interactions reflect statistical correlations among physical dimensions: Evidence from space-time interaction

Magnitudes of different physical dimensions have been assumed to be processed by a common metric in order to account for interactions between different dimensions (e.g., space, time). This paper tested a different hypothesis, that these cross-dimensional interactions reflect people's experience of statistical correlations among physical dimensions. In the experiment, we manipulated the correlation between space (length) and time (duration). A stimulus consisting of two vertical bars that demarcated a variable stimulus length was presented for a variable stimulus duration; participants were to reproduce either the stimulus length or the stimulus duration. Critically, to reproduce a stimulus length, participants held down the spacebar to grow or shrink (in a blocked design) a length to the stimulus length such that space (i.e. reproduced length) positively or negatively co-varied with time. Reproduced lengths did not vary as a function of stimulus duration under positive space-time correlation but decreased as a function of stimulus duration under negative space-time correlation; reproduced durations increased as a function of stimulus length under positive space-time correlation but this space-on-time effect appeared to be attenuated under negative space-time correlation. These findings are consistent with a Bayesian inference account whereby cross-dimensional interactions reflect people's prior belief/knowledge of cross-dimensional statistical correlation, which itself tunes to recent input.

Tau and kappa in interception - how perceptual spatiotemporal interrelations affect movements

Batting and catching are real-life examples of interception. Due to latencies between the processing of sensory input and the corresponding motor response, successful interception requires accurate spatiotemporal prediction. However, spatiotemporal predictions can be subject to bias. For instance, the more spatially distant two sequentially presented objects are, the longer the interval between their presentations is perceived (kappa effect) and vice versa (tau effect). In this study, we deployed these phenomena to test in two sensory modalities whether temporal representations depend asymmetrically on spatial representations, or whether both are symmetrically interrelated. We adapted the tau and kappa paradigms to an interception task by presenting four stimuli (visually or auditorily) one after another on four locations, from left to right, with constant spatial and temporal intervals in between. In two experiments, participants were asked to touch the screen where and when they predicted a fifth stimulus to appear. In Exp. 2, additional predictive gaze measures were examined. Across experiments, auditory but not visual stimuli produced a tau effect for interception, supporting the idea that the relationship between space and time is moderated by the sensory modality. Results did not reveal classical auditory or visual kappa effects and no visual tau effects. Gaze data in Exp. 2 showed that the (spatial) gaze orientation depended on temporal intervals while the timing of fixations was modulated by spatial intervals, thereby indicating tau and kappa effects across modalities. Together, the results suggest that sensory modality plays an important role in spatiotemporal predictions in interception.

Electrophysiological Evidence for a Common Magnitude Representation of Spatiotemporal Information in Working Memory

Spatiotemporal interference has attracted increasing attention because it provides a window for studying the neural representation of magnitude in the brain. We aimed to identify the neural basis of spatiotemporal interference using a Kappa effect task in which two circles were presented in sequence with two time intervals and three space distances. Participants reproduced the time intervals while ignoring the space distance when electroencephalogram signals were recorded synchronously. The behavior results showed that production time increased with time interval and space distance. Offset of the time intervals elicited typical P2 and P3b components. Larger parietal P2 and P3b amplitudes were elicited by the combination of longer time intervals and longer space distances. The parietal P2 and P3b amplitudes were positively correlated with the production time, and the corresponding neural source was located in the parietal cortex. The results suggest that the parietal P2 and P3b index updates a common representation of spatiotemporal information in working memory, which provides electrophysiological evidence for the mechanisms underlying spatiotemporal interferences. Our study supports a theory of magnitude, in which different dimensions can be integrated into a common magnitude representation in a generalized magnitude system that is localized at the parietal cortex.