The perceived present: What is it, and what is it there for?

The perceptual timescape: Perceptual history on the sub-second scale

There is a high-capacity store of brief time span (∼1000 ms) which information enters from perceptual processing, often called iconic memory or sensory memory. It is proposed that a main function of this store is to hold recent perceptual information in a temporally segregated representation, named the perceptual timescape. The perceptual timescape is a continually active representation of change and continuity over time that endows the perceived present with a perceived history. This is accomplished primarily by two kinds of time marking information: time distance information, which marks all items of information in the perceptual timescape according to how far in the past they occurred, and ordinal temporal information, which organises items of information in terms of their temporal order. Added to that is information about connectivity of perceptual objects over time. These kinds of information connect individual items over a brief span of time so as to represent change, persistence, and continuity over time. It is argued that there is a one-way street of information flow from perceptual processing either to the perceived present or directly into the perceptual timescape, and thence to working memory. Consistent with that, the information structure of the perceptual timescape supports postdictive reinterpretations of recent perceptual information. Temporal integration on a time scale of hundreds of milliseconds takes place in perceptual processing and does not draw on information in the perceptual timescape, which is concerned with temporal segregation, not integration.

A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time

This article presents an overview of a theory for performing temporal smoothing on temporal signals in such a way that: (i) temporally smoothed signals at coarser temporal scales are guaranteed to constitute simplifications of corresponding temporally smoothed signals at any finer temporal scale (including the original signal) and (ii) the temporal smoothing process is both time-causal and time-recursive, in the sense that it does not require access to future information and can be performed with no other temporal memory buffer of the past than the resulting smoothed temporal scale-space representations themselves. For specific subsets of parameter settings for the classes of linear and shift-invariant temporal smoothing operators that obey this property, it is shown how temporal scale covariance can be additionally obtained, guaranteeing that if the temporal input signal is rescaled by a uniform temporal scaling factor, then also the resulting temporal scale-space representations of the rescaled temporal signal will constitute mere rescalings of the temporal scale-space representations of the original input signal, complemented by a shift along the temporal scale dimension. The resulting time-causal limit kernel that obeys this property constitutes a canonical temporal kernel for processing temporal signals in real-time scenarios when the regular Gaussian kernel cannot be used, because of its non-causal access to information from the future, and we cannot additionally require the temporal smoothing process to comprise a complementary memory of the past beyond the information contained in the temporal smoothing process itself, which in this way also serves as a multi-scale temporal memory of the past. We describe how the time-causal limit kernel relates to previously used temporal models, such as Koenderink's scale-time kernels and the ex-Gaussian kernel. We do also give an overview of how the time-causal limit kernel can be used for modelling the temporal processing in models for spatio-temporal and spectro-temporal receptive fields, and how it more generally has a high potential for modelling neural temporal response functions in a purely time-causal and time-recursive way, that can also handle phenomena at multiple temporal scales in a theoretically well-founded manner. We detail how this theory can be efficiently implemented for discrete data, in terms of a set of recursive filters coupled in cascade. Hence, the theory is generally applicable for both: (i) modelling continuous temporal phenomena over multiple temporal scales and (ii) digital processing of measured temporal signals in real time. We conclude by stating implications of the theory for modelling temporal phenomena in biological, perceptual, neural and memory processes by mathematical models, as well as implications regarding the philosophy of time and perceptual agents. Specifically, we propose that for A-type theories of time, as well as for perceptual agents, the notion of a non-infinitesimal inner temporal scale of the temporal receptive fields has to be included in representations of the present, where the inherent nonzero temporal delay of such time-causal receptive fields implies a need for incorporating predictions from the actual time-delayed present in the layers of a perceptual hierarchy, to make it possible for a representation of the perceptual present to constitute a representation of the environment with timing properties closer to the actual present.

Time marking in perception

Several authors have proposed that perceptual information carries labels that identify temporal features, including time of occurrence, ordinal temporal relations, and brief durations. These labels serve to locate and organise perceptual objects, features, and events in time. In some proposals time marking has local, specific functions such as synchronisation of different features in perceptual processing. In other proposals time marking has general significance and is responsible for rendering perceptual experience temporally coherent, just as various forms of spatial information render the visual environment spatially coherent. These proposals, which all concern time marking on the millisecond time scale, are reviewed. It is concluded that time marking is vital to the construction of a multisensory perceptual world in which things are orderly with respect to both space and time, but that much more research is needed to ascertain its functions in perception and its neurophysiological foundations.

Perception in real-time: predicting the present, reconstructing the past

We feel that we perceive events in the environment as they unfold in real-time. However, this intuitive view of perception is impossible to implement in the nervous system due to biological constraints such as neural transmission delays. I propose a new way of thinking about real-time perception: at any given moment, instead of representing a single timepoint, perceptual mechanisms represent an entire timeline. On this timeline, predictive mechanisms predict ahead to compensate for delays in incoming sensory input, and reconstruction mechanisms retroactively revise perception when those predictions do not come true. This proposal integrates and extends previous work to address a crucial gap in our understanding of a fundamental aspect of our everyday life: the experience of perceiving the present.

Perception of Happening: How the Brain Deals with the No-History Problem

In physics, the temporal dimension has units of infinitesimally brief duration. Given this, how is it possible to perceive things, such as motion, music, and vibrotactile stimulation, that involve extension across many units of time? To address this problem, it is proposed that there is what is termed an "information construct of happening" (ICOH), a simultaneous representation of recent, temporally differentiated perceptual information on the millisecond time scale. The main features of the ICOH are (i) time marking, semantic labeling of all information in the ICOH with ordinal temporal information and distance from what is informationally identified as the present moment, (ii) vector informational features that specify kind, direction, and rate of change for every feature in a percept, and (iii) connectives, information relating vector informational features at adjacent temporal locations in the ICOH. The ICOH integrates products of perceptual processing with recent historical information in sensory memory on the subsecond time scale. Perceptual information about happening in informational sensory memory is encoded in semantic form that preserves connected semantic trails of vector and timing information. The basic properties of the ICOH must be supported by a general and widespread timing mechanism that generates ordinal and interval timing information and it is suggested that state-dependent networks may suffice for that purpose. Happening, therefore, is perceived at a moment and is constituted by an information structure of connected recent historical information.

The reconstructive nature of involuntary autobiographical memories

Influential clinical theories propose that parts of traumatic or stressful events are stored in an unprocessed, purely perceptual form, which is inaccessible for voluntary retrieval but may spontaneously return to consciousness involuntarily in response to matching cues. This leads to perceptually vivid, and largely veridical involuntary memories of the traumatic scene, remembered with the original emotions and from the same vantage point as during the event. Several lines of evidence contradict this position. First, involuntary memories are not more veridical than memories retrieved deliberately. Second, involuntary memories for trauma-related events are not more frequently remembered with the original first-person perspective, and individuals with PTSD do not report more first-person involuntary memories than individuals without PTSD. Third, involuntary memories of stressful moments that are subjectively experienced as repetitive do not come to mind in a fixed and unchangeable form. Fourth, involuntary memories do not have privileged access to the most emotional moments (so-called hotspots) of a stressful event and the content and choice of hotspots change over time. Fourth, although involuntary memories are associated with enhanced emotional impact, this does not imply reactivation of the original emotion. We conclude that involuntary memories, although effortless, are products of (re)constructive processes.