All in Good Time: Long-Lasting Postdictive Effects Reveal Discrete Perception

Temporal windows of unconscious processing cannot easily be disrupted

Conscious perception is preceded by long periods of unconscious processing. These periods are crucial for analyzing temporal information and for solving the many ill-posed problems of vision. An important question is what starts and ends these windows and how they may be interrupted. Most experimental paradigms do not offer the methodology required for such investigation. Here, we used the sequential metacontrast paradigm, in which two streams of lines, expanding from the center to the periphery, are presented, and participants are asked to attend to one of the motion streams. If several lines in the attended motion stream are offset, the offsets are known to integrate mandatorily and unconsciously, even if separated by up to 450 ms. Using this paradigm, we here found that external visual objects, such as an annulus, presented during the motion stream, do not disrupt mandatory temporal integration. Thus, if a window is started once, it appears to remain open even in the presence of disruptions that are known to interrupt visual processes normally. Further, we found that interrupting the motion stream with a gap disrupts temporal integration but does not terminate the overall unconscious processing window. Thus, while temporal integration is key to unconscious processing, not all stimuli in the same processing window are integrated together. These results strengthen the case for unconscious processing taking place in windows of sensemaking, during which temporal integration occurs in a flexible and perceptually meaningful manner.

Alpha-Band Frequency and Temporal Windows in Perception: A Review and Living Meta-analysis of 27 Experiments (and Counting)

Temporal windows in perception refer to windows of time within which distinct stimuli interact to influence perception. A simple example is two temporally proximal stimuli fusing into a single percept. It has long been hypothesized that the human alpha rhythm (an 8- to 13-Hz neural oscillation maximal over posterior cortex) is linked to temporal windows, with higher frequencies corresponding to shorter windows and finer-grained temporal resolution. This hypothesis has garnered support from studies demonstrating a correlation between individual differences in alpha-band frequency (IAF) and behavioral measures of temporal processing. However, nonsignificant effects have also been reported. Here, we review and meta-analyze 27 experiments correlating IAF with measures of visual and audiovisual temporal processing. Our results estimate the true correlation in the population to be between .39 and .53, a medium-to-large effect. The effect held when considering visual or audiovisual experiments separately, when examining different IAF estimation protocols (i.e., eyes open and eyes closed), and when using analysis choices that favor a null result. Our review shows that (1) effects have been internally and independently replicated, (2) several positive effects are based on larger sample sizes than the null effects, and (3) many reported null effects are actually in the direction predicted by the hypothesis. A free interactive web app was developed to allow users to replicate our meta-analysis and change or update the study selection at will, making this a "living" meta-analysis (randfxmeta.streamlit.app). We discuss possible factors underlying null reports, design recommendations, and open questions for future research.

New Evidence for Retrospectively Cued Perception

Past research suggests a continuity between perception and memory, as reflected in influences of orienting of spatial attention by cues presented after a visual target offset (post-target cues) on target perception. Conducting two experiments, we tested and confirmed this claim. Our study revealed an elevated reliance on post-target cues for target detection with diminishing target visibility, leading to better performance in validly versus invalidly cued trials, indicative of contrast gain. We demonstrated this post-target cueing impact on target perception without a postcue response prompt, meaning that our results truly reflected a continuity between perception and memory rather than a task-specific impact of having to memorize the target due to a response prompt. While previous studies found an improvement in accuracy through valid compared to invalid cues using liminal targets, in Experiment 1, we further showed an influence of attention on participants' response time by the post-target cues with cues presented away from a clearly visible target. This suggests that visual interactions at the target location provided no better explanation of post-target cueing effects. Our results generalize prior research with liminal targets and confirm the view of a perception-memory continuum so that visual target processing is not shielded against visuospatial orienting of attention elicited by events following the offset of the visual target.

Procedure for extracting temporal structure embedded within psychophysical data

The idea that mental events unfold over time with an intrinsically paced regularity has a long history within experimental psychology, and it has gained traction from the actual measurement of brain rhythms evident in EEG signals recorded from the human brain and from direct recordings of action potentials and local field potentials within the nervous systems of nonhumans. The weak link in this idea, however, is the challenge of extracting signatures of this temporal structure from behavioral measures. Because there is nothing in the seamless stream of conscious awareness that belies rhythmic modulations in sensitivity or mental acuity, one must deploy inferential strategies for extracting evidence for the existence of temporal regularities in neural activity. We have devised a parametric procedure for analysis of temporal structure embedded in behaviorally measured data comprising durations. We confirm that this procedure, dubbed PATS, achieves comparable results to those obtained using spectral analysis, and that it outperforms conventional spectral analysis when analyzing human response time data containing just a few hundred data points per condition. PATS offers an efficient, sensitive means for bridging the gap between oscillations identified neurophysiologically and estimates of rhythmicity embedded within durations measured behaviorally.

The moment of awareness influences the content of awareness in orientation repulsion

Through the neurally evolving process of dynamic contextual modulation of perceptual contents, it remains unclear how the content of awareness is determined. Here we quantified the visual illusion of orientation repulsion, wherein the target appears tilted against the surrounding's orientation, and examined whether its extent changed when the target awareness was quickened by a preceding flanker. Independently of spatial cueing, repulsion was reduced when the flanker preceded the target by 100 ms compared with when they appeared simultaneously. We confirmed that the preceding flanker quickened the awareness of a nearby target relative to distant ones by 40 ms. Furthermore, the preceding flanker that was greater than 7 degrees away from the target still evoked such reduction of repulsion. These findings imply that the content of awareness is determined by the temporal interaction of two distinct processes: one controls the moment of awareness, and the other represents the perceptual content.

Pre-stimulus alpha activity modulates long-lasting unconscious feature integration

Pre-stimulus alpha (α) activity can influence perception of shortly presented, low-contrast stimuli. The underlying mechanisms are often thought to affect perception exactly at the time of presentation. In addition, it is suggested that α cycles determine temporal windows of integration. However, in everyday situations, stimuli are usually presented for periods longer than ∼100 ms and perception is often an integration of information across space and time. Moving objects are just one example. Hence, the question is whether α activity plays a role also in temporal integration, especially when stimuli are integrated over several α cycles. Using electroencephalography (EEG), we investigated the relationship between pre-stimulus brain activity and long-lasting integration in the sequential metacontrast paradigm (SQM), where two opposite vernier offsets, embedded in a stream of lines, are unconsciously integrated into a single percept. We show that increases in α power, even 300 ms before the stimulus, affected the probability of reporting the first offset, shown at the very beginning of the SQM. This effect was mediated by the systematic slowing of the α rhythm that followed the peak in α power. No phase effects were found. Together, our results demonstrate a cascade of neural changes, following spontaneous bursts of α activity and extending beyond a single moment, which influences the sensory representation of visual features for hundreds of milliseconds. Crucially, as feature integration in the SQM occurs before a conscious percept is elicited, this also provides evidence that α activity is linked to mechanisms regulating unconscious processing.

Processing load, and not stimulus evidence, determines the duration of unconscious visual feature integration

Integration across space and time is essential for the analysis of motion, low contrast, and many more stimuli. A crucial question is what determines the duration of integration. Based on classical models of decision-making, one might expect that integration terminates as soon as sufficient evidence about a stimulus is accumulated and a threshold is crossed. However, there is very little research on this question as most experimental paradigms cannot monitor processing following stimulus presentation. In particular, it is difficult to determine when processing terminates. Here, using the sequential metacontrast paradigm (SQM), in which information is mandatorily integrated along motion trajectories, we show that the processing load determines the extent of integration but that evidence accumulation does not. Further, the extent of integration is determined by absolute time instead of the number of elements presented. These results have important implications for understanding the time course and mechanisms of temporal integration.

Holistic temporal order judgment of tones requires top-down disentanglement

How temporal sequence gets organized is a central topic in cognitive processing. In a high-frequency time window of tens of milliseconds, the temporal order is reconstructed rather than mirroring the sequence of events objectively in physical time. Two separate phases or strategies, a holistic coding phase that groups successively presented events as a gestalt and a disentanglement phase that decodes the temporal order of discrete events from the gestalt representation, may presumably be involved in the perception of temporal order across different modalities. With a temporal order adaptation protocol of pure tones using glide adaptors, the present study demonstrated a dissociation between constant discriminability and shifted subjective simultaneity across different adaptor directions. While discriminability of temporal order was not adapted by glides, revealing a constant coding sensitivity of different asynchronies, the shift of subjective simultaneity indicated the recalibration of a top-down disentanglement of the holistic processing under the influence of glide adaptors. The results suggest a dual-phase holistic processing in temporal order perception, supporting two separate cognitive strategies for event timing on the sub-second level.

Distinct ventral stream and prefrontal cortex representational dynamics during sustained conscious visual perception

Instances of sustained stationary sensory input are ubiquitous. However, previous work focused almost exclusively on transient onset responses. This presents a critical challenge for neural theories of consciousness, which should account for the full temporal extent of experience. To address this question, we use intracranial recordings from ten human patients with epilepsy to view diverse images of multiple durations. We reveal that, in sensory regions, despite dramatic changes in activation magnitude, the distributed representation of categories and exemplars remains sustained and stable. In contrast, in frontoparietal regions, we find transient content representation at stimulus onset. Our results highlight the connection between the anatomical and temporal correlates of experience. To the extent perception is sustained, it may rely on sensory representations and to the extent perception is discrete, centered on perceptual updating, it may rely on frontoparietal representations.

Perceptual stability reflected in neuronal pattern similarities in human visual cortex

The magnitude of neuronal activation is commonly considered a critical factor for conscious perception of visual content. However, this dogma contrasts with the phenomenon of rapid adaptation, in which the magnitude of neuronal activation drops dramatically in a rapid manner while the visual stimulus and the conscious experience it elicits remain stable. Here, we report that the profiles of multi-site activation patterns and their relational geometry-i.e., the similarity distances between activation patterns, as revealed using intracranial electroencephalographic (iEEG) recordings-are sustained during extended visual stimulation despite the major magnitude decrease. These results are compatible with the hypothesis that conscious perceptual content is associated with the neuronal pattern profiles and their similarity distances, rather than the overall activation magnitude, in human visual cortex.

A systematic approach to brain dynamics: cognitive evolution theory of consciousness

The brain integrates volition, cognition, and consciousness seamlessly over three hierarchical (scale-dependent) levels of neural activity for their emergence: a causal or 'hard' level, a computational (unconscious) or 'soft' level, and a phenomenal (conscious) or 'psyche' level respectively. The cognitive evolution theory (CET) is based on three general prerequisites: physicalism, dynamism, and emergentism, which entail five consequences about the nature of consciousness: discreteness, passivity, uniqueness, integrity, and graduation. CET starts from the assumption that brains should have primarily evolved as volitional subsystems of organisms, not as prediction machines. This emphasizes the dynamical nature of consciousness in terms of critical dynamics to account for metastability, avalanches, and self-organized criticality of brain processes, then coupling it with volition and cognition in a framework unified over the levels. Consciousness emerges near critical points, and unfolds as a discrete stream of momentary states, each volitionally driven from oldest subcortical arousal systems. The stream is the brain's way of making a difference via predictive (Bayesian) processing. Its objective observables could be complexity measures reflecting levels of consciousness and its dynamical coherency to reveal how much knowledge (information gain) the brain acquires over the stream. CET also proposes a quantitative classification of both disorders of consciousness and mental disorders within that unified framework.

A survey of neurophysiological differentiation across mouse visual brain areas and timescales

Neurophysiological differentiation (ND), a measure of the number of distinct activity states that a neural population visits over a time interval, has been used as a correlate of meaningfulness or subjective perception of visual stimuli. ND has largely been studied in non-invasive human whole-brain recordings where spatial resolution is limited. However, it is likely that perception is supported by discrete neuronal populations rather than the whole brain. Therefore, here we use Neuropixels recordings from the mouse brain to characterize the ND metric across a wide range of temporal scales, within neural populations recorded at single-cell resolution in localized regions. Using the spiking activity of thousands of simultaneously recorded neurons spanning 6 visual cortical areas and the visual thalamus, we show that the ND of stimulus-evoked activity of the entire visual cortex is higher for naturalistic stimuli relative to artificial ones. This finding holds in most individual areas throughout the visual hierarchy. Moreover, for animals performing an image change detection task, ND of the entire visual cortex (though not individual areas) is higher for successful detection compared to failed trials, consistent with the assumed perception of the stimulus. Together, these results suggest that ND computed on cellular-level neural recordings is a useful tool highlighting cell populations that may be involved in subjective perception.

Wrinkles in subsecond time perception are synchronized to the heart

The role of the heart in the experience of time has been long theorized but empirical evidence is scarce. Here, we examined the interaction between fine-grained cardiac dynamics and the momentary experience of subsecond intervals. Participants performed a temporal bisection task for brief tones (80-188 ms) synchronized with the heart. We developed a cardiac Drift-Diffusion Model (cDDM) that embedded contemporaneous heart rate dynamics into the temporal decision model. Results revealed the existence of temporal wrinkles-dilation or contraction of short intervals-in synchrony with cardiac dynamics. A lower prestimulus heart rate was associated with an initial bias in encoding the millisecond-level stimulus duration as longer, consistent with facilitation of sensory intake. Concurrently, a higher prestimulus heart rate aided more consistent and faster temporal judgments through more efficient evidence accumulation. Additionally, a higher speed of poststimulus cardiac deceleration, a bodily marker of attention, was associated with a greater accumulation of sensory temporal evidence in the cDDM. These findings suggest a unique role of cardiac dynamics in the momentary experience of time. Our cDDM framework opens a new methodological avenue for investigating the role of the heart in time perception and perceptual judgment.

Time marking in perception

Several authors have proposed that perceptual information carries labels that identify temporal features, including time of occurrence, ordinal temporal relations, and brief durations. These labels serve to locate and organise perceptual objects, features, and events in time. In some proposals time marking has local, specific functions such as synchronisation of different features in perceptual processing. In other proposals time marking has general significance and is responsible for rendering perceptual experience temporally coherent, just as various forms of spatial information render the visual environment spatially coherent. These proposals, which all concern time marking on the millisecond time scale, are reviewed. It is concluded that time marking is vital to the construction of a multisensory perceptual world in which things are orderly with respect to both space and time, but that much more research is needed to ascertain its functions in perception and its neurophysiological foundations.

Blurred Lines: Memory, Perceptions, and Consciousness: Commentary on "Consciousness as a Memory System" by Budson et al (2022)

In the previous issue, Budson, Richman, and Kensinger (2022) put forth the intriguing proposal that consciousness may have evolved from the episodic memory system. In addition to providing a possible evolutionary trajectory for consciousness, I believe that viewing consciousness as an extension of memory in this way is particularly useful for understanding some of the puzzling temporal complexities that are inherent to consciousness. For example, due to neural transmission delays, our conscious experience must necessarily lag the outside world, which creates a paradox for both conscious perception (Do we see the past, rather than the present?) and action (How can we make rapid decisions if it takes so long to become conscious of something?). These paradoxes can be elegantly solved by treating consciousness as a memory system. Finally, the proposal put forth by Budson and colleagues (2022) aligns with the emerging perspective that consciousness, like memory, represents a narrative time line of events rather than any single instant. However, I believe that this conceptualization can be further extended to include not only the past, but also the future. In this way, consciousness can be provocatively viewed as the remembered past, present, and future.

Perceptual Functioning

Perceptual disorders are not part of the diagnosis criteria for schizophrenia. Yet, a considerable amount of work has been conducted, especially on visual perception abnormalities, and there is little doubt that visual perception is altered in patients. There are several reasons why such perturbations are of interest in this pathology. They are observed during the prodromal phase of psychosis, they are related to the pathophysiology (clinical disorganization, disorders of the sense of self), and they are associated with neuronal connectivity disorders. Perturbations occur at different levels of processing and likely affect how patients interact and adapt to their surroundings. The literature has become very large, and here we try to summarize different models that have guided the exploration of perception in patients. We also illustrate several lines of research by showing how perception has been investigated and by discussing the interpretation of the results. In addition to discussing domains such as contrast sensitivity, masking, and visual grouping, we develop more recent fields like processing at the level of the retina, and the timing of perception.

Consciousness as a Memory System

We suggest that there is confusion between why consciousness developed and what additional functions, through continued evolution, it has co-opted. Consider episodic memory. If we believe that episodic memory evolved solely to accurately represent past events, it seems like a terrible system-prone to forgetting and false memories. However, if we believe that episodic memory developed to flexibly and creatively combine and rearrange memories of prior events in order to plan for the future, then it is quite a good system. We argue that consciousness originally developed as part of the episodic memory system-quite likely the part needed to accomplish that flexible recombining of information. We posit further that consciousness was subsequently co-opted to produce other functions that are not directly relevant to memory per se, such as problem-solving, abstract thinking, and language. We suggest that this theory is compatible with many phenomena, such as the slow speed and the after-the-fact order of consciousness, that cannot be explained well by other theories. We believe that our theory may have profound implications for understanding intentional action and consciousness in general. Moreover, we suggest that episodic memory and its associated memory systems of sensory, working, and semantic memory as a whole ought to be considered together as the conscious memory system in that they, together, give rise to the phenomenon of consciousness. Lastly, we suggest that the cerebral cortex is the part of the brain that makes consciousness possible, and that every cortical region contributes to this conscious memory system.

Non-Separability of Physical Systems as a Foundation of Consciousness

A hypothesis is presented that non-separability of degrees of freedom is the fundamental property underlying consciousness in physical systems. The amount of consciousness in a system is determined by the extent of non-separability and the number of degrees of freedom involved. Non-interacting and feedforward systems have zero consciousness, whereas most systems of interacting particles appear to have low non-separability and consciousness. By contrast, brain circuits exhibit high complexity and weak but tightly coordinated interactions, which appear to support high non-separability and therefore high amount of consciousness. The hypothesis applies to both classical and quantum cases, and we highlight the formalism employing the Wigner function (which in the classical limit becomes the Liouville density function) as a potentially fruitful framework for characterizing non-separability and, thus, the amount of consciousness in a system. The hypothesis appears to be consistent with both the Integrated Information Theory and the Orchestrated Objective Reduction Theory and may help reconcile the two. It offers a natural explanation for the physical properties underlying the amount of consciousness and points to methods of estimating the amount of non-separability as promising ways of characterizing the amount of consciousness.

How the Brain Becomes the Mind: Can Thermodynamics Explain the Emergence and Nature of Emotions?

The neural systems' electric activities are fundamental for the phenomenology of consciousness. Sensory perception triggers an information/energy exchange with the environment, but the brain's recurrent activations maintain a resting state with constant parameters. Therefore, perception forms a closed thermodynamic cycle. In physics, the Carnot engine is an ideal thermodynamic cycle that converts heat from a hot reservoir into work, or inversely, requires work to transfer heat from a low- to a high-temperature reservoir (the reversed Carnot cycle). We analyze the high entropy brain by the endothermic reversed Carnot cycle. Its irreversible activations provide temporal directionality for future orientation. A flexible transfer between neural states inspires openness and creativity. In contrast, the low entropy resting state parallels reversible activations, which impose past focus via repetitive thinking, remorse, and regret. The exothermic Carnot cycle degrades mental energy. Therefore, the brain's energy/information balance formulates motivation, sensed as position or negative emotions. Our work provides an analytical perspective of positive and negative emotions and spontaneous behavior from the free energy principle. Furthermore, electrical activities, thoughts, and beliefs lend themselves to a temporal organization, an orthogonal condition to physical systems. Here, we suggest that an experimental validation of the thermodynamic origin of emotions might inspire better treatment options for mental diseases.

The Irreducibility of Vision: Gestalt, Crowding and the Fundamentals of Vision

What is fundamental in vision has been discussed for millennia. For philosophical realists and the physiological approach to vision, the objects of the outer world are truly given, and failures to perceive objects properly, such as in illusions, are just sporadic misperceptions. The goal is to replace the subjectivity of the mind by careful physiological analyses. Continental philosophy and the Gestaltists are rather skeptical or ignorant about external objects. The percepts themselves are their starting point, because it is hard to deny the truth of one own′s percepts. I will show that, whereas both approaches can well explain many visual phenomena with classic visual stimuli, they both have trouble when stimuli become slightly more complex. I suggest that these failures have a deeper conceptual reason, namely that their foundations (objects, percepts) do not hold true. I propose that only physical states exist in a mind independent manner and that everyday objects, such as bottles and trees, are perceived in a mind-dependent way. The fundamental processing units to process objects are extended windows of unconscious processing, followed by short, discrete conscious percepts.

Cautionary Observations Concerning the Introduction of Psychophysiological Biomarkers into Neuropsychiatric Practice

The combination of statistical learning technologies with large databases of psychophysiological data has appropriately generated enthusiastic interest in future clinical applicability. It is argued here that this enthusiasm should be tempered with the understanding that significant obstacles must be overcome before the systematic introduction of psychophysiological measures into neuropsychiatric practice becomes possible. The objective of this study is to identify challenges to this effort. The nonspecificity of psychophysiological measures complicates their use in diagnosis. Low test-retest reliability complicates use in longitudinal assessment, and quantitative psychophysiological measures can normalize in response to placebo intervention. Ten cautionary observations are introduced and, in some instances, possible directions for remediation are suggested.

Neural correlates of temporal integration and segregation in metacontrast masking: A phenomenological study

Temporal integration and segregation have been investigated both in the research on the temporal mechanisms in visual perception and in the research on visual masking. Although both research lines share theoretical, methodological, and empirical similarities, there is little overlap between them and their models of temporal processing are incompatible. As a first step toward the unification of both lines of research, we investigated the electrophysiological correlates of temporal integration and segregation in a metacontrast masking paradigm. Participants reported in each trial whether they perceived the target-mask sequence as a simultaneous or temporally segregated percept while their EEG was recorded. A comparison of both temporal report categories resulted in an ERP difference after stimulus presentation (200-450 ms) that closely resembles the contour integration negativity. Moreover, we found that phase states were shifted between perceptual report categories in the alpha (450-250 ms) and beta (225-125 ms) frequency band before stimulus presentation and induced a sinusoidal periodicity in later temporal report proportions. Thus, we show that neural correlates of temporal integration and segregation can be generalized to metacontrast masking. These findings emphasize the potential role of temporal mechanisms in the emergence of the masking phenomenon. Additionally, our findings validate our phenomenological approach by demonstrating similar neural correlates of temporal integration and segregation as in performance-based tasks. Future research may profit from our phenomenological approach to disentangle the (neural) interplay between temporal and masking mechanisms.

Modelling the simultaneous encoding/serial experience theory of the perceptual moment: a blink of meta-experience

One way to understand a system is to explore how its behaviour degrades when it is overloaded. This approach can be applied to understanding conscious perception by presenting stimuli in rapid succession in the 'same' perceptual event/moment. In previous work, we have identified a striking dissociation during the perceptual moment, between what is encoded into working memory [Lag-1 sparing in the attentional blink (AB)] and what is consciously perceived (Lag-1 impairing in the experiential blink). This paper links this dissociation to what, taking inspiration from the metacognition literature, could be called meta-experience; i.e. how the ability to track and comment on one's visual experience with subjectivity ratings reflects objective performance. Specifically, we provide evidence that the information (in bits) associated with an encoding into working memory decouples from the experiential reflection upon that perceptual/encoding event and that this decoupling is largest when there is the greatest perceptual overload. This is the meta-experiential blink. Meta-experiential self-observation is common to many computational models, including connectionist interpretations of consciousness, Bayesian observers and the readout-enhanced simultaneous type/serial token (reSTST) model. We assess how our meta-experiential blink data could be modelled using the concept of self-observation, providing model fits to behavioural and electroencephalogram responses in the reSTST model. We discuss the implications of our computational modelling of parallel encoding but serial experience for theories of conscious perception. Specifically, we (i) inform theories of Lag-1 sparing during the AB and (ii) consider the implications for the global workspace theory of conscious perception and higher-order theories of consciousness.

From Shorter to Longer Timescales: Converging Integrated Information Theory (IIT) with the Temporo-Spatial Theory of Consciousness (TTC)

Time is a key element of consciousness as it includes multiple timescales from shorter to longer ones. This is reflected in our experience of various short-term phenomenal contents at discrete points in time as part of an ongoing, more continuous, and long-term ‘stream of consciousness.’ Can Integrated Information Theory (IIT) account for this multitude of timescales of consciousness? According to the theory, the relevant spatiotemporal scale for consciousness is the one in which the system reaches the maximum cause-effect power; IIT currently predicts that experience occurs on the order of short timescales, namely, between 100 and 300 ms (theta and alpha frequency range). This can well account for the integration of single inputs into a particular phenomenal content. However, such short timescales leave open the temporal relation of specific phenomenal contents to others during the course of the ongoing time, that is, the stream of consciousness. For that purpose, we converge the IIT with the Temporo-spatial Theory of Consciousness (TTC), which, assuming a multitude of different timescales, can take into view the temporal integration of specific phenomenal contents with other phenomenal contents over time. On the neuronal side, this is detailed by considering those neuronal mechanisms driving the non-additive interaction of pre-stimulus activity with the input resulting in stimulus-related activity. Due to their non-additive interaction, the single input is not only integrated with others in the short-term timescales of 100–300 ms (alpha and theta frequencies) (as predicted by IIT) but, at the same time, also virtually expanded in its temporal (and spatial) features; this is related to the longer timescales (delta and slower frequencies) that are carried over from pre-stimulus to stimulus-related activity. Such a non-additive pre-stimulus-input interaction amounts to temporo-spatial expansion as a key mechanism of TTC for the constitution of phenomenal contents including their embedding or nesting within the ongoing temporal dynamic, i.e., the stream of consciousness. In conclusion, we propose converging the short-term integration of inputs postulated in IIT (100–300 ms as in the alpha and theta frequency range) with the longer timescales (in delta and slower frequencies) of temporo-spatial expansion in TTC.

Perception in real-time: predicting the present, reconstructing the past

We feel that we perceive events in the environment as they unfold in real-time. However, this intuitive view of perception is impossible to implement in the nervous system due to biological constraints such as neural transmission delays. I propose a new way of thinking about real-time perception: at any given moment, instead of representing a single timepoint, perceptual mechanisms represent an entire timeline. On this timeline, predictive mechanisms predict ahead to compensate for delays in incoming sensory input, and reconstruction mechanisms retroactively revise perception when those predictions do not come true. This proposal integrates and extends previous work to address a crucial gap in our understanding of a fundamental aspect of our everyday life: the experience of perceiving the present.

Representational 'touch' and modulatory 'retouch'-two necessary neurobiological processes in thalamocortical interaction for conscious experience

Theories of consciousness using neurobiological data or being influenced by these data have been focused either on states of consciousness or contents of consciousness. These theories have occasionally used evidence from psychophysical phenomena where conscious experience is a dependent experimental variable. However, systematic catalog of many such relevant phenomena has not been offered in terms of these theories. In the perceptual retouch theory of thalamocortical interaction, recently developed to become a blend with the dendritic integration theory, consciousness states and contents of consciousness are explained by the same mechanism. This general-purpose mechanism has modulation of the cortical layer-5 pyramidal neurons that represent contents of consciousness as its core. As a surplus, many experimental psychophysical phenomena of conscious perception can be explained by the workings of this mechanism. Historical origins and current views inherent in this theory are presented and reviewed.

Information Integration and Information Storage in Retinotopic and Non-Retinotopic Sensory Memory

The first stage of the Atkinson–Shiffrin model of human memory is a sensory memory (SM). The visual component of the SM was shown to operate within a retinotopic reference frame. However, a retinotopic SM (rSM) is unable to account for vision under natural viewing conditions because, for example, motion information needs to be analyzed across space and time. For this reason, the SM store of the Atkinson–Shiffrin model has been extended to include a non-retinotopic component (nrSM). In this paper, we analyze findings from two experimental paradigms and show drastically different properties of rSM and nrSM. We show that nrSM involves complex processes such as motion-based reference frames and Gestalt grouping, which establish object identities across space and time. We also describe a quantitative model for nrSM and show drastic differences between the spatio-temporal properties of rSM and nrSM. Since the reference-frame of the latter is non-retinotopic and motion-stream based, we suggest that the spatiotemporal properties of the nrSM are in accordance with the spatiotemporal properties of the motion system. Overall, these findings indicate that, unlike the traditional rSM, which is a relatively passive store, nrSM exhibits sophisticated processing properties to manage the complexities of ecological perception.

Perception of Happening: How the Brain Deals with the No-History Problem

In physics, the temporal dimension has units of infinitesimally brief duration. Given this, how is it possible to perceive things, such as motion, music, and vibrotactile stimulation, that involve extension across many units of time? To address this problem, it is proposed that there is what is termed an "information construct of happening" (ICOH), a simultaneous representation of recent, temporally differentiated perceptual information on the millisecond time scale. The main features of the ICOH are (i) time marking, semantic labeling of all information in the ICOH with ordinal temporal information and distance from what is informationally identified as the present moment, (ii) vector informational features that specify kind, direction, and rate of change for every feature in a percept, and (iii) connectives, information relating vector informational features at adjacent temporal locations in the ICOH. The ICOH integrates products of perceptual processing with recent historical information in sensory memory on the subsecond time scale. Perceptual information about happening in informational sensory memory is encoded in semantic form that preserves connected semantic trails of vector and timing information. The basic properties of the ICOH must be supported by a general and widespread timing mechanism that generates ordinal and interval timing information and it is suggested that state-dependent networks may suffice for that purpose. Happening, therefore, is perceived at a moment and is constituted by an information structure of connected recent historical information.

Features integrate along a motion trajectory when object integrity is preserved

Information about a moving object is usually poor at each retinotopic location because photoreceptor activation is short, noisy, and affected by shadows, reflections of other objects, and so on. Integration across the motion trajectory may yield a much better estimate about the objects’ features. Using the sequential metacontrast paradigm, we have shown previously that features, indeed, integrate along a motion trajectory in a long-lasting window of unconscious processing. In the sequential metacontrast paradigm, a percept of two diverging streams is elicited by the presentation of a central line followed by a sequence of flanking pairs of lines. When several lines are spatially offset, the offsets integrate mandatorily for several hundreds of milliseconds along the motion trajectory of the streams. We propose that, within these long-lasting windows, stimuli are first grouped based on Gestalt principles of grouping. These processes establish reference frames that are used to attribute features. Features are then integrated following their respective reference frame. Here using occlusion and bouncing effects, we show that indeed such grouping operations are in place. We found that features integrate only when the spatiotemporal integrity of the object is preserved. Moreover, when several moving objects are present, only features belonging to the same object integrate. Overall, our results show that feature integration is a deliberate strategy of the brain and long-lasting windows of processing can be seen as periods of sense making.

The color phi phenomenon: Not so special, after all?

We show how anomalous time reversal of stimuli and their associated responses can exist in very small connectionist models. These networks are built from dynamical toy model neurons which adhere to a minimal set of biologically plausible properties. The appearance of a "ghost" response, temporally and spatially located in between responses caused by actual stimuli, as in the phi phenomenon, is demonstrated in a similar small network, where it is caused by priming and long-distance feedforward paths. We then demonstrate that the color phi phenomenon can be present in an echo state network, a recurrent neural network, without explicitly training for the presence of the effect, such that it emerges as an artifact of the dynamical processing. Our results suggest that the color phi phenomenon might simply be a feature of the inherent dynamical and nonlinear sensory processing in the brain and in and of itself is not related to consciousness.

Explanatory profiles of models of consciousness - towards a systematic classification

Models of consciousness aim to inspire new experimental protocols and aid interpretation of empirical evidence to reveal the structure of conscious experience. Nevertheless, no current model is univocally accepted on either theoretical or empirical grounds. Moreover, a straightforward comparison is difficult for conceptual reasons. In particular, we argue that different models explicitly or implicitly subscribe to different notions of what constitutes a satisfactory explanation, use different tools in their explanatory endeavours and even aim to explain very different phenomena. We thus present a framework to compare existing models in the field with respect to what we call their 'explanatory profiles'. We focus on the following minimal dimensions: mode of explanation, mechanisms of explanation and target of explanation. We also discuss the empirical consequences of the discussed discrepancies among models. This approach may eventually lead to identifying driving assumptions, theoretical commitments, experimental predictions and a better design of future testing experiments. Finally, our conclusion points to more integrative theoretical research, where axiomatic models may play a critical role in solving current theoretical and experimental contradictions.

Psychedelics and Consciousness: Distinctions, Demarcations, and Opportunities

Psychedelic substances produce unusual and compelling changes in conscious experience that have prompted some to propose that psychedelics may provide unique insights explaining the nature of consciousness. At present, psychedelics, like other current scientific tools and methods, seem unlikely to provide information relevant to the so-called "hard problem of consciousness," which involves explaining how first-person experience can emerge. However, psychedelics bear on multiple "easy problems of consciousness," which involve relations between subjectivity, brain function, and behavior. In this review, we discuss common meanings of the term "consciousness" when used with regard to psychedelics and consider some models of the effects of psychedelics on the brain that have also been associated with explanatory claims about consciousness. We conclude by calling for epistemic humility regarding the potential for psychedelic research to aid in explaining the hard problem of consciousness while pointing to ways in which psychedelics may advance the study of many specific aspects of consciousness.

Time and time again: a multi-scale hierarchical framework for time-consciousness and timing of cognition

Temporality and the feeling of ‘now’ is a fundamental property of consciousness. Different conceptualizations of time-consciousness have argued that both the content of our experiences and the representations of those experiences evolve in time, or neither have temporal extension, or only content does. Accounting for these different positions, we propose a nested hierarchical model of multiple timescales that accounts for findings on timing of cognition and phenomenology of temporal experience. This framework hierarchically combines the three major philosophical positions on time-consciousness (i.e. cinematic, extensional and retentional) and presents a common basis for temporal experience. We detail the properties of these hierarchical levels and speculate how they could coexist mechanistically. We also place several findings on timing and temporal experience at different levels in this hierarchy and show how they can be brought together. Finally, the framework is used to derive novel predictions for both timing of our experiences and time perception. The theoretical framework offers a novel dynamic space that can bring together sub-fields of cognitive science like perception, attention, action and consciousness research in understanding and describing our experiences both in and of time.

The extrafoveal preview paradigm as a measure of predictive, active sampling in visual perception

A key feature of visual processing in humans is the use of saccadic eye movements to look around the environment. Saccades are typically used to bring relevant information, which is glimpsed with extrafoveal vision, into the high-resolution fovea for further processing. With the exception of some unusual circumstances, such as the first fixation when walking into a room, our saccades are mainly guided based on this extrafoveal preview. In contrast, the majority of experimental studies in vision science have investigated "passive" behavioral and neural responses to suddenly appearing and often temporally or spatially unpredictable stimuli. As reviewed here, a growing number of studies have investigated visual processing of objects under more natural viewing conditions in which observers move their eyes to a stationary stimulus, visible previously in extrafoveal vision, during each trial. These studies demonstrate that the extrafoveal preview has a profound influence on visual processing of objects, both for behavior and neural activity. Starting from the preview effect in reading research we follow subsequent developments in vision research more generally and finally argue that taking such evidence seriously leads to a reconceptualization of the nature of human visual perception that incorporates the strong influence of prediction and action on sensory processing. We review theoretical perspectives on visual perception under naturalistic viewing conditions, including theories of active vision, active sensing, and sampling. Although the extrafoveal preview paradigm has already provided useful information about the timing of, and potential mechanisms for, the close interaction of the oculomotor and visual systems while reading and in natural scenes, the findings thus far also raise many new questions for future research.

No evidence for a single oscillator underlying discrete visual percepts

Theories of perception based on discrete sampling posit that visual consciousness is reconstructed based on snapshot-like perceptual moments, as opposed to being updated continuously. According to a model proposed by Schneider (2018), discrete sampling can explain both the flash-lag and the Fröhlich illusion, whereby a lag in the conscious updating of a moving stimulus alters its perceived spatial location in comparison to stationary stimulus. The alpha-band frequency, which is associated with phasic modulation of stimulus detection and the temporal resolution of perception, has been proposed to reflect the duration of perceptual moments. The goal of this study was to determine whether a single oscillator (e.g., alpha) is underlying the duration of perceptual moments, which would predict that the point of subjective equality (PSE) in the flash-lag and Fröhlich illusions are positively correlated across individuals. Although our displays induced robust flash-lag and Fröhlich effects, virtually zero correlation was seen between the PSE in the two illusions, indicating that the illusion magnitudes are unrelated across observers. These findings suggest that, if discrete sampling theory is true, these illusory percepts either rely on different oscillatory frequencies or not on oscillations at all. Alternatively, discrete sampling may not be the mechanism underlying these two motion illusions or our methods were ill-suited to test the theory.

A guideline for linking brain wave findings to the various aspects of discrete perception

Brain waves, determined by electrical and magnetic brain recordings (e.g., EEG and MEG), and fluctuating behavioral responses, determined by response time or accuracy measures, are frequently taken to support discrete perception. For example, it has been proposed that humans experience only one conscious percept per brain wave (e.g., during one alpha cycle). However, the proposed link between brain waves and discrete perception is typically rather vague. More importantly, there are many models and aspects of discrete perception and it is often not apparent in what theoretical framework brain wave findings are interpreted and to what specific aspects of discrete perception they relate. Here, we review different approaches to discrete perception and highlight issues with particular interpretations. We then discuss how certain findings on brain waves may relate to certain aspects of discrete perception. The main purpose of this meta‐contribution is to give a short overview of discrete models of perception and to illustrate the need to make explicit what aspects of discrete theories are addressed by what aspects of brain wave findings.

Research on Discrete Semantics in Continuous Hand Joint Movement Based on Perception and Expression

Continuous movements of the hand contain discrete expressions of meaning, forming a variety of semantic gestures. For example, it is generally considered that the bending of the finger includes three semantic states of bending, half bending, and straightening. However, there is still no research on the number of semantic states that can be conveyed by each movement primitive of the hand, especially the interval of each semantic state and the representative movement angle. To clarify these issues, we conducted experiments of perception and expression. Experiments 1 and 2 focused on perceivable semantic levels and boundaries of different motion primitive units from the perspective of visual semantic perception. Experiment 3 verified and optimized the segmentation results obtained above and further determined the typical motion values of each semantic state. Furthermore, in Experiment 4, the empirical application of the above semantic state segmentation was illustrated by using Leap Motion as an example. We ended up with the discrete gesture semantic expression space both in the real world and Leap Motion Digital World, containing the clearly defined number of semantic states of each hand motion primitive unit and boundaries and typical motion angle values of each state. Construction of this quantitative semantic expression will play a role in guiding and advancing research in the fields of gesture coding, gesture recognition, and gesture design.

New methods for oscillation analyses push new theories of discrete cognition

Classical ways of analyzing neural time series data has led to static views on cognition, in which the cognitive processes are linked to sustained neural activity and interpreted as stationary states. The core analytical focus was on slow power modulations of neural oscillations averaged across many experimental trials. Whereas this custom analytical approach reduces the complexity and increases the signal-to-noise ratio, it may disregard or even remove important aspects of the underlying neural dynamics. Novel analysis methods investigate the instantaneous frequency and phase of neural oscillations and relate them to the precisely controlled timing of brief successive sensory stimuli. This enables to capture how cognitive processes unfold in discrete windows within and across oscillatory cycles. Moreover, several recent studies analyze the oscillatory power modulations on single experimental trials. They suggest that the power modulations are packed into discrete bursts of activity, which occur at different rates and times, and with different durations from trial-to-trial. Here, we review the current work that made use of these methodological advances for neural oscillations. These novel analysis perspectives emphasize that cognitive processes occur in discrete time windows, instead of sustained, stationary states. Evidence for discretization was observed for the entire range of cognitive functions from perception and attention to working memory, goal-directed thought and motor actions, as well as throughout the entire cortical hierarchy and in subcortical regions. These empirical observations create demand for new psychological theories and computational models of cognition in the brain, which integrate its discrete temporal dynamics.

Special Issue: Consciousness science and its theories Time consciousness: the missing link in theories of consciousness

There are plenty of issues to be solved in order for researchers to agree on a neural model of consciousness. Here we emphasize an often under-represented aspect in the debate: time consciousness. Consciousness and the present moment both extend in time. Experience flows through a succession of moments and progresses from future predictions, to present experiences, to past memories. However, a brief review finds that many dominant theories of consciousness only refer to brief, static, and discrete "functional moments" of time. Very few refer to more extended, dynamic, and continuous time, which is associated with conscious experience (cf. the "experienced moment"). This confusion between short and discrete versus long and continuous is, we argue, one of the core issues in theories of consciousness. Given the lack of work dedicated to time consciousness, its study could test novel predictions of rival theories of consciousness. It may be that different theories of consciousness are compatible/complementary if the different aspects of time are taken into account. Or, if it turns out that no existing theory can fully accommodate time consciousness, then perhaps it has something new to add. Regardless of outcome, the crucial step is to make subjective time a central object of study.

The Ouroboros Model, Proposal for Self-Organizing General Cognition Substantiated

The Ouroboros Model has been proposed as a biologically-inspired comprehensive cognitive architecture for general intelligence, comprising natural and artificial manifestations. The approach addresses very diverse fundamental desiderata of research in natural cognition and also artificial intelligence, AI. Here, it is described how the postulated structures have met with supportive evidence over recent years. The associated hypothesized processes could remedy pressing problems plaguing many, and even the most powerful current implementations of AI, including in particular deep neural networks. Some selected recent findings from very different fields are summoned, which illustrate the status and substantiate the proposal.

Windows of Integration Hypothesis Revisited

In the ongoing research of the functions of consciousness, special emphasis has been put on integration of information: the ability to combine different signals into a coherent, unified one. Several theories of consciousness hold that this ability depends on - or at least goes hand in hand with - conscious processing. Yet some empirical findings have suggested otherwise, claiming that integration of information could take place even without awareness. Trying to reconcile this apparent contradiction, the "windows of integration" (WOI) hypothesis claims that conscious access enables signal processing over large integration windows. The hypothesis applies to integration windows defined either temporally, spatially, or semantically. In this review, we explain the hypothesis and re-examine it in light of new studies published since it was suggested. In line with the hypothesis, these studies provide compelling evidence for unconscious integration, but also demonstrate its limits with respect to time, space, and semantic distance. The review further highlights open questions that still need to be pursued to demonstrate the applicability of the WOI hypothesis as a guiding principle for understanding the depth and scope of unconscious processes.

The postdictive effect of choice reflects the modulation of attention on choice

Our conscious perception of the world is not an instantaneous, moment-by-moment construction. Rather, our perception of an event is influenced, over time, by information gained after the event; this is known as a postdictive effect. A recent study reported that this postdictive effect could occur even in choice. The present study sought to test whether the striking postdictive effect of choice reflects the modulation of attention on choice, by directly and systematically manipulating attention in two experiments. Specifically, Experiment 1 revealed that the robust postdictive effect of choice was almost completely eliminated when attentional bias was removed. More important, Experiment 2 demonstrated that the postdictive effect of choice could be modulated by directly manipulating participants’ attention with a spatial cue, in particular, when the cue appeared at short time delays. These results suggest that choice could be considerably postdictively influenced by attention and this effect was most pronounced within a short time window wherein decision making was most likely in progress. The current study not only enables clarification of the mechanism of the newly discovered postdictive effect of choice, but also extends evidence of the modulation of attention on decision making.